Skip to contentSkip to conversation

devtools/test-meson-builds: allow custom set of examples

PATCH Applied Bruce Richardson 3 participants 15 messages Oct 27, 2020 – Nov 12, 2020 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 devtools/test-meson-builds: allow custom set of examples

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

---
 devtools/test-meson-builds.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
devtools/test-meson-builds.sh +5 -1
index a87de635a2..b9cde5b366 100755
@@ -238,11 +238,15 @@ fi
 load_env cc
 pc_file=$(find $DESTDIR -name libdpdk.pc)
 export PKG_CONFIG_PATH=$(dirname $pc_file):$PKG_CONFIG_PATH
+libdir=$(dirname $(find $DESTDIR -name librte_eal.so))
+export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH
+
+examples_to_test=${DPDK_BUILD_TEST_EXAMPLES:-"cmdline helloworld l2fwd l3fwd skeleton timer"}
 
 # if pkg-config defines the necessary flags, test building some examples
 if pkg-config --define-prefix libdpdk >/dev/null 2>&1; then
 	export PKGCONF="pkg-config --define-prefix"
-	for example in cmdline helloworld l2fwd l3fwd skeleton timer; do
+	for example in $examples_to_test; do
 		echo "## Building $example"
 		$MAKE -C $DESTDIR/usr/local/share/dpdk/examples/$example clean shared static
 	done
Signature · 1 line
2.25.1
Acked-by on v1 1/1
#1 lore raw
Quoted 41 lines · Bruce Richardson

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

---
 devtools/test-meson-builds.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
devtools/test-meson-builds.sh +5 -2
index a87de635a2..b9cde5b366 100755
@@ -238,11 +238,15 @@ fi
 load_env cc
 pc_file=$(find $DESTDIR -name libdpdk.pc)
 export PKG_CONFIG_PATH=$(dirname $pc_file):$PKG_CONFIG_PATH
+libdir=$(dirname $(find $DESTDIR -name librte_eal.so))
+export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH
+
+examples_to_test=${DPDK_BUILD_TEST_EXAMPLES:-"cmdline helloworld l2fwd l3fwd skeleton timer"}
 
 # if pkg-config defines the necessary flags, test building some examples
 if pkg-config --define-prefix libdpdk >/dev/null 2>&1; then
        export PKGCONF="pkg-config --define-prefix"
-       for example in cmdline helloworld l2fwd l3fwd skeleton timer; do
+       for example in $examples_to_test; do
                echo "## Building $example"
                $MAKE -C $DESTDIR/usr/local/share/dpdk/examples/$example clean shared static
        done
--
Quoted 2 lines

2.25.1

I had something similar in store for a while, happy to get a better fix.

Caught more issues when compiling examples externally, I will send fixes on kni, l2fwd-crypto, vhost and vhost_blk.

Signature · 1 line
David Marchand
Acked-by Applied
#2 lore raw

09/11/2020 11:01, David Marchand:

Quoted 16 lines
Quoted 14 lines · Bruce Richardson

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 5 lines
Quoted 2 lines

---

I had something similar in store for a while, happy to get a better fix.

The behaviour of pkg-config --define-prefix is kind of magic, but that's unrelated to this patch or this script :)

Applied, thanks

#3 lore raw
In reply to Bruce Richardson

27/10/2020 18:38, Bruce Richardson:

Quoted 12 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

[...]

devtools/test-meson-builds.sh fi · line 242

Quoted 2 lines
devtools/test-meson-builds.sh +2 -0
+libdir=$(dirname $(find $DESTDIR -name librte_eal.so))
+export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

Quoted 1 line

+examples_to_test=${DPDK_BUILD_TEST_EXAMPLES:-"cmdline helloworld l2fwd l3fwd skeleton timer"}

It makes me think that we should rename TEST_MESON_BUILD_VERY_VERBOSE to DPDK_BUILD_TEST_VERY_VERBOSE for consistency.

#4 lore raw
Quoted 13 lines · Thomas Monjalon

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

devtools/test-meson-builds.sh fi · line 242

Quoted 2 lines
devtools/test-meson-builds.sh +2 -0
+libdir=$(dirname $(find $DESTDIR -name librte_eal.so))
+export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

It's only needed for examples that link against drivers directly.

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

Quoted 5 lines
Quoted 1 line

+examples_to_test=${DPDK_BUILD_TEST_EXAMPLES:-"cmdline helloworld l2fwd l3fwd skeleton timer"}

It makes me think that we should rename TEST_MESON_BUILD_VERY_VERBOSE to DPDK_BUILD_TEST_VERY_VERBOSE for consistency.

Sure.

#5 lore raw

09/11/2020 19:02, Bruce Richardson:

Quoted 22 lines · Thomas Monjalon
Quoted 13 lines

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

devtools/test-meson-builds.sh fi · line 242

Quoted 2 lines
devtools/test-meson-builds.sh +2 -0
+libdir=$(dirname $(find $DESTDIR -name librte_eal.so))
+export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

Quoted 7 lines

It's only needed for examples that link against drivers directly.

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

If it's just a matter of ordering,
it would be a better example to fix the ordering in the Makefile, isn't it?

#6 lore raw
Quoted 25 lines · Thomas Monjalon

09/11/2020 19:02, Bruce Richardson:

Quoted 24 lines · Thomas Monjalon
Quoted 13 lines

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

Quoted 2 lines

+libdir=$(dirname $(find $DESTDIR -name librte_eal.so)) +export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

It's only needed for examples that link against drivers directly.

Quoted 9 lines
Quoted 5 lines

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

If it's just a matter of ordering, it would be a better example to fix the ordering in the Makefile, isn't it?

I thought about that, but it seems strange to have the DPDK linker flags appear before the application specific flags. The general style is to have the apps own linker flags apply first, and then the list of dependencies, so that any app linker flags take precedence. The other option is to have two separate LDFLAG variables for the app, one for before pkg-config flags and the other afterwards, but that is an untidy solution.

Therefore, I think it's better to keep the ordering in the examples as-is and just set the library path in the script. It's only a single extra assignment that is necessary because we are installing to a non-standard path using DESTDIR.

/Bruce

#7 lore raw

10/11/2020 11:08, Bruce Richardson:

Quoted 43 lines · Thomas Monjalon
Quoted 25 lines

09/11/2020 19:02, Bruce Richardson:

Quoted 24 lines · Thomas Monjalon
Quoted 13 lines

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

Quoted 2 lines

+libdir=$(dirname $(find $DESTDIR -name librte_eal.so)) +export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

It's only needed for examples that link against drivers directly.

Quoted 9 lines
Quoted 5 lines

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

If it's just a matter of ordering, it would be a better example to fix the ordering in the Makefile, isn't it?

I thought about that, but it seems strange to have the DPDK linker flags appear before the application specific flags. The general style is to have the apps own linker flags apply first, and then the list of dependencies, so that any app linker flags take precedence. The other option is to have two separate LDFLAG variables for the app, one for before pkg-config flags and the other afterwards, but that is an untidy solution.

Quoted 4 lines

Therefore, I think it's better to keep the ordering in the examples as-is and just set the library path in the script. It's only a single extra assignment that is necessary because we are installing to a non-standard path using DESTDIR.

It is OK to add LD_LIBRARY_PATH in test-meson-builds.sh because DPDK is "installed" with a DESTDIR prefix.

But this change is unrelated to test more examples, it is a general fix for examples compilation.
The thing I'm missing, as I said above,
"why libdir is required for some examples, and not for others?" I feel something wrong made linking work where it should not.

#8 lore raw
Quoted 49 lines · Thomas Monjalon

10/11/2020 11:08, Bruce Richardson:

Quoted 43 lines · Thomas Monjalon
Quoted 25 lines

09/11/2020 19:02, Bruce Richardson:

Quoted 24 lines · Thomas Monjalon
Quoted 13 lines

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

Quoted 2 lines

+libdir=$(dirname $(find $DESTDIR -name librte_eal.so)) +export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

It's only needed for examples that link against drivers directly.

Quoted 9 lines
Quoted 5 lines

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

If it's just a matter of ordering, it would be a better example to fix the ordering in the Makefile, isn't it?

I thought about that, but it seems strange to have the DPDK linker flags appear before the application specific flags. The general style is to have the apps own linker flags apply first, and then the list of dependencies, so that any app linker flags take precedence. The other option is to have two separate LDFLAG variables for the app, one for before pkg-config flags and the other afterwards, but that is an untidy solution.

Quoted 4 lines

Therefore, I think it's better to keep the ordering in the examples as-is and just set the library path in the script. It's only a single extra assignment that is necessary because we are installing to a non-standard path using DESTDIR.

Quoted 9 lines

It is OK to add LD_LIBRARY_PATH in test-meson-builds.sh because DPDK is "installed" with a DESTDIR prefix.

But this change is unrelated to test more examples, it is a general fix for examples compilation.
The thing I'm missing, as I said above,
"why libdir is required for some examples, and not for others?" I feel something wrong made linking work where it should not.

As I explained above, libdir is required for examples that link directly against DPDK drivers in shared builds. There are only a few examples that do so, which is why the majority worked fine, and why this was not needed before.

/Bruce

#9 lore raw

10/11/2020 12:34, Bruce Richardson:

Quoted 60 lines · Thomas Monjalon
Quoted 49 lines

10/11/2020 11:08, Bruce Richardson:

Quoted 43 lines · Thomas Monjalon
Quoted 25 lines

09/11/2020 19:02, Bruce Richardson:

Quoted 24 lines · Thomas Monjalon
Quoted 13 lines

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

Quoted 2 lines

+libdir=$(dirname $(find $DESTDIR -name librte_eal.so)) +export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

It's only needed for examples that link against drivers directly.

Quoted 9 lines
Quoted 5 lines

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

If it's just a matter of ordering, it would be a better example to fix the ordering in the Makefile, isn't it?

I thought about that, but it seems strange to have the DPDK linker flags appear before the application specific flags. The general style is to have the apps own linker flags apply first, and then the list of dependencies, so that any app linker flags take precedence. The other option is to have two separate LDFLAG variables for the app, one for before pkg-config flags and the other afterwards, but that is an untidy solution.

Quoted 4 lines

Therefore, I think it's better to keep the ordering in the examples as-is and just set the library path in the script. It's only a single extra assignment that is necessary because we are installing to a non-standard path using DESTDIR.

Quoted 9 lines

It is OK to add LD_LIBRARY_PATH in test-meson-builds.sh because DPDK is "installed" with a DESTDIR prefix.

But this change is unrelated to test more examples, it is a general fix for examples compilation.
The thing I'm missing, as I said above,
"why libdir is required for some examples, and not for others?" I feel something wrong made linking work where it should not.

Quoted 4 lines

As I explained above, libdir is required for examples that link directly against DPDK drivers in shared builds. There are only a few examples that do so, which is why the majority worked fine, and why this was not needed before.

Sorry I don't understand how we link other libraries without LD_LIBRARY_PATH. Where the library path was taken from before this change?

#10 lore raw
Quoted 61 lines · Thomas Monjalon

10/11/2020 12:34, Bruce Richardson:

Quoted 60 lines · Thomas Monjalon
Quoted 58 lines

10/11/2020 11:08, Bruce Richardson:

Quoted 43 lines · Thomas Monjalon
Quoted 25 lines

09/11/2020 19:02, Bruce Richardson:

Quoted 24 lines · Thomas Monjalon
Quoted 13 lines

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

Quoted 2 lines

+libdir=$(dirname $(find $DESTDIR -name librte_eal.so)) +export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

It's only needed for examples that link against drivers directly.

Quoted 9 lines
Quoted 5 lines

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

If it's just a matter of ordering, it would be a better example to fix the ordering in the Makefile, isn't it?

I thought about that, but it seems strange to have the DPDK linker flags appear before the application specific flags. The general style is to have the apps own linker flags apply first, and then the list of dependencies, so that any app linker flags take precedence. The other option is to have two separate LDFLAG variables for the app, one for before pkg-config flags and the other afterwards, but that is an untidy solution.

Quoted 4 lines

Therefore, I think it's better to keep the ordering in the examples as-is and just set the library path in the script. It's only a single extra assignment that is necessary because we are installing to a non-standard path using DESTDIR.

It is OK to add LD_LIBRARY_PATH in test-meson-builds.sh because DPDK is "installed" with a DESTDIR prefix.

But this change is unrelated to test more examples, it is a general fix for examples compilation.
The thing I'm missing, as I said above,
"why libdir is required for some examples, and not for others?" I feel something wrong made linking work where it should not.

Quoted 8 lines
Quoted 4 lines

As I explained above, libdir is required for examples that link directly against DPDK drivers in shared builds. There are only a few examples that do so, which is why the majority worked fine, and why this was not needed before.

Sorry I don't understand how we link other libraries without LD_LIBRARY_PATH. Where the library path was taken from before this change?

pkg-config outputs the library path via -L flag for all the other libraries, which is why they are picked up. Right now the flags passed are (in the case of the bonding example app):

	LDFLAGS += -lrte_net_bond
	...
	LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk)
	...
	build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
		$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)

which resolves to (roughly):
"-lrte_net_bond -L/path/to/dpdk/libs -lrte_ethdev ..."

This means that all the regular DPDK libs are found based off the "-L" flag, but the rte_net_bond and its dependencies are not as they occur before the -L flag.

The alternative, as you suggested previously, is to move the -lrte_net_bond to the end of the list, but that strikes me as less elegant as the LDFLAGS for the app need to be split into two.

A third choice is to modify the makefiles to use pkg-config "--libs-only-L" flag to put in the relevant pkg-config path flag before the rte_net_bond. (However, I have yet to check that all versions of pkg-config support this flag, since we've been bitten in the past of using flags supported by only some versions on some distros!)

	LDFLAGS += $(shell $(PKGCONF) --libs-only-L libdpdk) -lrte_net_bond

Overall, given we are looking at a test script where we install to a non-standard location, putting that location in the LD_LIBRARY_PATH seems the easiest, sane solution. It also most closely matches real-world use, where if one installs DPDK to a novel location, the path to the library folder does need to be set somewhere for the runtime loader to find the libs.

/Bruce

#11 lore raw

10/11/2020 14:19, Bruce Richardson:

Quoted 82 lines · Thomas Monjalon
Quoted 61 lines

10/11/2020 12:34, Bruce Richardson:

Quoted 60 lines · Thomas Monjalon
Quoted 58 lines

10/11/2020 11:08, Bruce Richardson:

Quoted 43 lines · Thomas Monjalon
Quoted 25 lines

09/11/2020 19:02, Bruce Richardson:

Quoted 24 lines · Thomas Monjalon
Quoted 13 lines

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

Quoted 2 lines

+libdir=$(dirname $(find $DESTDIR -name librte_eal.so)) +export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

It's only needed for examples that link against drivers directly.

Quoted 9 lines
Quoted 5 lines

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

If it's just a matter of ordering, it would be a better example to fix the ordering in the Makefile, isn't it?

I thought about that, but it seems strange to have the DPDK linker flags appear before the application specific flags. The general style is to have the apps own linker flags apply first, and then the list of dependencies, so that any app linker flags take precedence. The other option is to have two separate LDFLAG variables for the app, one for before pkg-config flags and the other afterwards, but that is an untidy solution.

Quoted 4 lines

Therefore, I think it's better to keep the ordering in the examples as-is and just set the library path in the script. It's only a single extra assignment that is necessary because we are installing to a non-standard path using DESTDIR.

It is OK to add LD_LIBRARY_PATH in test-meson-builds.sh because DPDK is "installed" with a DESTDIR prefix.

But this change is unrelated to test more examples, it is a general fix for examples compilation.
The thing I'm missing, as I said above,
"why libdir is required for some examples, and not for others?" I feel something wrong made linking work where it should not.

Quoted 8 lines
Quoted 4 lines

As I explained above, libdir is required for examples that link directly against DPDK drivers in shared builds. There are only a few examples that do so, which is why the majority worked fine, and why this was not needed before.

Sorry I don't understand how we link other libraries without LD_LIBRARY_PATH. Where the library path was taken from before this change?

pkg-config outputs the library path via -L flag for all the other libraries, which is why they are picked up. Right now the flags passed are (in the case of the bonding example app):

	LDFLAGS += -lrte_net_bond
	...
	LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk)
	...
	build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
		$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
Quoted 2 lines

which resolves to (roughly):
"-lrte_net_bond -L/path/to/dpdk/libs -lrte_ethdev ..."

This is the part I still don't understand.
How it can resolve to path including DESTDIR?
In my test, it resolves to -L/usr/local/lib so it cannot work.

Quoted 14 lines

This means that all the regular DPDK libs are found based off the "-L" flag, but the rte_net_bond and its dependencies are not as they occur before the -L flag.

The alternative, as you suggested previously, is to move the -lrte_net_bond to the end of the list, but that strikes me as less elegant as the LDFLAGS for the app need to be split into two.

A third choice is to modify the makefiles to use pkg-config "--libs-only-L" flag to put in the relevant pkg-config path flag before the rte_net_bond. (However, I have yet to check that all versions of pkg-config support this flag, since we've been bitten in the past of using flags supported by only some versions on some distros!)

Quoted 8 lines
	LDFLAGS += $(shell $(PKGCONF) --libs-only-L libdpdk) -lrte_net_bond

Overall, given we are looking at a test script where we install to a non-standard location, putting that location in the LD_LIBRARY_PATH seems the easiest, sane solution. It also most closely matches real-world use, where if one installs DPDK to a novel location, the path to the library folder does need to be set somewhere for the runtime loader to find the libs.

I agree with setting LD_LIBRARY_PATH in the test script because of the "non-standard location".
This is not a question.

#12 lore raw
Quoted 83 lines · Thomas Monjalon

10/11/2020 14:19, Bruce Richardson:

Quoted 82 lines · Thomas Monjalon
Quoted 61 lines

10/11/2020 12:34, Bruce Richardson:

Quoted 60 lines · Thomas Monjalon
Quoted 58 lines

10/11/2020 11:08, Bruce Richardson:

Quoted 43 lines · Thomas Monjalon
Quoted 25 lines

09/11/2020 19:02, Bruce Richardson:

Quoted 24 lines · Thomas Monjalon
Quoted 13 lines

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

Quoted 2 lines

+libdir=$(dirname $(find $DESTDIR -name librte_eal.so)) +export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

It's only needed for examples that link against drivers directly.

Quoted 9 lines
Quoted 5 lines

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

If it's just a matter of ordering, it would be a better example to fix the ordering in the Makefile, isn't it?

I thought about that, but it seems strange to have the DPDK linker flags appear before the application specific flags. The general style is to have the apps own linker flags apply first, and then the list of dependencies, so that any app linker flags take precedence. The other option is to have two separate LDFLAG variables for the app, one for before pkg-config flags and the other afterwards, but that is an untidy solution.

Quoted 4 lines

Therefore, I think it's better to keep the ordering in the examples as-is and just set the library path in the script. It's only a single extra assignment that is necessary because we are installing to a non-standard path using DESTDIR.

It is OK to add LD_LIBRARY_PATH in test-meson-builds.sh because DPDK is "installed" with a DESTDIR prefix.

But this change is unrelated to test more examples, it is a general fix for examples compilation.
The thing I'm missing, as I said above,
"why libdir is required for some examples, and not for others?" I feel something wrong made linking work where it should not.

Quoted 8 lines
Quoted 4 lines

As I explained above, libdir is required for examples that link directly against DPDK drivers in shared builds. There are only a few examples that do so, which is why the majority worked fine, and why this was not needed before.

Sorry I don't understand how we link other libraries without LD_LIBRARY_PATH. Where the library path was taken from before this change?

pkg-config outputs the library path via -L flag for all the other libraries, which is why they are picked up. Right now the flags passed are (in the case of the bonding example app):

	LDFLAGS += -lrte_net_bond
	...
	LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk)
	...
	build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
		$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
Quoted 7 lines
Quoted 2 lines

which resolves to (roughly):
"-lrte_net_bond -L/path/to/dpdk/libs -lrte_ethdev ..."

This is the part I still don't understand.
How it can resolve to path including DESTDIR?
In my test, it resolves to -L/usr/local/lib so it cannot work.

It's the --define-prefix parameter we pass to pkg-config that does the trick.

	$ export PKG_CONFIG_LIBDIR=$(pwd)/usr/local/lib/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig
	$ pkg-config --libs-only-L libdpdk
	-L/usr/local/lib -L/usr/lib/x86_64-linux-gnu
	$ pkg-config --define-prefix --libs-only-L libdpdk
	-L/home/bruce/dpdk.org/__BUILDS/build-x86-default/install/usr/local/lib -L/usr/lib/lib/x86_64-linux-gnu

This is done in test-meson-builds right before we call make for each example:

	# if pkg-config defines the necessary flags, test building some examples
	if pkg-config --define-prefix libdpdk >/dev/null 2>&1; then
		export PKGCONF="pkg-config --define-prefix"
		...
	fi
#13 lore raw

10/11/2020 15:19, Bruce Richardson:

Quoted 102 lines · Thomas Monjalon
Quoted 83 lines

10/11/2020 14:19, Bruce Richardson:

Quoted 82 lines · Thomas Monjalon
Quoted 61 lines

10/11/2020 12:34, Bruce Richardson:

Quoted 60 lines · Thomas Monjalon
Quoted 58 lines

10/11/2020 11:08, Bruce Richardson:

Quoted 43 lines · Thomas Monjalon
Quoted 25 lines

09/11/2020 19:02, Bruce Richardson:

Quoted 24 lines · Thomas Monjalon
Quoted 13 lines

27/10/2020 18:38, Bruce Richardson:

Quoted 6 lines

To test the installation process of DPDK using "ninja install" test-meson-builds.sh builds a subset of the examples using "make". To allow more flexibility for people testing, allow the set of examples chosen for this make test to be overridden using variable "DPDK_BUILD_TEST_EXAMPLES" in the environment.

Quoted 6 lines

Since a number of example apps link against drivers directly even for shared builds, we need to ensure that LD_LIBRARY_PATH points to the main DPDK lib folder so any dependencies of those drivers can be found e.g. that the PCI/vdev bus driver .so is found. [All drivers are symlinked from drivers dir back to lib dir on install, so only one dir rather than two is needed in the path.]

Quoted 7 lines

[...]

Quoted 2 lines

+libdir=$(dirname $(find $DESTDIR -name librte_eal.so)) +export LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH

I don't get why libdir is required for some examples, and not for others? The pkg-config file is not enough?

It's only needed for examples that link against drivers directly.

Quoted 9 lines
Quoted 5 lines

I believe it's needed in those cases, because app linker flags (including e.g. -lrte_pmd_bond) occur before the pkg-config flags, which means that the linker at that point does not have the path to find the dependencies of the driver. [In a normal build, this wouldn't be necessary because the library directory would be a standard path]

If it's just a matter of ordering, it would be a better example to fix the ordering in the Makefile, isn't it?

I thought about that, but it seems strange to have the DPDK linker flags appear before the application specific flags. The general style is to have the apps own linker flags apply first, and then the list of dependencies, so that any app linker flags take precedence. The other option is to have two separate LDFLAG variables for the app, one for before pkg-config flags and the other afterwards, but that is an untidy solution.

Quoted 4 lines

Therefore, I think it's better to keep the ordering in the examples as-is and just set the library path in the script. It's only a single extra assignment that is necessary because we are installing to a non-standard path using DESTDIR.

It is OK to add LD_LIBRARY_PATH in test-meson-builds.sh because DPDK is "installed" with a DESTDIR prefix.

But this change is unrelated to test more examples, it is a general fix for examples compilation.
The thing I'm missing, as I said above,
"why libdir is required for some examples, and not for others?" I feel something wrong made linking work where it should not.

Quoted 8 lines
Quoted 4 lines

As I explained above, libdir is required for examples that link directly against DPDK drivers in shared builds. There are only a few examples that do so, which is why the majority worked fine, and why this was not needed before.

Sorry I don't understand how we link other libraries without LD_LIBRARY_PATH. Where the library path was taken from before this change?

pkg-config outputs the library path via -L flag for all the other libraries, which is why they are picked up. Right now the flags passed are (in the case of the bonding example app):

	LDFLAGS += -lrte_net_bond
	...
	LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk)
	...
	build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
		$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
Quoted 7 lines
Quoted 2 lines

which resolves to (roughly):
"-lrte_net_bond -L/path/to/dpdk/libs -lrte_ethdev ..."

This is the part I still don't understand.
How it can resolve to path including DESTDIR?
In my test, it resolves to -L/usr/local/lib so it cannot work.

It's the --define-prefix parameter we pass to pkg-config that does the trick.

	$ export PKG_CONFIG_LIBDIR=$(pwd)/usr/local/lib/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig
	$ pkg-config --libs-only-L libdpdk
	-L/usr/local/lib -L/usr/lib/x86_64-linux-gnu
	$ pkg-config --define-prefix --libs-only-L libdpdk
	-L/home/bruce/dpdk.org/__BUILDS/build-x86-default/install/usr/local/lib -L/usr/lib/lib/x86_64-linux-gnu
Quoted 8 lines

This is done in test-meson-builds right before we call make for each example:

	# if pkg-config defines the necessary flags, test building some examples
	if pkg-config --define-prefix libdpdk >/dev/null 2>&1; then
		export PKGCONF="pkg-config --define-prefix"
		...
	fi

Yes this is what David suggested to me,
but it doesn't work well for me (while test-meson-builds.sh works):

PKG_CONFIG_PATH=../dpdk-build/build-x86-default/meson-private \
pkg-config --define-prefix --libs-only-L libdpdk -L/usr/local/lib

Oh! I am testing on the build directory instead of installed one.

PKG_CONFIG_PATH=../dpdk-build/build-x86-default/install/usr/local/lib/pkgconfig \
pkg-config --define-prefix --libs-only-L libdpdk -L../dpdk-build/build-x86-default/install/usr/local/lib

Good :)
Now THE question: what is the difference between these two .pc files?

#14 lore raw
Quoted 4 lines · Thomas Monjalon

PKG_CONFIG_PATH=../dpdk-build/build-x86-default/meson-private \
pkg-config --define-prefix --libs-only-L libdpdk -L/usr/local/lib

Quoted 8 lines

Oh! I am testing on the build directory instead of installed one.

PKG_CONFIG_PATH=../dpdk-build/build-x86-default/install/usr/local/lib/pkgconfig \
pkg-config --define-prefix --libs-only-L libdpdk -L../dpdk-build/build-x86-default/install/usr/local/lib

Good :)
Now THE question: what is the difference between these two .pc files?

I moved libdpdk{,-libs}.pc files in different folders.
pkg-config seems to check the path to the .pc files to decide what to return. There seems to be a combination of having $prefix in the path and /pkgconfig as the last directory.

Signature · 1 line
David Marchand