Skip to contentSkip to conversation

install: fix flattening of examples directory

PATCH Superseded Bruce Richardson 3 participants 27 messages Nov 10, 2020 – Nov 12, 2020 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 install: fix flattening of examples directory

By installing the examples one-by-one in a loop in the examples meson.build file we effectively flatted out the structure of the examples folder and omitted some common and shared subfolders that were never directly built. Instead, we can remove the loop and just have the whole "examples" folder installed as-is in a single statement, preserving its directory structure, and thereby fixing the build of a number of the examples.

Fixes 2daf565f91b5 ("examples: install as part of ninja install")
---
 examples/meson.build | 7 -------
 meson.build          | 3 +++
 2 files changed, 3 insertions(+), 7 deletions(-)
examples/meson.build +0 -7
index f17995532..46ec80919 100644
@@ -47,13 +47,6 @@ all_examples = [
 	'vm_power_manager/guest_cli',
 	'vmdq', 'vmdq_dcb',
 ]
-# install all example code on install - irrespective of whether the example in
-# question is to be built as part of this build or not.
-foreach ex:all_examples
-	install_subdir(ex,
-			install_dir: get_option('datadir') + '/dpdk/examples',
-			exclude_files: 'meson.build')
-endforeach
 
 if get_option('examples') == ''
 	subdir_done()
meson.build +3 -0
index 61d9a4f5f..45d974cd2 100644
@@ -59,6 +59,9 @@ subdir('doc')
 # build any examples explicitly requested - useful for developers - and
 # install any example code into the appropriate install path
 subdir('examples')
+install_subdir('examples',
+	install_dir: get_option('datadir') + '/dpdk',
+	exclude_files: 'meson.build')
 
 # build kernel modules if enabled
 if get_option('enable_kmods')
Signature · 1 line
2.25.1
#1 lore raw

Using correct address for David. :-(

Quoted 52 lines · Bruce Richardson

By installing the examples one-by-one in a loop in the examples meson.build file we effectively flatted out the structure of the examples folder and omitted some common and shared subfolders that were never directly built. Instead, we can remove the loop and just have the whole "examples" folder installed as-is in a single statement, preserving its directory structure, and thereby fixing the build of a number of the examples.

Fixes 2daf565f91b5 ("examples: install as part of ninja install")
---
 examples/meson.build | 7 -------
 meson.build          | 3 +++
 2 files changed, 3 insertions(+), 7 deletions(-)
examples/meson.build +0 -7
index f17995532..46ec80919 100644
@@ -47,13 +47,6 @@ all_examples = [
 	'vm_power_manager/guest_cli',
 	'vmdq', 'vmdq_dcb',
 ]
-# install all example code on install - irrespective of whether the example in
-# question is to be built as part of this build or not.
-foreach ex:all_examples
-	install_subdir(ex,
-			install_dir: get_option('datadir') + '/dpdk/examples',
-			exclude_files: 'meson.build')
-endforeach
 
 if get_option('examples') == ''
 	subdir_done()
meson.build +3 -0
index 61d9a4f5f..45d974cd2 100644
@@ -59,6 +59,9 @@ subdir('doc')
 # build any examples explicitly requested - useful for developers - and
 # install any example code into the appropriate install path
 subdir('examples')
+install_subdir('examples',
+	install_dir: get_option('datadir') + '/dpdk',
+	exclude_files: 'meson.build')
 
 # build kernel modules if enabled
 if get_option('enable_kmods')
Signature · 1 line
2.25.1
#2 lore raw
Quoted 2 lines · Bruce Richardson

Using correct address for David. :-(

A clone @intel.com would sure be helpful :-).

Signature · 1 line
David Marchand
#3 lore raw
In reply to Bruce Richardson
Quoted 3 lines · Bruce Richardson

By installing the examples one-by-one in a loop in the examples meson.build file we effectively flatted out the structure of the examples

flattened*

Quoted 5 lines

folder and omitted some common and shared subfolders that were never directly built. Instead, we can remove the loop and just have the whole "examples" folder installed as-is in a single statement, preserving its directory structure, and thereby fixing the build of a number of the examples.

It seems wrong to provide all examples without looking at all_examples. We would provide examples that can't link against the available dpdk libraries.

Signature · 1 line
David Marchand
#4 lore raw
Quoted 8 lines · David Marchand
Quoted 3 lines · Bruce Richardson

By installing the examples one-by-one in a loop in the examples meson.build file we effectively flatted out the structure of the examples

flattened*

Quoted 9 lines
Quoted 5 lines

folder and omitted some common and shared subfolders that were never directly built. Instead, we can remove the loop and just have the whole "examples" folder installed as-is in a single statement, preserving its directory structure, and thereby fixing the build of a number of the examples.

It seems wrong to provide all examples without looking at all_examples. We would provide examples that can't link against the available dpdk libraries.

Yes, let me think on this a bit about what is best to do, or how to manage it. I'll try and get a v2 tomorrow.

For older releases with "make" buildsystem, did we provide an installation method for examples, because I can't think of how we handled this fact there?

#5 lore raw

PATCH v2 1/2 examples: fix flattening directory layout on install

In reply to Bruce Richardson

By installing the examples one-by-one in a loop in the examples meson.build file we effectively flattened out the structure of the examples folder and omitted some common and shared subfolders that were never directly built. Instead, we can remove the loop and just have the whole "examples" folder installed as-is in a single statement, preserving its directory structure, and thereby fixing the build of a number of the examples.

Fixes 2daf565f91b5 ("examples: install as part of ninja install")
---
 examples/meson.build | 7 -------
 meson.build          | 3 +++
 2 files changed, 3 insertions(+), 7 deletions(-)
examples/meson.build +0 -7
index f179955325..46ec80919e 100644
@@ -47,13 +47,6 @@ all_examples = [
 	'vm_power_manager/guest_cli',
 	'vmdq', 'vmdq_dcb',
 ]
-# install all example code on install - irrespective of whether the example in
-# question is to be built as part of this build or not.
-foreach ex:all_examples
-	install_subdir(ex,
-			install_dir: get_option('datadir') + '/dpdk/examples',
-			exclude_files: 'meson.build')
-endforeach
 
 if get_option('examples') == ''
 	subdir_done()
meson.build +3 -0
index 61d9a4f5fa..45d974cd2c 100644
@@ -59,6 +59,9 @@ subdir('doc')
 # build any examples explicitly requested - useful for developers - and
 # install any example code into the appropriate install path
 subdir('examples')
+install_subdir('examples',
+	install_dir: get_option('datadir') + '/dpdk',
+	exclude_files: 'meson.build')
 
 # build kernel modules if enabled
 if get_option('enable_kmods')
Signature · 1 line
2.25.1
#6 lore raw

PATCH v2 2/2 examples: skip installing unbuildable examples

Rather than just installing all examples, we can use the build checks to filter out any examples that are missing dependencies or are otherwise unbuildable on the current system.

---
 examples/meson.build | 54 +++++++++++++++++++++++++++-----------------
 meson.build          |  2 ++
 2 files changed, 35 insertions(+), 21 deletions(-)
examples/meson.build +33 -21
index 46ec80919e..9d268c9786 100644
@@ -48,15 +48,11 @@ all_examples = [
 	'vmdq', 'vmdq_dcb',
 ]
 
-if get_option('examples') == ''
-	subdir_done()
-endif
-
 if get_option('examples').to_lower() == 'all'
-	examples = all_examples
+	requested_examples = all_examples
 	allow_skips = true # don't flag an error if we can't build an app
 else
-	examples = get_option('examples').split(',')
+	requested_examples = get_option('examples').split(',')
 	allow_skips = false # error out if we can't build a requested app
 endif
 default_cflags = machine_args
@@ -64,7 +60,9 @@ if cc.has_argument('-Wno-format-truncation')
 	default_cflags += '-Wno-format-truncation'
 endif
 
-foreach example: examples
+# iterate through all examples to see what we need to install
+# only actually build requested ones
+foreach example: all_examples
 	name = example.split('/')[-1]
 	build = true
 	sources = []
@@ -81,22 +79,36 @@ foreach example: examples
 		foreach d:deps
 			var_name = get_option('default_library') + '_rte_' + d
 			if not is_variable(var_name)
-				error('Missing dependency "@0@" for example "@1@"'.format(d, name))
+				message('Missing dependency "@0@" for example "@1@"'.format(d, name))
+				build = false
+			else
+				dep_objs += [get_variable(var_name)]
 			endif
-			dep_objs += [get_variable(var_name)]
 		endforeach
-		if allow_experimental_apis
-			cflags += '-DALLOW_EXPERIMENTAL_API'
+	endif # build
+
+	if not build
+		# exclude based on top-level directory only
+		dir = example.split('/')[0]
+		dpdk_examples_exclude += dir
+		message('Excluding example directory "@0@" from install'.format(dir))
+	endif
+
+	if requested_examples.contains(example)
+		if build
+			if allow_experimental_apis
+				cflags += '-DALLOW_EXPERIMENTAL_API'
+			endif
+			executable('dpdk-' + name, sources,
+				include_directories: includes,
+				link_whole: link_whole_libs,
+				link_args: dpdk_extra_ldflags,
+				c_args: cflags,
+				dependencies: dep_objs)
+		elif not allow_skips
+			error('Cannot build requested example "' + name + '"')
+		else
+			message('Skipping example "' + name + '"')
 		endif
-		executable('dpdk-' + name, sources,
-			include_directories: includes,
-			link_whole: link_whole_libs,
-			link_args: dpdk_extra_ldflags,
-			c_args: cflags,
-			dependencies: dep_objs)
-	elif not allow_skips
-		error('Cannot build requested example "' + name + '"')
-	else
-		message('Skipping example "' + name + '"')
 	endif
 endforeach
meson.build +2 -0
index 45d974cd2c..559a9d2f1b 100644
@@ -58,9 +58,11 @@ subdir('doc')
 
 # build any examples explicitly requested - useful for developers - and
 # install any example code into the appropriate install path
+dpdk_examples_exclude = []
 subdir('examples')
 install_subdir('examples',
 	install_dir: get_option('datadir') + '/dpdk',
+	exclude_directories: dpdk_examples_exclude,
 	exclude_files: 'meson.build')
 
 # build kernel modules if enabled
Signature · 1 line
2.25.1
#7 lore raw

Re: [dpdk-dev] [PATCH v2 2/2] examples: skip installing unbuildable examples

Quoted 10 lines · Bruce Richardson

Rather than just installing all examples, we can use the build checks to filter out any examples that are missing dependencies or are otherwise unbuildable on the current system.

---
 examples/meson.build | 54 +++++++++++++++++++++++++++-----------------
 meson.build          |  2 ++
 2 files changed, 35 insertions(+), 21 deletions(-)

This seems to have broken the windows builds because we now process all the examples meson.build file, even if not asked to build them. l2fwd-keepalive has a mandatory, unconditional dependency on librt, which needs to be made optional.

Based on that, V3 pending...

#8 lore raw

Re: [dpdk-dev] [PATCH v2 2/2] examples: skip installing unbuildable examples

Quoted 12 lines · Bruce Richardson
Quoted 10 lines · Bruce Richardson

Rather than just installing all examples, we can use the build checks to filter out any examples that are missing dependencies or are otherwise unbuildable on the current system.

---
 examples/meson.build | 54 +++++++++++++++++++++++++++-----------------
 meson.build          |  2 ++
 2 files changed, 35 insertions(+), 21 deletions(-)
Quoted 6 lines

This seems to have broken the windows builds because we now process all the examples meson.build file, even if not asked to build them. l2fwd-keepalive has a mandatory, unconditional dependency on librt, which needs to be made optional.

Based on that, V3 pending...

Strange, the v3 patches 2 and 3 linked to the v2 series. As far as I can see, the lab only tested the v3 series with patches 84029 and 30, so the error on Windows is still reported.

Signature · 1 line
David Marchand
#9 lore raw

Re: [dpdk-dev] [PATCH v2 2/2] examples: skip installing unbuildable examples

Quoted 19 lines · David Marchand
Quoted 17 lines · Bruce Richardson
Quoted 10 lines · Bruce Richardson

Rather than just installing all examples, we can use the build checks to filter out any examples that are missing dependencies or are otherwise unbuildable on the current system.

---
 examples/meson.build | 54 +++++++++++++++++++++++++++-----------------
 meson.build          |  2 ++
 2 files changed, 35 insertions(+), 21 deletions(-)

This seems to have broken the windows builds because we now process all the examples meson.build file, even if not asked to build them. l2fwd-keepalive has a mandatory, unconditional dependency on librt, which needs to be made optional.

Quoted 9 lines
Quoted 1 line

Based on that, V3 pending...

Strange, the v3 patches 2 and 3 linked to the v2 series. As far as I can see, the lab only tested the v3 series with patches 84029 and 30, so the error on Windows is still reported.

Signature · 1 line
David Marchand

Yes, I see that in patchwork, and it's a bit strange. The emails seem correctly threaded to me in my mail client though.

I'll resubmit as v4 with a cover letter to see if that allows the CI to run properly.

/Bruce

#10 lore raw

Re: [dpdk-dev] [PATCH v2 2/2] examples: skip installing unbuildable examples

Quoted 9 lines · Bruce Richardson
Quoted 7 lines

Strange, the v3 patches 2 and 3 linked to the v2 series. As far as I can see, the lab only tested the v3 series with patches 84029 and 30, so the error on Windows is still reported.

--
David Marchand

Yes, I see that in patchwork, and it's a bit strange. The emails seem correctly threaded to me in my mail client though.

Yes, it looks correct to me too..

Quoted 2 lines

I'll resubmit as v4 with a cover letter to see if that allows the CI to run properly.

I don't think the cover letter matters.. but let's see how it behaves.

Thanks Bruce.

Signature · 1 line
David Marchand
#11 lore raw

Re: [dpdk-dev] [PATCH v2 2/2] examples: skip installing unbuildable examples

Quoted 12 lines · David Marchand
Quoted 9 lines · Bruce Richardson
Quoted 7 lines

Strange, the v3 patches 2 and 3 linked to the v2 series. As far as I can see, the lab only tested the v3 series with patches 84029 and 30, so the error on Windows is still reported.

--
David Marchand

Yes, I see that in patchwork, and it's a bit strange. The emails seem correctly threaded to me in my mail client though.

Quoted 7 lines

Yes, it looks correct to me too..

Quoted 2 lines

I'll resubmit as v4 with a cover letter to see if that allows the CI to run properly.

I don't think the cover letter matters.. but let's see how it behaves.

It shouldn't matter, I agree, but I'm leaving nothing to chance on this :-)

#12 lore raw

PATCH v3 1/4 examples: fix flattening directory layout on install

In reply to Bruce Richardson

By installing the examples one-by-one in a loop in the examples meson.build file we effectively flattened out the structure of the examples folder and omitted some common and shared subfolders that were never directly built. Instead, we can remove the loop and just have the whole "examples" folder installed as-is in a single statement, preserving its directory structure, and thereby fixing the build of a number of the examples.

Fixes 2daf565f91b5 ("examples: install as part of ninja install")
---
 examples/meson.build | 7 -------
 meson.build          | 3 +++
 2 files changed, 3 insertions(+), 7 deletions(-)
examples/meson.build +0 -7
index f17995532..46ec80919 100644
@@ -47,13 +47,6 @@ all_examples = [
 	'vm_power_manager/guest_cli',
 	'vmdq', 'vmdq_dcb',
 ]
-# install all example code on install - irrespective of whether the example in
-# question is to be built as part of this build or not.
-foreach ex:all_examples
-	install_subdir(ex,
-			install_dir: get_option('datadir') + '/dpdk/examples',
-			exclude_files: 'meson.build')
-endforeach
 
 if get_option('examples') == ''
 	subdir_done()
meson.build +3 -0
index 61d9a4f5f..45d974cd2 100644
@@ -59,6 +59,9 @@ subdir('doc')
 # build any examples explicitly requested - useful for developers - and
 # install any example code into the appropriate install path
 subdir('examples')
+install_subdir('examples',
+	install_dir: get_option('datadir') + '/dpdk',
+	exclude_files: 'meson.build')
 
 # build kernel modules if enabled
 if get_option('enable_kmods')
Signature · 1 line
2.25.1
#13 lore raw

PATCH v3 2/4 examples/l2fwd-keepalive: skip build when no librt

When librt is not present on a system, processing the meson.build file for this example application causes an error. Make the library non-mandatory and just mark the example as unbuildable if it is not present.

Fixes 89f0711f9ddf ("examples: build some samples with meson")
---
 examples/l2fwd-keepalive/meson.build | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
examples/l2fwd-keepalive/meson.build +7 -1
index d678a8ddd..a56d67967 100644
@@ -7,7 +7,13 @@
 # DPDK instance, use 'make'
 
 allow_experimental_apis = true
-ext_deps += cc.find_library('rt')
+librt = cc.find_library('rt', required: false)
+if not librt.found()
+	build = false
+	subdir_done()
+endif
+
+ext_deps += librt
 deps += 'timer'
 sources = files(
 	'main.c', 'shm.c'
Signature · 1 line
2.25.1
#14 lore raw

PATCH v3 3/4 examples: stop processing build file if build is impossible

In reply to Bruce Richardson

Once it has been determined that an example cannot be built, there is little point in continuing to process the meson.build file for that example, so we can use subdir_done() to return to the calling file. This can potentially prevent problems where later statement in the file may cause an error on systems where the app cannot be built, e.g. on Windows or FreeBSD.

---
 examples/distributor/meson.build                     | 3 +++
 examples/ethtool/meson.build                         | 4 ++++
 examples/ioat/meson.build                            | 3 +++
 examples/ip_pipeline/meson.build                     | 4 ++++
 examples/kni/meson.build                             | 4 ++++
 examples/l2fwd-cat/meson.build                       | 4 ++++
 examples/ntb/meson.build                             | 2 ++
 examples/performance-thread/l3fwd-thread/meson.build | 4 ++++
 examples/performance-thread/pthread_shim/meson.build | 4 ++++
 examples/pipeline/meson.build                        | 4 ++++
 examples/vdpa/meson.build                            | 2 ++
 examples/vhost/meson.build                           | 2 ++
 examples/vhost_blk/meson.build                       | 2 ++
 examples/vhost_crypto/meson.build                    | 4 ++++
 14 files changed, 46 insertions(+)
examples/distributor/meson.build +3 -0
index 5244cd4ff..d8dbc235f 100644
@@ -8,6 +8,9 @@
 
 # require the power library
 build = dpdk_conf.has('RTE_LIB_POWER')
+if not build
+	subdir_done()
+endif
 
 allow_experimental_apis = true
 deps += ['distributor', 'power']
examples/ethtool/meson.build +4 -0
index cc8edac3d..4d08bc4c5 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = is_linux
+if not build
+	subdir_done()
+endif
+
 sources = files('lib/rte_ethtool.c',
 	'ethtool-app/ethapp.c',
 	'ethtool-app/main.c')
examples/ioat/meson.build +3 -0
index 6afbaa680..e348196ba 100644
@@ -8,6 +8,9 @@
 
 allow_experimental_apis = true
 build = dpdk_conf.has('RTE_RAW_IOAT')
+if not build
+	subdir_done()
+endif
 
 deps += ['raw_ioat']
 
examples/ip_pipeline/meson.build +4 -0
index 664223c97..945e28b58 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = cc.has_header('sys/epoll.h')
+if not build
+	subdir_done()
+endif
+
 deps += ['pipeline', 'bus_pci']
 allow_experimental_apis = true
 sources = files(
examples/kni/meson.build +4 -0
index 58639b139..e119eebab 100644
@@ -8,6 +8,10 @@
 
 # this app can be built if-and-only-if KNI library is buildable
 build = dpdk_conf.has('RTE_LIB_KNI')
+if not build
+	subdir_done()
+endif
+
 deps += ['kni', 'bus_pci']
 sources = files(
 	'main.c'
examples/l2fwd-cat/meson.build +4 -0
index 2bed18e74..60169bcbd 100644
@@ -8,6 +8,10 @@
 
 pqos = cc.find_library('pqos', required: false)
 build = pqos.found()
+if not build
+	subdir_done()
+endif
+
 ext_deps += pqos
 allow_experimental_apis = true
 cflags += '-I/usr/local/include' # assume pqos lib installed in /usr/local
examples/ntb/meson.build +2 -0
index b0201f68b..02be9fc80 100644
@@ -9,7 +9,9 @@
 allow_experimental_apis = true
 if not is_linux
 	build = false
+	subdir_done()
 endif
+
 deps += 'rawdev'
 cflags += ['-D_FILE_OFFSET_BITS=64']
 sources = files(
examples/performance-thread/l3fwd-thread/meson.build +4 -0
index 99de24be7..4858b201e 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = dpdk_conf.has('RTE_ARCH_X86_64')
+if not build
+	subdir_done()
+endif
+
 deps += ['timer', 'lpm']
 allow_experimental_apis = true
 
examples/performance-thread/pthread_shim/meson.build +4 -0
index 26ef78635..d49979930 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = dpdk_conf.has('RTE_ARCH_X86_64') or dpdk_conf.has('RTE_ARCH_ARM64')
+if not build
+	subdir_done()
+endif
+
 deps += ['timer']
 allow_experimental_apis = true
 
examples/pipeline/meson.build +4 -0
index e47d483de..4f5925d7c 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = cc.has_header('sys/epoll.h')
+if not build
+	subdir_done()
+endif
+
 deps += ['pipeline', 'bus_pci']
 allow_experimental_apis = true
 sources = files(
examples/vdpa/meson.build +2 -0
index 73f129cd9..26f6089c9 100644
@@ -8,7 +8,9 @@
 
 if not is_linux
 	build = false
+	subdir_done()
 endif
+
 deps += 'vhost'
 allow_experimental_apis = true
 sources = files(
examples/vhost/meson.build +2 -0
index 24f1f7131..7e5b9d938 100644
@@ -8,7 +8,9 @@
 
 if not is_linux
 	build = false
+	subdir_done()
 endif
+
 deps += 'vhost'
 allow_experimental_apis = true
 sources = files(
examples/vhost_blk/meson.build +2 -0
index 857367192..354ba0584 100644
@@ -8,10 +8,12 @@
 
 if not is_linux
 	build = false
+	subdir_done()
 endif
 
 if not cc.has_header('linux/virtio_blk.h')
 	build = false
+	subdir_done()
 endif
 
 deps += 'vhost'
examples/vhost_crypto/meson.build +4 -0
index b2c125e2f..403f21098 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = dpdk_conf.has('RTE_LIB_VHOST')
+if not build
+	subdir_done()
+endif
+
 allow_experimental_apis = true
 deps += ['vhost', 'cryptodev']
 sources = files(
Signature · 1 line
2.25.1
#15 lore raw

PATCH v3 4/4 examples: skip installing unbuildable examples

In reply to Bruce Richardson

Rather than just installing all examples, we can use the build checks to filter out any examples that are missing dependencies or are otherwise unbuildable on the current system.

---
 examples/meson.build | 54 +++++++++++++++++++++++++++-----------------
 meson.build          |  2 ++
 2 files changed, 35 insertions(+), 21 deletions(-)
examples/meson.build +33 -21
index 46ec80919..9d268c978 100644
@@ -48,15 +48,11 @@ all_examples = [
 	'vmdq', 'vmdq_dcb',
 ]
 
-if get_option('examples') == ''
-	subdir_done()
-endif
-
 if get_option('examples').to_lower() == 'all'
-	examples = all_examples
+	requested_examples = all_examples
 	allow_skips = true # don't flag an error if we can't build an app
 else
-	examples = get_option('examples').split(',')
+	requested_examples = get_option('examples').split(',')
 	allow_skips = false # error out if we can't build a requested app
 endif
 default_cflags = machine_args
@@ -64,7 +60,9 @@ if cc.has_argument('-Wno-format-truncation')
 	default_cflags += '-Wno-format-truncation'
 endif
 
-foreach example: examples
+# iterate through all examples to see what we need to install
+# only actually build requested ones
+foreach example: all_examples
 	name = example.split('/')[-1]
 	build = true
 	sources = []
@@ -81,22 +79,36 @@ foreach example: examples
 		foreach d:deps
 			var_name = get_option('default_library') + '_rte_' + d
 			if not is_variable(var_name)
-				error('Missing dependency "@0@" for example "@1@"'.format(d, name))
+				message('Missing dependency "@0@" for example "@1@"'.format(d, name))
+				build = false
+			else
+				dep_objs += [get_variable(var_name)]
 			endif
-			dep_objs += [get_variable(var_name)]
 		endforeach
-		if allow_experimental_apis
-			cflags += '-DALLOW_EXPERIMENTAL_API'
+	endif # build
+
+	if not build
+		# exclude based on top-level directory only
+		dir = example.split('/')[0]
+		dpdk_examples_exclude += dir
+		message('Excluding example directory "@0@" from install'.format(dir))
+	endif
+
+	if requested_examples.contains(example)
+		if build
+			if allow_experimental_apis
+				cflags += '-DALLOW_EXPERIMENTAL_API'
+			endif
+			executable('dpdk-' + name, sources,
+				include_directories: includes,
+				link_whole: link_whole_libs,
+				link_args: dpdk_extra_ldflags,
+				c_args: cflags,
+				dependencies: dep_objs)
+		elif not allow_skips
+			error('Cannot build requested example "' + name + '"')
+		else
+			message('Skipping example "' + name + '"')
 		endif
-		executable('dpdk-' + name, sources,
-			include_directories: includes,
-			link_whole: link_whole_libs,
-			link_args: dpdk_extra_ldflags,
-			c_args: cflags,
-			dependencies: dep_objs)
-	elif not allow_skips
-		error('Cannot build requested example "' + name + '"')
-	else
-		message('Skipping example "' + name + '"')
 	endif
 endforeach
meson.build +2 -0
index 45d974cd2..559a9d2f1 100644
@@ -58,9 +58,11 @@ subdir('doc')
 
 # build any examples explicitly requested - useful for developers - and
 # install any example code into the appropriate install path
+dpdk_examples_exclude = []
 subdir('examples')
 install_subdir('examples',
 	install_dir: get_option('datadir') + '/dpdk',
+	exclude_directories: dpdk_examples_exclude,
 	exclude_files: 'meson.build')
 
 # build kernel modules if enabled
Signature · 1 line
2.25.1
#16 lore raw

v4 cover improve examples installation

In reply to Bruce Richardson

This patch improves the installation of the examples when "ninja install" is run. It removes issues with the directory structure being flattened out, and removes from the install list any examples that are not buildable with built DPDK code.

V4: resubmit v3 to retest in CI hopefully with proper patch tracking

V3: fix issues with building on Windows

V2: add support for filtering examples that won't build

Bruce Richardson (4):
examples: fix flattening directory layout on install examples/l2fwd-keepalive: skip build when no librt
examples: stop processing build file if build is impossible
examples: skip installing unbuildable examples

 examples/distributor/meson.build              |  3 +
 examples/ethtool/meson.build                  |  4 ++
 examples/ioat/meson.build                     |  3 +
 examples/ip_pipeline/meson.build              |  4 ++
 examples/kni/meson.build                      |  4 ++
 examples/l2fwd-cat/meson.build                |  4 ++
 examples/l2fwd-keepalive/meson.build          |  8 ++-
 examples/meson.build                          | 61 ++++++++++---------
 examples/ntb/meson.build                      |  2 +
 .../l3fwd-thread/meson.build                  |  4 ++
 .../pthread_shim/meson.build                  |  4 ++
 examples/pipeline/meson.build                 |  4 ++
 examples/vdpa/meson.build                     |  2 +
 examples/vhost/meson.build                    |  2 +
 examples/vhost_blk/meson.build                |  2 +
 examples/vhost_crypto/meson.build             |  4 ++
 meson.build                                   |  5 ++
 17 files changed, 91 insertions(+), 29 deletions(-)

--
2.25.1

#17 lore raw

PATCH v4 1/4 examples: fix flattening directory layout on install

By installing the examples one-by-one in a loop in the examples meson.build file we effectively flattened out the structure of the examples folder and omitted some common and shared subfolders that were never directly built. Instead, we can remove the loop and just have the whole "examples" folder installed as-is in a single statement, preserving its directory structure, and thereby fixing the build of a number of the examples.

Fixes 2daf565f91b5 ("examples: install as part of ninja install")
---
 examples/meson.build | 7 -------
 meson.build          | 3 +++
 2 files changed, 3 insertions(+), 7 deletions(-)
examples/meson.build +0 -7
index f17995532..46ec80919 100644
@@ -47,13 +47,6 @@ all_examples = [
 	'vm_power_manager/guest_cli',
 	'vmdq', 'vmdq_dcb',
 ]
-# install all example code on install - irrespective of whether the example in
-# question is to be built as part of this build or not.
-foreach ex:all_examples
-	install_subdir(ex,
-			install_dir: get_option('datadir') + '/dpdk/examples',
-			exclude_files: 'meson.build')
-endforeach
 
 if get_option('examples') == ''
 	subdir_done()
meson.build +3 -0
index 61d9a4f5f..45d974cd2 100644
@@ -59,6 +59,9 @@ subdir('doc')
 # build any examples explicitly requested - useful for developers - and
 # install any example code into the appropriate install path
 subdir('examples')
+install_subdir('examples',
+	install_dir: get_option('datadir') + '/dpdk',
+	exclude_files: 'meson.build')
 
 # build kernel modules if enabled
 if get_option('enable_kmods')
Signature · 1 line
2.25.1
#18 lore raw

PATCH v4 2/4 examples/l2fwd-keepalive: skip build when no librt

In reply to Bruce Richardson

When librt is not present on a system, processing the meson.build file for this example application causes an error. Make the library non-mandatory and just mark the example as unbuildable if it is not present.

Fixes 89f0711f9ddf ("examples: build some samples with meson")
---
 examples/l2fwd-keepalive/meson.build | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
examples/l2fwd-keepalive/meson.build +7 -1
index d678a8ddd..a56d67967 100644
@@ -7,7 +7,13 @@
 # DPDK instance, use 'make'
 
 allow_experimental_apis = true
-ext_deps += cc.find_library('rt')
+librt = cc.find_library('rt', required: false)
+if not librt.found()
+	build = false
+	subdir_done()
+endif
+
+ext_deps += librt
 deps += 'timer'
 sources = files(
 	'main.c', 'shm.c'
Signature · 1 line
2.25.1
#19 lore raw

PATCH v4 3/4 examples: stop processing build file if build is impossible

In reply to Bruce Richardson

Once it has been determined that an example cannot be built, there is little point in continuing to process the meson.build file for that example, so we can use subdir_done() to return to the calling file. This can potentially prevent problems where later statement in the file may cause an error on systems where the app cannot be built, e.g. on Windows or FreeBSD.

---
 examples/distributor/meson.build                     | 3 +++
 examples/ethtool/meson.build                         | 4 ++++
 examples/ioat/meson.build                            | 3 +++
 examples/ip_pipeline/meson.build                     | 4 ++++
 examples/kni/meson.build                             | 4 ++++
 examples/l2fwd-cat/meson.build                       | 4 ++++
 examples/ntb/meson.build                             | 2 ++
 examples/performance-thread/l3fwd-thread/meson.build | 4 ++++
 examples/performance-thread/pthread_shim/meson.build | 4 ++++
 examples/pipeline/meson.build                        | 4 ++++
 examples/vdpa/meson.build                            | 2 ++
 examples/vhost/meson.build                           | 2 ++
 examples/vhost_blk/meson.build                       | 2 ++
 examples/vhost_crypto/meson.build                    | 4 ++++
 14 files changed, 46 insertions(+)
examples/distributor/meson.build +3 -0
index 5244cd4ff..d8dbc235f 100644
@@ -8,6 +8,9 @@
 
 # require the power library
 build = dpdk_conf.has('RTE_LIB_POWER')
+if not build
+	subdir_done()
+endif
 
 allow_experimental_apis = true
 deps += ['distributor', 'power']
examples/ethtool/meson.build +4 -0
index cc8edac3d..4d08bc4c5 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = is_linux
+if not build
+	subdir_done()
+endif
+
 sources = files('lib/rte_ethtool.c',
 	'ethtool-app/ethapp.c',
 	'ethtool-app/main.c')
examples/ioat/meson.build +3 -0
index 6afbaa680..e348196ba 100644
@@ -8,6 +8,9 @@
 
 allow_experimental_apis = true
 build = dpdk_conf.has('RTE_RAW_IOAT')
+if not build
+	subdir_done()
+endif
 
 deps += ['raw_ioat']
 
examples/ip_pipeline/meson.build +4 -0
index 664223c97..945e28b58 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = cc.has_header('sys/epoll.h')
+if not build
+	subdir_done()
+endif
+
 deps += ['pipeline', 'bus_pci']
 allow_experimental_apis = true
 sources = files(
examples/kni/meson.build +4 -0
index 58639b139..e119eebab 100644
@@ -8,6 +8,10 @@
 
 # this app can be built if-and-only-if KNI library is buildable
 build = dpdk_conf.has('RTE_LIB_KNI')
+if not build
+	subdir_done()
+endif
+
 deps += ['kni', 'bus_pci']
 sources = files(
 	'main.c'
examples/l2fwd-cat/meson.build +4 -0
index 2bed18e74..60169bcbd 100644
@@ -8,6 +8,10 @@
 
 pqos = cc.find_library('pqos', required: false)
 build = pqos.found()
+if not build
+	subdir_done()
+endif
+
 ext_deps += pqos
 allow_experimental_apis = true
 cflags += '-I/usr/local/include' # assume pqos lib installed in /usr/local
examples/ntb/meson.build +2 -0
index b0201f68b..02be9fc80 100644
@@ -9,7 +9,9 @@
 allow_experimental_apis = true
 if not is_linux
 	build = false
+	subdir_done()
 endif
+
 deps += 'rawdev'
 cflags += ['-D_FILE_OFFSET_BITS=64']
 sources = files(
examples/performance-thread/l3fwd-thread/meson.build +4 -0
index 99de24be7..4858b201e 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = dpdk_conf.has('RTE_ARCH_X86_64')
+if not build
+	subdir_done()
+endif
+
 deps += ['timer', 'lpm']
 allow_experimental_apis = true
 
examples/performance-thread/pthread_shim/meson.build +4 -0
index 26ef78635..d49979930 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = dpdk_conf.has('RTE_ARCH_X86_64') or dpdk_conf.has('RTE_ARCH_ARM64')
+if not build
+	subdir_done()
+endif
+
 deps += ['timer']
 allow_experimental_apis = true
 
examples/pipeline/meson.build +4 -0
index e47d483de..4f5925d7c 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = cc.has_header('sys/epoll.h')
+if not build
+	subdir_done()
+endif
+
 deps += ['pipeline', 'bus_pci']
 allow_experimental_apis = true
 sources = files(
examples/vdpa/meson.build +2 -0
index 73f129cd9..26f6089c9 100644
@@ -8,7 +8,9 @@
 
 if not is_linux
 	build = false
+	subdir_done()
 endif
+
 deps += 'vhost'
 allow_experimental_apis = true
 sources = files(
examples/vhost/meson.build +2 -0
index 24f1f7131..7e5b9d938 100644
@@ -8,7 +8,9 @@
 
 if not is_linux
 	build = false
+	subdir_done()
 endif
+
 deps += 'vhost'
 allow_experimental_apis = true
 sources = files(
examples/vhost_blk/meson.build +2 -0
index 857367192..354ba0584 100644
@@ -8,10 +8,12 @@
 
 if not is_linux
 	build = false
+	subdir_done()
 endif
 
 if not cc.has_header('linux/virtio_blk.h')
 	build = false
+	subdir_done()
 endif
 
 deps += 'vhost'
examples/vhost_crypto/meson.build +4 -0
index b2c125e2f..403f21098 100644
@@ -7,6 +7,10 @@
 # DPDK instance, use 'make'
 
 build = dpdk_conf.has('RTE_LIB_VHOST')
+if not build
+	subdir_done()
+endif
+
 allow_experimental_apis = true
 deps += ['vhost', 'cryptodev']
 sources = files(
Signature · 1 line
2.25.1
#20 lore raw

PATCH v4 4/4 examples: skip installing unbuildable examples

In reply to Bruce Richardson

Rather than just installing all examples, we can use the build checks to filter out any examples that are missing dependencies or are otherwise unbuildable on the current system.

---
 examples/meson.build | 54 +++++++++++++++++++++++++++-----------------
 meson.build          |  2 ++
 2 files changed, 35 insertions(+), 21 deletions(-)
examples/meson.build +33 -21
index 46ec80919..9d268c978 100644
@@ -48,15 +48,11 @@ all_examples = [
 	'vmdq', 'vmdq_dcb',
 ]
 
-if get_option('examples') == ''
-	subdir_done()
-endif
-
 if get_option('examples').to_lower() == 'all'
-	examples = all_examples
+	requested_examples = all_examples
 	allow_skips = true # don't flag an error if we can't build an app
 else
-	examples = get_option('examples').split(',')
+	requested_examples = get_option('examples').split(',')
 	allow_skips = false # error out if we can't build a requested app
 endif
 default_cflags = machine_args
@@ -64,7 +60,9 @@ if cc.has_argument('-Wno-format-truncation')
 	default_cflags += '-Wno-format-truncation'
 endif
 
-foreach example: examples
+# iterate through all examples to see what we need to install
+# only actually build requested ones
+foreach example: all_examples
 	name = example.split('/')[-1]
 	build = true
 	sources = []
@@ -81,22 +79,36 @@ foreach example: examples
 		foreach d:deps
 			var_name = get_option('default_library') + '_rte_' + d
 			if not is_variable(var_name)
-				error('Missing dependency "@0@" for example "@1@"'.format(d, name))
+				message('Missing dependency "@0@" for example "@1@"'.format(d, name))
+				build = false
+			else
+				dep_objs += [get_variable(var_name)]
 			endif
-			dep_objs += [get_variable(var_name)]
 		endforeach
-		if allow_experimental_apis
-			cflags += '-DALLOW_EXPERIMENTAL_API'
+	endif # build
+
+	if not build
+		# exclude based on top-level directory only
+		dir = example.split('/')[0]
+		dpdk_examples_exclude += dir
+		message('Excluding example directory "@0@" from install'.format(dir))
+	endif
+
+	if requested_examples.contains(example)
+		if build
+			if allow_experimental_apis
+				cflags += '-DALLOW_EXPERIMENTAL_API'
+			endif
+			executable('dpdk-' + name, sources,
+				include_directories: includes,
+				link_whole: link_whole_libs,
+				link_args: dpdk_extra_ldflags,
+				c_args: cflags,
+				dependencies: dep_objs)
+		elif not allow_skips
+			error('Cannot build requested example "' + name + '"')
+		else
+			message('Skipping example "' + name + '"')
 		endif
-		executable('dpdk-' + name, sources,
-			include_directories: includes,
-			link_whole: link_whole_libs,
-			link_args: dpdk_extra_ldflags,
-			c_args: cflags,
-			dependencies: dep_objs)
-	elif not allow_skips
-		error('Cannot build requested example "' + name + '"')
-	else
-		message('Skipping example "' + name + '"')
 	endif
 endforeach
meson.build +2 -0
index 45d974cd2..559a9d2f1 100644
@@ -58,9 +58,11 @@ subdir('doc')
 
 # build any examples explicitly requested - useful for developers - and
 # install any example code into the appropriate install path
+dpdk_examples_exclude = []
 subdir('examples')
 install_subdir('examples',
 	install_dir: get_option('datadir') + '/dpdk',
+	exclude_directories: dpdk_examples_exclude,
 	exclude_files: 'meson.build')
 
 # build kernel modules if enabled
Signature · 1 line
2.25.1
#21 lore raw

Re: [dpdk-dev] [PATCH v4 4/4] examples: skip installing unbuildable examples

12/11/2020 10:41, Bruce Richardson:

Quoted 5 lines

Rather than just installing all examples, we can use the build checks to filter out any examples that are missing dependencies or are otherwise unbuildable on the current system.

This patch does not say why unbuildable examples should not be installed?

#22 lore raw

Re: [dpdk-dev] [PATCH v4 0/4] improve examples installation

In reply to Bruce Richardson
Quoted 10 lines · Bruce Richardson

This patch improves the installation of the examples when "ninja install" is run. It removes issues with the directory structure being flattened out, and removes from the install list any examples that are not buildable with built DPDK code.

V4: resubmit v3 to retest in CI hopefully with proper patch tracking

V3: fix issues with building on Windows

Quoted 7 lines

V2: add support for filtering examples that won't build

Bruce Richardson (4):
examples: fix flattening directory layout on install examples/l2fwd-keepalive: skip build when no librt
examples: stop processing build file if build is impossible
examples: skip installing unbuildable examples

Thanks Bruce.
This series looks good to me.

My only comment is about vm_power_manager which won't build without the internal librte_power header.
It builds fine with meson, because I guess meson pulls this library directory into the include flags of the example, but it won't build outside.

We can skip it in devtools/test-meson-builds.sh but, on the other hand, I would exclude it from installing until it is fixed (in a future release).

Signature · 1 line
David Marchand
#23 lore raw

Re: [dpdk-dev] [PATCH v4 0/4] improve examples installation

Quoted 29 lines · David Marchand
Quoted 10 lines · Bruce Richardson

This patch improves the installation of the examples when "ninja install" is run. It removes issues with the directory structure being flattened out, and removes from the install list any examples that are not buildable with built DPDK code.

V4: resubmit v3 to retest in CI hopefully with proper patch tracking

V3: fix issues with building on Windows

Quoted 7 lines

V2: add support for filtering examples that won't build

Bruce Richardson (4):
examples: fix flattening directory layout on install examples/l2fwd-keepalive: skip build when no librt
examples: stop processing build file if build is impossible
examples: skip installing unbuildable examples

Thanks Bruce.
This series looks good to me.

My only comment is about vm_power_manager which won't build without the internal librte_power header.
It builds fine with meson, because I guess meson pulls this library directory into the include flags of the example, but it won't build outside.

Quoted 4 lines

We can skip it in devtools/test-meson-builds.sh but, on the other hand, I would exclude it from installing until it is fixed (in a future release).

Yes, good point.
Rather than adding special logic for it, I think we just add it in the initialization of the exclusion array in patch 4, i.e.

-dpdk_examples_exclude = []$
+dpdk_examples_exclude = ['vm_power_manager']

Can this be done on apply, or do you want a v5 for it?

/Bruce

#24 lore raw

Re: [dpdk-dev] [PATCH v4 0/4] improve examples installation

Quoted 17 lines · Bruce Richardson
Quoted 10 lines

My only comment is about vm_power_manager which won't build without the internal librte_power header.
It builds fine with meson, because I guess meson pulls this library directory into the include flags of the example, but it won't build outside.

We can skip it in devtools/test-meson-builds.sh but, on the other hand, I would exclude it from installing until it is fixed (in a future release).

Yes, good point.
Rather than adding special logic for it, I think we just add it in the initialization of the exclusion array in patch 4, i.e.

(quoted diff) +1 -1
-dpdk_examples_exclude = []$
+dpdk_examples_exclude = ['vm_power_manager']
Quoted 1 line

Can this be done on apply, or do you want a v5 for it?

CI is happy with v4.

I tested this additional change, lgtm too.
I would add something in the commitlog to give some context:

"""
vm_power_manager relies on an internal header and can't build outside the dpdk tree.
Hence, it is still built with meson but it is excluded from install examples until it has been fixed.
"""

WDYT?

Signature · 1 line
David Marchand
#25 lore raw

Re: [dpdk-dev] [PATCH v4 0/4] improve examples installation

Quoted 26 lines · David Marchand
Quoted 17 lines · Bruce Richardson
Quoted 10 lines

My only comment is about vm_power_manager which won't build without the internal librte_power header.
It builds fine with meson, because I guess meson pulls this library directory into the include flags of the example, but it won't build outside.

We can skip it in devtools/test-meson-builds.sh but, on the other hand, I would exclude it from installing until it is fixed (in a future release).

Yes, good point.
Rather than adding special logic for it, I think we just add it in the initialization of the exclusion array in patch 4, i.e.

(quoted diff) +1 -1
-dpdk_examples_exclude = []$
+dpdk_examples_exclude = ['vm_power_manager']
Quoted 1 line

Can this be done on apply, or do you want a v5 for it?

CI is happy with v4.

I tested this additional change, lgtm too.
I would add something in the commitlog to give some context:

Quoted 9 lines

"""
vm_power_manager relies on an internal header and can't build outside the dpdk tree.
Hence, it is still built with meson but it is excluded from install examples until it has been fixed.
"""

WDYT?

LGTM

#26 lore raw

Re: [dpdk-dev] [PATCH v4 0/4] improve examples installation

In reply to Bruce Richardson

12/11/2020 10:41, Bruce Richardson:

Quoted 9 lines

This patch improves the installation of the examples when "ninja install" is run. It removes issues with the directory structure being flattened out, and removes from the install list any examples that are not buildable with built DPDK code.

V4: resubmit v3 to retest in CI hopefully with proper patch tracking

V3: fix issues with building on Windows

Quoted 7 lines

V2: add support for filtering examples that won't build

Bruce Richardson (4):
examples: fix flattening directory layout on install examples/l2fwd-keepalive: skip build when no librt
examples: stop processing build file if build is impossible
examples: skip installing unbuildable examples

Applied without last patch, thanks.

After offline discussions with Bruce and David,
the decision is to rework the patch 4 as part of another series that David is preparing.
In short: we keep the option to install all examples and add a new option to install only buildable examples.