Skip to contentSkip to conversation

RFC: Support disabling DPDK drivers in meson builds

PATCH Applied Bruce Richardson 5 participants 13 messages Sep 25, 2019 – Jan 7, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover RFC: Support disabling DPDK drivers in meson builds

Following on from the discussion about meson at userspace 2019, here is a patchset which allows a user to disable drivers in a DPDK build. This set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa also disables net/dpaa and event/dpaa with suitable reasons provided in the output summary at the end of the meson configuration phase.

A number of changes are made to the lib/meson.build file too, in order to keep the general logic flow between drivers/meson.build and lib/meson.build as similar as possible.

Example of use:

meson -Ddisable_drivers=mempool/dpaa,net/i40e build

Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson

 drivers/meson.build | 45 +++++++++++++++++++++++++++------------------
 lib/meson.build     | 23 +++++++++++++----------
 meson_options.txt   |  2 ++
 3 files changed, 42 insertions(+), 28 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

PATCH 1/3 build: align variable names between drivers and libs

The variable names in the library and drivers meson.build files are slighty different with "static_deps" in one and "static_objs" in the other. Rename to use "static_deps" in both for consistency.

---
 drivers/meson.build | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)
drivers/meson.build +11 -11
index 2ed2e9541..b54601697 100644
@@ -79,18 +79,18 @@ foreach class:dpdk_driver_classes
 			endif
 
 			# get dependency objs from strings
-			shared_objs = []
-			static_objs = []
+			shared_deps = []
+			static_deps = []
 			foreach d:deps
 				if not is_variable('shared_rte_' + d)
 					error('Missing dependency ' + d +
 						' for driver ' + lib_name)
 				endif
-				shared_objs += [get_variable('shared_rte_' + d)]
-				static_objs += [get_variable('static_rte_' + d)]
+				shared_deps += [get_variable('shared_rte_' + d)]
+				static_deps += [get_variable('static_rte_' + d)]
 			endforeach
-			shared_objs += ext_deps
-			static_objs += ext_deps
+			shared_deps += ext_deps
+			static_deps += ext_deps
 			dpdk_extra_ldflags += pkgconfig_extra_libs
 
 			# generate pmdinfo sources by building a temporary
@@ -101,7 +101,7 @@ foreach class:dpdk_driver_classes
 			tmp_lib = static_library('tmp_' + lib_name,
 					sources,
 					include_directories: includes,
-					dependencies: static_objs,
+					dependencies: static_deps,
 					c_args: cflags)
 			objs += tmp_lib.extract_all_objects()
 			sources = custom_target(out_filename,
@@ -123,7 +123,7 @@ foreach class:dpdk_driver_classes
 				sources,
 				objects: objs,
 				include_directories: includes,
-				dependencies: static_objs,
+				dependencies: static_deps,
 				c_args: cflags,
 				install: true)
 
@@ -135,7 +135,7 @@ foreach class:dpdk_driver_classes
 				sources,
 				objects: objs,
 				include_directories: includes,
-				dependencies: shared_objs,
+				dependencies: shared_deps,
 				c_args: cflags,
 				link_args: '-Wl,--version-script=' + version_map,
 				link_depends: version_map,
@@ -148,10 +148,10 @@ foreach class:dpdk_driver_classes
 			# testpmd or other built-in apps can find it if necessary
 			shared_dep = declare_dependency(link_with: shared_lib,
 					include_directories: includes,
-					dependencies: shared_objs)
+					dependencies: shared_deps)
 			static_dep = declare_dependency(link_with: static_lib,
 					include_directories: includes,
-					dependencies: static_objs)
+					dependencies: static_deps)
 
 			dpdk_drivers += static_lib
 
Signature · 1 line
2.21.0
#2 lore raw

PATCH 2/3 build: process dependencies before main build check

In reply to Bruce Richardson

If we want to add support for turning off components because of missing dependencies, then we need to check for those dependencies before we make a determination as to whether a component should be built or not, assuming that the component says it should be built.

---
 drivers/meson.build | 27 ++++++++++++++-------------
 lib/meson.build     | 23 +++++++++++++----------
 2 files changed, 27 insertions(+), 23 deletions(-)
drivers/meson.build +14 -13
index b54601697..ee5db4157 100644
@@ -59,6 +59,20 @@ foreach class:dpdk_driver_classes
 		# pull in driver directory which should assign to each of the above
 		subdir(drv_path)
 
+		if build
+			# get dependency objs from strings
+			shared_deps = ext_deps
+			static_deps = ext_deps
+			foreach d:deps
+				if not is_variable('shared_rte_' + d)
+					error('Missing internal dependency "@0@" for @1@ [@2@]'
+							.format(d, name, 'drivers/' + drv_path))
+				endif
+				shared_deps += [get_variable('shared_rte_' + d)]
+				static_deps += [get_variable('static_rte_' + d)]
+			endforeach
+		endif
+
 		if not build
 			# some driver directories are placeholders which
 			# are never built, so we allow suppression of the
@@ -78,19 +92,6 @@ foreach class:dpdk_driver_classes
 				cflags += '-DALLOW_EXPERIMENTAL_API'
 			endif
 
-			# get dependency objs from strings
-			shared_deps = []
-			static_deps = []
-			foreach d:deps
-				if not is_variable('shared_rte_' + d)
-					error('Missing dependency ' + d +
-						' for driver ' + lib_name)
-				endif
-				shared_deps += [get_variable('shared_rte_' + d)]
-				static_deps += [get_variable('static_rte_' + d)]
-			endforeach
-			shared_deps += ext_deps
-			static_deps += ext_deps
 			dpdk_extra_ldflags += pkgconfig_extra_libs
 
 			# generate pmdinfo sources by building a temporary
lib/meson.build +13 -10
index e5ff83893..fde2f5a02 100644
@@ -66,6 +66,19 @@ foreach l:libraries
 	dir_name = 'librte_' + l
 	subdir(dir_name)
 
+	if build
+		shared_deps = ext_deps
+		static_deps = ext_deps
+		foreach d:deps
+			if not is_variable('shared_rte_' + d)
+				error('Missing internal dependency "@0@" for @1@ [@2@]'
+						.format(d, name, 'lib/' + dir_name))
+			endif
+			shared_deps += [get_variable('shared_rte_' + d)]
+			static_deps += [get_variable('static_rte_' + d)]
+		endforeach
+	endif
+
 	if not build
 		dpdk_libs_disabled += name
 		set_variable(name.underscorify() + '_disable_reason', reason)
@@ -82,16 +95,6 @@ foreach l:libraries
 			shared_dep = declare_dependency(include_directories: includes)
 			static_dep = shared_dep
 		else
-			shared_deps = ext_deps
-			static_deps = ext_deps
-			foreach d:deps
-				if not is_variable('shared_rte_' + d)
-					error('Missing dependency ' + d +
-						' for library ' + libname)
-				endif
-				shared_deps += [get_variable('shared_rte_' + d)]
-				static_deps += [get_variable('static_rte_' + d)]
-			endforeach
 
 			if allow_experimental_apis
 				cflags += '-DALLOW_EXPERIMENTAL_API'
Signature · 1 line
2.21.0
#3 lore raw

PATCH 3/3 build: support disabling drivers with meson

In reply to Bruce Richardson

Add support for a new build option to turn off certain drivers. Any other drivers which depend on the one being disabled will also be disabled with a suitable debug message.

---
 drivers/meson.build | 16 ++++++++++++----
 meson_options.txt   |  2 ++
 2 files changed, 14 insertions(+), 4 deletions(-)
drivers/meson.build +12 -4
index ee5db4157..e7a696d52 100644
@@ -16,6 +16,8 @@ dpdk_driver_classes = ['common',
 	       'baseband', # depends on common and bus.
 	       'raw']     # depends on common, bus, mempool, net and event.
 
+disabled_drivers = get_option('disable_drivers').split(',')
+
 default_cflags = machine_args
 if cc.has_argument('-Wno-format-truncation')
 	default_cflags += '-Wno-format-truncation'
@@ -59,17 +61,23 @@ foreach class:dpdk_driver_classes
 		# pull in driver directory which should assign to each of the above
 		subdir(drv_path)
 
-		if build
+		if drv_path in disabled_drivers
+			build = false
+			reason = 'Explicitly disabled via build config'
+		elif build
 			# get dependency objs from strings
 			shared_deps = ext_deps
 			static_deps = ext_deps
 			foreach d:deps
 				if not is_variable('shared_rte_' + d)
-					error('Missing internal dependency "@0@" for @1@ [@2@]'
+					build = false
+					reason = 'Missing internal dependency, "@0@"'.format(d)
+					message('Disabling @1@ [@2@]: missing internal dependency "@0@"'
 							.format(d, name, 'drivers/' + drv_path))
+				else
+					shared_deps += [get_variable('shared_rte_' + d)]
+					static_deps += [get_variable('static_rte_' + d)]
 				endif
-				shared_deps += [get_variable('shared_rte_' + d)]
-				static_deps += [get_variable('static_rte_' + d)]
 			endforeach
 		endif
 
meson_options.txt +2 -0
index 448f3e63d..89650b0e9 100644
@@ -2,6 +2,8 @@
 
 option('allow_invalid_socket_id', type: 'boolean', value: false,
 	description: 'allow out-of-range NUMA socket id\'s for platforms that don\'t report the value correctly')
+option('disable_drivers', type: 'string', value: '',
+	description: 'Comma-separated list of drivers to explicitly disable.')
 option('drivers_install_subdir', type: 'string', value: 'dpdk/pmds-<VERSION>',
 	description: 'Subdirectory of libdir where to install PMDs. Defaults to using a versioned subdirectory.')
 option('enable_docs', type: 'boolean', value: false,
Signature · 1 line
2.21.0
#4 lore raw

RE: [PATCH 3/3] build: support disabling drivers with meson

Quoted 10 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 25 September 2019 16.56

Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also be disabled with a
suitable debug message.

Hi Bruce,

I was trying something along the lines of:

meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e

But it seems that enable_drivers does not override disable_drivers. Ninja says:

[...]

net/hinic:      not in enabled drivers build config
net/hns3:       not in enabled drivers build config
net/i40e:       explicitly disabled via build config
net/iavf:       not in enabled drivers build config

[...]

Could you please fix that?

Also, since all sorts of exotic libraries are continuously being added to DPDK, it would be nice to have a similar Meson option for libs, i.e. enable_libs to override disable_libs=*.

Med venlig hilsen / Kind regards,
-Morten Brørup

#5 lore raw

Re: [PATCH 3/3] build: support disabling drivers with meson

06/01/2022 09:23, Morten Brørup:

Quoted 11 lines
Quoted 10 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 25 September 2019 16.56

Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also be disabled with a
suitable debug message.

Quoted 7 lines

Hi Bruce,

I was trying something along the lines of:

meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e

But it seems that enable_drivers does not override disable_drivers. Ninja says:

It is the opposite logic. "disable" overrides "enable":

if not enable_drivers.contains(drv_path)
build = false
reason = 'not in enabled drivers build config'
elif disable_drivers.contains(drv_path)
if always_enable.contains(drv_path)
message('Driver @0@ cannot be disabled, not disabling.'.format(drv_path))
else
build = false
reason = 'explicitly disabled via build config'
endif
endif

In this case, you don't need to disable everything because of this:

if enable_drivers.length() == 0

    enable_drivers = run_command(list_dir_globs, '*/*', check: true).stdout().split()

endif

#6 lore raw

RE: [PATCH 3/3] build: support disabling drivers with meson

Quoted 40 lines

From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, 6 January 2022 10.49

06/01/2022 09:23, Morten Brørup:

Quoted 6 lines
Quoted 6 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 25 September 2019 16.56

Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also be

disabled

Quoted 5 lines
Quoted 4 lines

with a
suitable debug message.

Quoted 7 lines

Hi Bruce,

I was trying something along the lines of:

meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e

But it seems that enable_drivers does not override disable_drivers.

Ninja says:

It is the opposite logic. "disable" overrides "enable":

if not enable_drivers.contains(drv_path)
build = false
reason = 'not in enabled drivers build config'
elif disable_drivers.contains(drv_path)
if always_enable.contains(drv_path)
message('Driver @0@ cannot be disabled, not
disabling.'.format(drv_path))
else
build = false
reason = 'explicitly disabled via build config'
endif
endif

Quoted 7 lines

In this case, you don't need to disable everything because of this:

if enable_drivers.length() == 0
enable_drivers = run_command(list_dir_globs, '*/*', check:
true).stdout().split()
endif

Thank you, @Thomas! That solved the drivers part of my question.

@Bruce:

If you have too much time on your hands, and want to experiment with set theory on Meson, you could implement a "longest prefix match" for this, making the narrower selection take precedence. Just kidding! Enabling/disabling drivers as Thomas described works great.

Still, a similar Meson option for selecting libraries would be nice.

#7 lore raw

Re: [PATCH 3/3] build: support disabling drivers with meson

Quoted 48 lines · Morten Brørup
Quoted 40 lines

From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, 6 January 2022 10.49

06/01/2022 09:23, Morten Brørup:

Quoted 6 lines
Quoted 6 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 25 September 2019 16.56

Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also be

disabled

Quoted 5 lines
Quoted 4 lines

with a
suitable debug message.

Quoted 7 lines

Hi Bruce,

I was trying something along the lines of:

meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e

But it seems that enable_drivers does not override disable_drivers.

Ninja says:

It is the opposite logic. "disable" overrides "enable":

if not enable_drivers.contains(drv_path)
build = false
reason = 'not in enabled drivers build config'
elif disable_drivers.contains(drv_path)
if always_enable.contains(drv_path)
message('Driver @0@ cannot be disabled, not
disabling.'.format(drv_path))
else
build = false
reason = 'explicitly disabled via build config'
endif
endif

Quoted 7 lines

In this case, you don't need to disable everything because of this:

if enable_drivers.length() == 0
enable_drivers = run_command(list_dir_globs, '*/*', check:
true).stdout().split()
endif

Quoted 8 lines

Thank you, @Thomas! That solved the drivers part of my question.

@Bruce:

If you have too much time on your hands, and want to experiment with set theory on Meson, you could implement a "longest prefix match" for this, making the narrower selection take precedence. Just kidding! Enabling/disabling drivers as Thomas described works great.

Still, a similar Meson option for selecting libraries would be nice.

Yes, I understand. I have a few concerns about implementing this just now though. David previously posted a patch[1] to do what you are looking for, and please see the discussion in that thread [2].

[1] http://inbox.dpdk.org/dev/[email protected]/
[2] http://inbox.dpdk.org/dev/[email protected]/

#8 lore raw
In reply to Bruce Richardson
Quoted 25 lines · Bruce Richardson

Following on from the discussion about meson at userspace 2019, here is
a patchset which allows a user to disable drivers in a DPDK build. This
set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa
also disables net/dpaa and event/dpaa with suitable reasons provided in
the output summary at the end of the meson configuration phase.

A number of changes are made to the lib/meson.build file too, in order
to keep the general logic flow between drivers/meson.build and lib/meson.build as similar as possible.

Example of use:

meson -Ddisable_drivers=mempool/dpaa,net/i40e build

Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson

Quoted 5 lines
 drivers/meson.build | 45 +++++++++++++++++++++++++++--------------
----
 lib/meson.build     | 23 +++++++++++++----------
 meson_options.txt   |  2 ++
 3 files changed, 42 insertions(+), 28 deletions(-)
Series-grumbly-acked-by Luca Boccassi <[email protected]>
Signature · 2 lines
Kind regards,
Luca Boccassi
#9 lore raw
Quoted 26 lines · Luca Boccassi
Quoted 25 lines · Bruce Richardson

Following on from the discussion about meson at userspace 2019, here is
a patchset which allows a user to disable drivers in a DPDK build. This
set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa
also disables net/dpaa and event/dpaa with suitable reasons provided in
the output summary at the end of the meson configuration phase.

A number of changes are made to the lib/meson.build file too, in order
to keep the general logic flow between drivers/meson.build and lib/meson.build as similar as possible.

Example of use:

meson -Ddisable_drivers=mempool/dpaa,net/i40e build

Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson

Quoted 8 lines
Quoted 5 lines
 drivers/meson.build | 45 +++++++++++++++++++++++++++--------------
----
 lib/meson.build     | 23 +++++++++++++----------
 meson_options.txt   |  2 ++
 3 files changed, 42 insertions(+), 28 deletions(-)
Series-grumbly-acked-by Luca Boccassi <[email protected]>

Not exactly doing my happy-dance here myself about this set, but I think it's something those building their own DPDK will appreciate.

#10 lore raw
Quoted 41 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Bruce Richardson
Sent: Thursday, September 26, 2019 10:34 AM
To: Luca Boccassi
Cc: [email protected]; [email protected]; Morten Brørup; [email protected]
Subject: Re: [dpdk-dev] [PATCH 0/3] RFC: Support disabling DPDK drivers in meson builds

Quoted 32 lines · Luca Boccassi
Quoted 30 lines · Bruce Richardson

Following on from the discussion about meson at userspace 2019, here is
a patchset which allows a user to disable drivers in a DPDK build. This
set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa
also disables net/dpaa and event/dpaa with suitable reasons provided in
the output summary at the end of the meson configuration phase.

A number of changes are made to the lib/meson.build file too, in order
to keep the general logic flow between drivers/meson.build and lib/meson.build as similar as possible.

Example of use:

meson -Ddisable_drivers=mempool/dpaa,net/i40e build

Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson

 drivers/meson.build | 45 +++++++++++++++++++++++++++--------------
----
 lib/meson.build     | 23 +++++++++++++----------
 meson_options.txt   |  2 ++
 3 files changed, 42 insertions(+), 28 deletions(-)
Quoted 4 lines
Quoted 2 lines
Series-grumbly-acked-by Luca Boccassi <[email protected]>

Not exactly doing my happy-dance here myself about this set, but I think it's something those building their own DPDK will appreciate.

Yes, we will.

Med venlig hilsen / kind regards
- Morten Brørup

#11 lore raw
In reply to Luca Boccassi

25/09/2019 20:01, Luca Boccassi:

Quoted 7 lines · Bruce Richardson
Quoted 4 lines

Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson

Series-grumbly-acked-by Luca Boccassi <[email protected]>

Applied, thanks for implementing a community request

Acked-by on the v1 series
#12 lore raw
In reply to Bruce Richardson
Quoted 20 lines · Bruce Richardson

Following on from the discussion about meson at userspace 2019, here is a patchset which allows a user to disable drivers in a DPDK build. This set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa also disables net/dpaa and event/dpaa with suitable reasons provided in the output summary at the end of the meson configuration phase.

A number of changes are made to the lib/meson.build file too, in order to keep the general logic flow between drivers/meson.build and lib/meson.build as similar as possible.

Example of use:

meson -Ddisable_drivers=mempool/dpaa,net/i40e build

Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson

Quoted 5 lines
 drivers/meson.build | 45 +++++++++++++++++++++++++++------------------
 lib/meson.build     | 23 +++++++++++++----------
 meson_options.txt   |  2 ++
 3 files changed, 42 insertions(+), 28 deletions(-)

Thanks,
Maxime