RFC: Support disabling DPDK drivers in meson builds
Subsystems (2)
cover RFC: Support disabling DPDK drivers in meson builds
Following on from the discussion about meson at userspace 2019, here is a patchset which allows a user to disable drivers in a DPDK build. This set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa also disables net/dpaa and event/dpaa with suitable reasons provided in the output summary at the end of the meson configuration phase.
A number of changes are made to the lib/meson.build file too, in order to keep the general logic flow between drivers/meson.build and lib/meson.build as similar as possible.
Example of use:
meson -Ddisable_drivers=mempool/dpaa,net/i40e build
Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson
drivers/meson.build | 45 +++++++++++++++++++++++++++------------------ lib/meson.build | 23 +++++++++++++---------- meson_options.txt | 2 ++ 3 files changed, 42 insertions(+), 28 deletions(-)
Signature · 1 line
2.21.0
PATCH 1/3 build: align variable names between drivers and libs
The variable names in the library and drivers meson.build files are slighty different with "static_deps" in one and "static_objs" in the other. Rename to use "static_deps" in both for consistency.
--- drivers/meson.build | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-)
drivers/meson.build +11 -11
@@ -79,18 +79,18 @@ foreach class:dpdk_driver_classes endif # get dependency objs from strings - shared_objs = [] - static_objs = [] + shared_deps = [] + static_deps = [] foreach d:deps if not is_variable('shared_rte_' + d) error('Missing dependency ' + d + ' for driver ' + lib_name) endif - shared_objs += [get_variable('shared_rte_' + d)] - static_objs += [get_variable('static_rte_' + d)] + shared_deps += [get_variable('shared_rte_' + d)] + static_deps += [get_variable('static_rte_' + d)] endforeach - shared_objs += ext_deps - static_objs += ext_deps + shared_deps += ext_deps + static_deps += ext_deps dpdk_extra_ldflags += pkgconfig_extra_libs # generate pmdinfo sources by building a temporary
@@ -101,7 +101,7 @@ foreach class:dpdk_driver_classes tmp_lib = static_library('tmp_' + lib_name, sources, include_directories: includes, - dependencies: static_objs, + dependencies: static_deps, c_args: cflags) objs += tmp_lib.extract_all_objects() sources = custom_target(out_filename,
@@ -123,7 +123,7 @@ foreach class:dpdk_driver_classes sources, objects: objs, include_directories: includes, - dependencies: static_objs, + dependencies: static_deps, c_args: cflags, install: true)
@@ -135,7 +135,7 @@ foreach class:dpdk_driver_classes sources, objects: objs, include_directories: includes, - dependencies: shared_objs, + dependencies: shared_deps, c_args: cflags, link_args: '-Wl,--version-script=' + version_map, link_depends: version_map,
@@ -148,10 +148,10 @@ foreach class:dpdk_driver_classes # testpmd or other built-in apps can find it if necessary shared_dep = declare_dependency(link_with: shared_lib, include_directories: includes, - dependencies: shared_objs) + dependencies: shared_deps) static_dep = declare_dependency(link_with: static_lib, include_directories: includes, - dependencies: static_objs) + dependencies: static_deps) dpdk_drivers += static_lib
Signature · 1 line
2.21.0
PATCH 2/3 build: process dependencies before main build check
In reply to Bruce RichardsonIf we want to add support for turning off components because of missing dependencies, then we need to check for those dependencies before we make a determination as to whether a component should be built or not, assuming that the component says it should be built.
--- drivers/meson.build | 27 ++++++++++++++------------- lib/meson.build | 23 +++++++++++++---------- 2 files changed, 27 insertions(+), 23 deletions(-)
drivers/meson.build +14 -13
@@ -59,6 +59,20 @@ foreach class:dpdk_driver_classes # pull in driver directory which should assign to each of the above subdir(drv_path) + if build + # get dependency objs from strings + shared_deps = ext_deps + static_deps = ext_deps + foreach d:deps + if not is_variable('shared_rte_' + d) + error('Missing internal dependency "@0@" for @1@ [@2@]' + .format(d, name, 'drivers/' + drv_path)) + endif + shared_deps += [get_variable('shared_rte_' + d)] + static_deps += [get_variable('static_rte_' + d)] + endforeach + endif + if not build # some driver directories are placeholders which # are never built, so we allow suppression of the
@@ -78,19 +92,6 @@ foreach class:dpdk_driver_classes cflags += '-DALLOW_EXPERIMENTAL_API' endif - # get dependency objs from strings - shared_deps = [] - static_deps = [] - foreach d:deps - if not is_variable('shared_rte_' + d) - error('Missing dependency ' + d + - ' for driver ' + lib_name) - endif - shared_deps += [get_variable('shared_rte_' + d)] - static_deps += [get_variable('static_rte_' + d)] - endforeach - shared_deps += ext_deps - static_deps += ext_deps dpdk_extra_ldflags += pkgconfig_extra_libs # generate pmdinfo sources by building a temporary
lib/meson.build +13 -10
@@ -66,6 +66,19 @@ foreach l:libraries dir_name = 'librte_' + l subdir(dir_name) + if build + shared_deps = ext_deps + static_deps = ext_deps + foreach d:deps + if not is_variable('shared_rte_' + d) + error('Missing internal dependency "@0@" for @1@ [@2@]' + .format(d, name, 'lib/' + dir_name)) + endif + shared_deps += [get_variable('shared_rte_' + d)] + static_deps += [get_variable('static_rte_' + d)] + endforeach + endif + if not build dpdk_libs_disabled += name set_variable(name.underscorify() + '_disable_reason', reason)
@@ -82,16 +95,6 @@ foreach l:libraries shared_dep = declare_dependency(include_directories: includes) static_dep = shared_dep else - shared_deps = ext_deps - static_deps = ext_deps - foreach d:deps - if not is_variable('shared_rte_' + d) - error('Missing dependency ' + d + - ' for library ' + libname) - endif - shared_deps += [get_variable('shared_rte_' + d)] - static_deps += [get_variable('static_rte_' + d)] - endforeach if allow_experimental_apis cflags += '-DALLOW_EXPERIMENTAL_API'
Signature · 1 line
2.21.0
PATCH 3/3 build: support disabling drivers with meson
In reply to Bruce RichardsonAdd support for a new build option to turn off certain drivers. Any other drivers which depend on the one being disabled will also be disabled with a suitable debug message.
--- drivers/meson.build | 16 ++++++++++++---- meson_options.txt | 2 ++ 2 files changed, 14 insertions(+), 4 deletions(-)
drivers/meson.build +12 -4
@@ -16,6 +16,8 @@ dpdk_driver_classes = ['common', 'baseband', # depends on common and bus. 'raw'] # depends on common, bus, mempool, net and event. +disabled_drivers = get_option('disable_drivers').split(',') + default_cflags = machine_args if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation'
@@ -59,17 +61,23 @@ foreach class:dpdk_driver_classes # pull in driver directory which should assign to each of the above subdir(drv_path) - if build + if drv_path in disabled_drivers + build = false + reason = 'Explicitly disabled via build config' + elif build # get dependency objs from strings shared_deps = ext_deps static_deps = ext_deps foreach d:deps if not is_variable('shared_rte_' + d) - error('Missing internal dependency "@0@" for @1@ [@2@]' + build = false + reason = 'Missing internal dependency, "@0@"'.format(d) + message('Disabling @1@ [@2@]: missing internal dependency "@0@"' .format(d, name, 'drivers/' + drv_path)) + else + shared_deps += [get_variable('shared_rte_' + d)] + static_deps += [get_variable('static_rte_' + d)] endif - shared_deps += [get_variable('shared_rte_' + d)] - static_deps += [get_variable('static_rte_' + d)] endforeach endif
meson_options.txt +2 -0
@@ -2,6 +2,8 @@ option('allow_invalid_socket_id', type: 'boolean', value: false, description: 'allow out-of-range NUMA socket id\'s for platforms that don\'t report the value correctly') +option('disable_drivers', type: 'string', value: '', + description: 'Comma-separated list of drivers to explicitly disable.') option('drivers_install_subdir', type: 'string', value: 'dpdk/pmds-<VERSION>', description: 'Subdirectory of libdir where to install PMDs. Defaults to using a versioned subdirectory.') option('enable_docs', type: 'boolean', value: false,
Signature · 1 line
2.21.0
RE: [PATCH 3/3] build: support disabling drivers with meson
Quoted 10 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 25 September 2019 16.56Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also be disabled with a
suitable debug message.Signed-off-by Bruce Richardson <[email protected]>
Hi Bruce,
I was trying something along the lines of:
meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e
But it seems that enable_drivers does not override disable_drivers. Ninja says:
[...]
net/hinic: not in enabled drivers build config net/hns3: not in enabled drivers build config net/i40e: explicitly disabled via build config net/iavf: not in enabled drivers build config
[...]
Could you please fix that?
Also, since all sorts of exotic libraries are continuously being added to DPDK, it would be nice to have a similar Meson option for libs, i.e. enable_libs to override disable_libs=*.
Med venlig hilsen / Kind regards,
-Morten Brørup
Re: [PATCH 3/3] build: support disabling drivers with meson
06/01/2022 09:23, Morten Brørup:
Quoted 11 lines
Quoted 10 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 25 September 2019 16.56Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also be disabled with a
suitable debug message.Signed-off-by Bruce Richardson <[email protected]>
Quoted 7 lines
Hi Bruce,
I was trying something along the lines of:
meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e
But it seems that
enable_driversdoes not overridedisable_drivers. Ninja says:
It is the opposite logic. "disable" overrides "enable":
if not enable_drivers.contains(drv_path)
build = false
reason = 'not in enabled drivers build config'
elif disable_drivers.contains(drv_path)
if always_enable.contains(drv_path)
message('Driver @0@ cannot be disabled, not disabling.'.format(drv_path))
else
build = false
reason = 'explicitly disabled via build config'
endif
endif
In this case, you don't need to disable everything because of this:
if enable_drivers.length() == 0
enable_drivers = run_command(list_dir_globs, '*/*', check: true).stdout().split()
endif
RE: [PATCH 3/3] build: support disabling drivers with meson
Quoted 40 lines
From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, 6 January 2022 10.4906/01/2022 09:23, Morten Brørup:
Quoted 6 lines
Quoted 6 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 25 September 2019 16.56Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also bedisabled
Quoted 5 lines
Quoted 4 lines
with a
suitable debug message.Signed-off-by Bruce Richardson <[email protected]>Quoted 7 lines
Hi Bruce,
I was trying something along the lines of:
meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e
But it seems that
enable_driversdoes not overridedisable_drivers.Ninja says:
It is the opposite logic. "disable" overrides "enable":
if not
enable_drivers.contains(drv_path)
build = false
reason = 'not in enabled drivers build config'
elifdisable_drivers.contains(drv_path)
ifalways_enable.contains(drv_path)
message('Driver @0@ cannot be disabled, not
disabling.'.format(drv_path))
else
build = false
reason = 'explicitly disabled via build config'
endif
endif
Quoted 7 lines
In this case, you don't need to disable everything because of this:
if
enable_drivers.length()== 0
enable_drivers= run_command(list_dir_globs, '*/*', check:
true).stdout().split()
endif
Thank you, @Thomas! That solved the drivers part of my question.
@Bruce:
If you have too much time on your hands, and want to experiment with set theory on Meson, you could implement a "longest prefix match" for this, making the narrower selection take precedence. Just kidding! Enabling/disabling drivers as Thomas described works great.
Still, a similar Meson option for selecting libraries would be nice.
Re: [PATCH 3/3] build: support disabling drivers with meson
Quoted 48 lines · Morten Brørup
Quoted 40 lines
From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, 6 January 2022 10.4906/01/2022 09:23, Morten Brørup:
Quoted 6 lines
Quoted 6 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 25 September 2019 16.56Add support for a new build option to turn off certain drivers. Any other
drivers which depend on the one being disabled will also bedisabled
Quoted 5 lines
Quoted 4 lines
with a
suitable debug message.Signed-off-by Bruce Richardson <[email protected]>Quoted 7 lines
Hi Bruce,
I was trying something along the lines of:
meson configure -Ddisable_drivers=net/* -Denable_drivers=net/i40e
But it seems that
enable_driversdoes not overridedisable_drivers.Ninja says:
It is the opposite logic. "disable" overrides "enable":
if not
enable_drivers.contains(drv_path)
build = false
reason = 'not in enabled drivers build config'
elifdisable_drivers.contains(drv_path)
ifalways_enable.contains(drv_path)
message('Driver @0@ cannot be disabled, not
disabling.'.format(drv_path))
else
build = false
reason = 'explicitly disabled via build config'
endif
endifQuoted 7 lines
In this case, you don't need to disable everything because of this:
if
enable_drivers.length()== 0
enable_drivers= run_command(list_dir_globs, '*/*', check:
true).stdout().split()
endif
Quoted 8 lines
Thank you, @Thomas! That solved the drivers part of my question.
@Bruce:
If you have too much time on your hands, and want to experiment with set theory on Meson, you could implement a "longest prefix match" for this, making the narrower selection take precedence. Just kidding! Enabling/disabling drivers as Thomas described works great.
Still, a similar Meson option for selecting libraries would be nice.
Yes, I understand. I have a few concerns about implementing this just now though. David previously posted a patch[1] to do what you are looking for, and please see the discussion in that thread [2].
[1] http://inbox.dpdk.org/dev/[email protected]/
[2] http://inbox.dpdk.org/dev/[email protected]/
Quoted 25 lines · Bruce Richardson
Following on from the discussion about meson at userspace 2019, here is
a patchset which allows a user to disable drivers in a DPDK build. This
set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa
also disables net/dpaa and event/dpaa with suitable reasons provided in
the output summary at the end of the meson configuration phase.A number of changes are made to the
lib/meson.buildfile too, in order
to keep the general logic flow betweendrivers/meson.buildandlib/meson.buildas similar as possible.Example of use:
meson -Ddisable_drivers=mempool/dpaa,net/i40e build
Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson
Quoted 5 lines
drivers/meson.build | 45 +++++++++++++++++++++++++++-------------- ---- lib/meson.build | 23 +++++++++++++---------- meson_options.txt | 2 ++ 3 files changed, 42 insertions(+), 28 deletions(-)
Signature · 2 lines
Kind regards, Luca Boccassi
Quoted 26 lines · Luca Boccassi
Quoted 25 lines · Bruce Richardson
Following on from the discussion about meson at userspace 2019, here is
a patchset which allows a user to disable drivers in a DPDK build. This
set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa
also disables net/dpaa and event/dpaa with suitable reasons provided in
the output summary at the end of the meson configuration phase.A number of changes are made to the
lib/meson.buildfile too, in order
to keep the general logic flow betweendrivers/meson.buildandlib/meson.buildas similar as possible.Example of use:
meson -Ddisable_drivers=mempool/dpaa,net/i40e build
Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson
Quoted 8 lines
Quoted 5 lines
drivers/meson.build | 45 +++++++++++++++++++++++++++-------------- ---- lib/meson.build | 23 +++++++++++++---------- meson_options.txt | 2 ++ 3 files changed, 42 insertions(+), 28 deletions(-)Series-grumbly-acked-by Luca Boccassi <[email protected]>
Not exactly doing my happy-dance here myself about this set, but I think it's something those building their own DPDK will appreciate.
Quoted 41 lines
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Bruce Richardson
Sent: Thursday, September 26, 2019 10:34 AM
To: Luca Boccassi
Cc: [email protected]; [email protected]; Morten Brørup; [email protected]
Subject: Re: [dpdk-dev] [PATCH 0/3] RFC: Support disabling DPDK drivers in meson buildsQuoted 32 lines · Luca Boccassi
Quoted 30 lines · Bruce Richardson
Following on from the discussion about meson at userspace 2019, here is
a patchset which allows a user to disable drivers in a DPDK build. This
set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa
also disables net/dpaa and event/dpaa with suitable reasons provided in
the output summary at the end of the meson configuration phase.A number of changes are made to the
lib/meson.buildfile too, in order
to keep the general logic flow betweendrivers/meson.buildandlib/meson.buildas similar as possible.Example of use:
meson -Ddisable_drivers=mempool/dpaa,net/i40e build
Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with mesondrivers/meson.build | 45 +++++++++++++++++++++++++++-------------- ---- lib/meson.build | 23 +++++++++++++---------- meson_options.txt | 2 ++ 3 files changed, 42 insertions(+), 28 deletions(-)
Quoted 4 lines
Quoted 2 lines
Series-grumbly-acked-by Luca Boccassi <[email protected]>Not exactly doing my happy-dance here myself about this set, but I think it's something those building their own DPDK will appreciate.
Yes, we will.
Med venlig hilsen / kind regards
- Morten Brørup
Thomas Monjalon
Applied25/09/2019 20:01, Luca Boccassi:
Quoted 7 lines · Bruce Richardson
Quoted 4 lines
Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with mesonSeries-grumbly-acked-by Luca Boccassi <[email protected]>
Applied, thanks for implementing a community request
Quoted 20 lines · Bruce Richardson
Following on from the discussion about meson at userspace 2019, here is a patchset which allows a user to disable drivers in a DPDK build. This set also includes the changes necessary to recursively disable any drivers that depend upon the disabled one, e.g. disabling mempool/dpaa also disables net/dpaa and event/dpaa with suitable reasons provided in the output summary at the end of the meson configuration phase.
A number of changes are made to the
lib/meson.buildfile too, in order to keep the general logic flow betweendrivers/meson.buildandlib/meson.buildas similar as possible.Example of use:
meson -Ddisable_drivers=mempool/dpaa,net/i40e build
Bruce Richardson (3):
build: align variable names between drivers and libs
build: process dependencies before main build check
build: support disabling drivers with meson
Quoted 5 lines
drivers/meson.build | 45 +++++++++++++++++++++++++++------------------ lib/meson.build | 23 +++++++++++++---------- meson_options.txt | 2 ++ 3 files changed, 42 insertions(+), 28 deletions(-)
Thanks,
Maxime