build: allow using wildcards to disable drivers
PATCH 1/1 build: allow using wildcards to disable drivers
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
--- buildtools/list-dir-globs.sh | 15 +++++++++++++++ buildtools/meson.build | 2 +- drivers/meson.build | 3 ++- 3 files changed, 18 insertions(+), 2 deletions(-) create mode 100755 buildtools/list-dir-globs.sh
buildtools/list-dir-globs.sh new file +15 -0
@@ -0,0 +1,15 @@ +#! /usr/bin/env python +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +from __future__ import print_function +from os import chdir, environ +from sys import argv +from glob import iglob # glob iterator +from os.path import isdir, join + +chdir(join(environ['MESON_SOURCE_ROOT'], environ['MESON_SUBDIR'])) +dirs = [] +for path in argv[1].split(','): + dirs.extend([entry for entry in iglob(path) if isdir(entry)]) +print(",".join(dirs))
buildtools/meson.build +1 -1
@@ -4,7 +4,7 @@ subdir('pmdinfogen') pmdinfo = find_program('gen-pmdinfo-cfile.sh') - +list_dir_globs = find_program('list-dir-globs.sh') check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external
drivers/meson.build +2 -1
@@ -17,7 +17,8 @@ dpdk_driver_classes = ['common', 'event', # depends on common, bus, mempool and net. 'baseband'] # depends on common and bus. -disabled_drivers = get_option('disable_drivers').split(',') +disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), + ).stdout().strip().split(',') default_cflags = machine_args if cc.has_argument('-Wno-format-truncation')
Signature · 1 line
2.20.1
20/01/2020 18:37, Bruce Richardson:
Quoted 15 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Signed-off-by Bruce Richardson <[email protected]>---
buildtools/list-dir-globs.sh line 1
Quoted 1 line
buildtools/list-dir-globs.sh new file +1
-0@@ -0,0 +1,15 @@ +#! /usr/bin/env python
Did you start implementing it as a shell script? The extension is still .sh though I guess it might be .py.
2020-01-20, Bruce Richardson:
Quoted 12 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Signed-off-by Bruce Richardson <[email protected]>---
[snip]
Quoted 6 lines
+++
b/buildtools/list-dir-globs.shbuildtools/list-dir-globs.sh new file +4
-0@@ -0,0 +1,15 @@ +#! /usr/bin/env python +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation +
buildtools/list-dir-globs.sh line 14
Quoted 10 lines
buildtools/list-dir-globs.sh new file +10
-0+from __future__ import print_function +from os import chdir, environ +from sys import argv +from glob import iglob # glob iterator +from os.path import isdir, join + +chdir(join(environ['MESON_SOURCE_ROOT'], environ['MESON_SUBDIR'])) +dirs = [] +for path in argv[1].split(','): + dirs.extend([entry for entry in iglob(path) if isdir(entry)])
I do not fancy changing the current directory in scripts. This can lead to unexpected behavior. You don't need to, you can achieve the same result with the following code:
from __future__ import print_function
import argparse
import os
import glob
parser = argparse.ArgumentParser() parser.add_argument('disable_drivers', type=lambda s: s.split(',')) args = parser.parse_args()
root = os.path.join(os.environ['MESON_SOURCE_ROOT'],
os.environ['MESON_SUBDIR'])
dirs = []
for driver in args.disable_drivers:
driver_path = os.path.join(root, driver)
for path in glob.iglob(driver_path):
if os.path.isdir(path):
dirs.append(os.path.relpath(path, root))
print(','.join(dirs))
I did not see why you used iglob instead of glob. iglob is case insensitive. Is that for windows support?
drivers/meson.build dpdk_driver_classes = ['common', · line 21
Quoted 8 lines
drivers/meson.build +3
-1+++ b/drivers/meson.build@@ -17,7 +17,8 @@ dpdk_driver_classes = ['common', 'event', # depends on common, bus, mempool and net. 'baseband'] # depends on common and bus. -disabled_drivers = get_option('disable_drivers').split(',') +disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), + ).stdout().strip().split(',')
Why adding commas again? Since you are processing the option value through a script, you might as well print one folder per line and use the .splitlines() meson function. It will make the code simpler as you do not need to use an intermediate list to store the disabled drivers before printing them.
Signature · 1 line
Robin
Quoted 26 lines · Robin Jarry
2020-01-20, Bruce Richardson:
Quoted 12 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Signed-off-by Bruce Richardson <[email protected]>---
[snip]
Quoted 12 lines
+++
b/buildtools/list-dir-globs.shbuildtools/list-dir-globs.sh new file +10
-0@@ -0,0 +1,15 @@ +#! /usr/bin/env python +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +from __future__ import print_function +from os import chdir, environ +from sys import argv +from glob import iglob # glob iterator +from os.path import isdir, join +
Quoted 8 lines
buildtools/list-dir-globs.sh line 14
Quoted 4 lines
buildtools/list-dir-globs.sh new file +4
-0+chdir(join(environ['MESON_SOURCE_ROOT'], environ['MESON_SUBDIR'])) +dirs = [] +for path in argv[1].split(','): + dirs.extend([entry for entry in iglob(path) if isdir(entry)])I do not fancy changing the current directory in scripts. This can lead to unexpected behavior. You don't need to, you can achieve the same result with the following code:
First version indeed did not use chdir, but it seemed simpler to just chdir and work off that than try and get back to relative paths. However, I'll fix as you suggest below.
Quoted 19 lines
from
__future__importprint_function
import argparse
import os
import globparser =
argparse.ArgumentParser()parser.add_argument('disable_drivers', type=lambda s: s.split(','))args =parser.parse_args()
root = os.path.join(os.environ['MESON_SOURCE_ROOT'],
os.environ['MESON_SUBDIR'])
dirs = []
for driver in args.disable_drivers:
driver_path=os.path.join(root, driver)
for path inglob.iglob(driver_path):
ifos.path.isdir(path):
dirs.append(os.path.relpath(path, root))
print(','.join(dirs))
Quoted 2 lines
I did not see why you used iglob instead of glob. iglob is case insensitive. Is that for windows support?
No, iglob returns an iterator rather than a full list. It's just slightly more efficient if there are a large number of items.
Quoted 6 lines
Quoted 6 lines
drivers/meson.build +1
-0+++ b/drivers/meson.build@@ -17,7 +17,8 @@ dpdk_driver_classes = ['common', 'event', # depends on common, bus, mempool and net. 'baseband'] # depends on common and bus.
Quoted 10 lines
drivers/meson.build dpdk_driver_classes = ['common', · line 21
Quoted 3 lines
drivers/meson.build +2
-1-disabled_drivers = get_option('disable_drivers').split(',') +disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), + ).stdout().strip().split(',')Why adding commas again? Since you are processing the option value through a script, you might as well print one folder per line and use the .splitlines() meson function. It will make the code simpler as you do not need to use an intermediate list to store the disabled drivers before printing them.
Good point. Will update.
/Bruce
Quoted 11 lines · Thomas Monjalon
20/01/2020 18:37, Bruce Richardson:
Quoted 10 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Quoted 10 lines
buildtools/list-dir-globs.sh line 1
Quoted 6 lines
Signed-off-by Bruce Richardson <[email protected]>---
buildtools/list-dir-globs.sh new file +1
-0@@ -0,0 +1,15 @@ +#! /usr/bin/env pythonDid you start implementing it as a shell script? The extension is still .sh though I guess it might be .py.
Yes, good catch, thanks.
PATCH v2 1/1 build: allow using wildcards to disable drivers
In reply to Bruce RichardsonRather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
---
V2:
- fixed file suffix
- since it's being called from meson, make this python3 only
- remove use of chdir()
- use '\n' rather than ',' between entries
--- buildtools/list-dir-globs.py | 13 +++++++++++++ buildtools/meson.build | 2 +- drivers/meson.build | 3 ++- 3 files changed, 16 insertions(+), 2 deletions(-) create mode 100755 buildtools/list-dir-globs.py
buildtools/list-dir-globs.py new file +13 -0
@@ -0,0 +1,13 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +import sys +import os +from glob import iglob # glob iterator +from os.path import join, relpath, isdir + +root = join(os.environ['MESON_SOURCE_ROOT'], os.environ['MESON_SUBDIR']) +for path in sys.argv[1].split(','): + relpaths = [relpath(p, root) for p in iglob(join(root, path)) if isdir(p)] + print("\n".join(relpaths))
buildtools/meson.build +1 -1
@@ -4,7 +4,7 @@ subdir('pmdinfogen') pmdinfo = find_program('gen-pmdinfo-cfile.sh') - +list_dir_globs = find_program('list-dir-globs.py') check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external
drivers/meson.build +2 -1
@@ -17,7 +17,8 @@ dpdk_driver_classes = ['common', 'event', # depends on common, bus, mempool and net. 'baseband'] # depends on common and bus. -disabled_drivers = get_option('disable_drivers').split(',') +disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), + ).stdout().split() default_cflags = machine_args if cc.has_argument('-Wno-format-truncation')
Signature · 1 line
2.20.1
2020-01-21, Bruce Richardson:
Quoted 11 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Signed-off-by Bruce Richardson <[email protected]>
[snip]
buildtools/list-dir-globs.py line 7
Quoted 9 lines
+++
b/buildtools/list-dir-globs.pybuildtools/list-dir-globs.py new file +7
-0@@ -0,0 +1,13 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +import sys +import os +from glob import iglob # glob iterator
No need to make it explicit. People can read the description in the official docs.
buildtools/list-dir-globs.py line 8
Quoted 1 line
buildtools/list-dir-globs.py new file +1
-0+from os.path import join, relpath, isdir
You already imported 'os'. These functions are available under the 'os.path' namespace. No need to import them again.
buildtools/list-dir-globs.py line 11
Quoted 2 lines
buildtools/list-dir-globs.py new file +2
-0+root = join(os.environ['MESON_SOURCE_ROOT'], os.environ['MESON_SUBDIR']) +for path in sys.argv[1].split(','):
Directly accessing sys.argv exposes you to ugly errors when the script is called with the wrong number of arguments. It would be better to use argparse which will handle the error reporting for you.
buildtools/list-dir-globs.py line 13
Quoted 2 lines
buildtools/list-dir-globs.py new file +2
-0+ relpaths = [relpath(p, root) for p in iglob(join(root, path)) if isdir(p)] + print("\n".join(relpaths))
Using one-liner syntax like these really makes the code hard to understand. Explicit for loops with explicit if tests would be a lot nicer.
Also, why use an intermediate variable to then, join each element with '\n' and print that? You can print the matching dirs as you iterate over them.
Have a look at my previous reply for a complete example of what I mean.
Signature · 1 line
Robin
Quoted 13 lines · Robin Jarry
2020-01-21, Bruce Richardson:
Quoted 11 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Signed-off-by Bruce Richardson <[email protected]>[snip]
Quoted 12 lines
buildtools/list-dir-globs.py line 7
Quoted 9 lines
+++
b/buildtools/list-dir-globs.pybuildtools/list-dir-globs.py new file +7
-0@@ -0,0 +1,13 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +import sys +import os +from glob import iglob # glob iteratorNo need to make it explicit. People can read the description in the official docs.
Except the fact that you yourself mistook it for a case-insensitive glob implies that a comment is needed! :-)
Quoted 6 lines
buildtools/list-dir-globs.py line 8
Quoted 2 lines
buildtools/list-dir-globs.py new file +1
-0+from os.path import join, relpath, isdirYou already imported 'os'. These functions are available under the 'os.path' namespace. No need to import them again.
Yes, using more words. Simpler syntax to call join() and isdir() rather than os.path.join(), os.path.isdir()
Quoted 6 lines
buildtools/list-dir-globs.py line 11
Quoted 2 lines
buildtools/list-dir-globs.py new file +2
-0+root = join(os.environ['MESON_SOURCE_ROOT'], os.environ['MESON_SUBDIR']) +for path in sys.argv[1].split(','):Directly accessing sys.argv exposes you to ugly errors when the script is called with the wrong number of arguments. It would be better to use argparse which will handle the error reporting for you.
This is script is to be called from the build system. I'm not worried about incorrect numbers of args, and argparse itself seems overkill. I'll add a check to print an error if len(argv) != 1.
Quoted 8 lines
buildtools/list-dir-globs.py line 13
Quoted 3 lines
buildtools/list-dir-globs.py new file +2
-0+ relpaths = [relpath(p, root) for p in iglob(join(root, path)) if isdir(p)] + print("\n".join(relpaths))Using one-liner syntax like these really makes the code hard to understand. Explicit for loops with explicit if tests would be a lot nicer.
Quoted 6 lines
Also, why use an intermediate variable to then, join each element with '\n' and print that? You can print the matching dirs as you iterate over them.
Have a look at my previous reply for a complete example of what I mean.
Just keeping the code short. I actually had originally got the print join in the same line as the list comprehension but that even I felt was a bit unreadable - even if it did leave a nice short script!
I'll expand to explicit for loop in V3.
/Bruce
2020-01-21, Bruce Richardson:
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
+from glob import iglob # glob iterator
No need to make it explicit. People can read the description in the official docs.
Except the fact that you yourself mistook it for a case-insensitive glob implies that a comment is needed! :-)
The fact that I should check things before I make comments should not force people to add comments next to imports :-)
Quoted 3 lines
This is script is to be called from the build system. I'm not worried about incorrect numbers of args, and argparse itself seems overkill. I'll add a check to print an error if len(argv) != 1.
Fair enough. However, argparse is never overkill. Using it is not going to slow down your script.
Quoted 3 lines
Just keeping the code short. I actually had originally got the print join in the same line as the list comprehension but that even I felt was a bit unreadable - even if it did leave a nice short script!
This is not perl^Wa competition :-)
Signature · 1 line
Robin
PATCH v3 1/1 build: allow using wildcards to disable drivers
In reply to Bruce RichardsonRather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
---
V3:
- added check for correct number of params
- replaced list comprehension with loops for simplicity
- allow running without meson environmental vars set (for easier testing)
V2:
- fixed file suffix
- since it's being called from meson, make this python3 only
- remove use of chdir()
- use '\n' rather than ',' between entries
--- buildtools/list-dir-globs.py | 21 +++++++++++++++++++++ buildtools/meson.build | 2 +- drivers/meson.build | 3 ++- 3 files changed, 24 insertions(+), 2 deletions(-) create mode 100755 buildtools/list-dir-globs.py
buildtools/list-dir-globs.py new file +21 -0
@@ -0,0 +1,21 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +import sys +import os +from glob import iglob +from os.path import join, relpath, isdir + +if len(sys.argv) != 2: + print("Usage: {0} <path-glob>[,<path-glob>[,...]]".format(sys.argv[0])) + sys.exit(1) + +root = '.' +if 'MESON_SOURCE_ROOT' in os.environ and 'MESON_SUBDIR' in os.environ: + root = join(os.environ['MESON_SOURCE_ROOT'], os.environ['MESON_SUBDIR']) + +for path in sys.argv[1].split(','): + for p in iglob(join(root, path)): + if isdir(p): + print(relpath(p))
buildtools/meson.build +1 -1
@@ -4,7 +4,7 @@ subdir('pmdinfogen') pmdinfo = find_program('gen-pmdinfo-cfile.sh') - +list_dir_globs = find_program('list-dir-globs.py') check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external
drivers/meson.build +2 -1
@@ -17,7 +17,8 @@ dpdk_driver_classes = ['common', 'event', # depends on common, bus, mempool and net. 'baseband'] # depends on common and bus. -disabled_drivers = get_option('disable_drivers').split(',') +disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), + ).stdout().split() default_cflags = machine_args if cc.has_argument('-Wno-format-truncation')
Signature · 1 line
2.20.1
2020-01-24, Bruce Richardson:
Quoted 11 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Signed-off-by Bruce Richardson <[email protected]>
[snip]
Quoted 1 line
+from os.path import join, relpath, isdir
As a general rule, it is better to only import one symbol per line. This makes subsequent patches easier to read.
buildtools/list-dir-globs.py line 12
Quoted 3 lines
buildtools/list-dir-globs.py new file +3
-0+if len(sys.argv) != 2: + print("Usage: {0} <path-glob>[,<path-glob>[,...]]".format(sys.argv[0])) + sys.exit(1)
PEP8 advises to use 4 spaces per indentation level. This is the indentation style adopted by all other python scripts in dpdk (see doc/guides/contributing/coding_style.rst). Could you fix that?
buildtools/list-dir-globs.py line 16
Quoted 3 lines
buildtools/list-dir-globs.py new file +3
-0+root = '.' +if 'MESON_SOURCE_ROOT' in os.environ and 'MESON_SUBDIR' in os.environ: + root = join(os.environ['MESON_SOURCE_ROOT'], os.environ['MESON_SUBDIR'])
You can do simpler and shorter:
root = join(os.getenv('MESON_SOURCE_ROOT', ''),
os.getenv('MESON_SUBDIR', ''), '.')Sorry to pester you with all this, but python code in DPDK really needs some loving :-)
Signature · 1 line
Robin
Quoted 44 lines
-----Original Message-----
From: Robin Jarry <[email protected]>
Sent: Friday, January 24, 2020 12:28 PM
To: Richardson, Bruce <[email protected]>
Cc: [email protected]; [email protected]
Subject: Re: [PATCH v3] build: allow using wildcards to disable drivers2020-01-24, Bruce Richardson:
Quoted 3 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure
meson using e.g.
Quoted 8 lines
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Signed-off-by Bruce Richardson <[email protected]>[snip]
Quoted 1 line
+from os.path import join, relpath, isdir
As a general rule, it is better to only import one symbol per line. This makes subsequent patches easier to read.
buildtools/list-dir-globs.py line 12
Quoted 4 lines
buildtools/list-dir-globs.py new file +4
-0+if len(sys.argv) != 2: + print("Usage: {0} +<path-glob>[,<path-glob>[,...]]".format(sys.argv[0])) + sys.exit(1)PEP8 advises to use 4 spaces per indentation level. This is the indentation style adopted by all other python scripts in dpdk (see
doc/guides/contributing/coding_style.rst). Could you fix that?buildtools/list-dir-globs.py
Quoted 4 lines
buildtools/list-dir-globs.py new file +4
-0+root = '.' +if 'MESON_SOURCE_ROOT' in os.environ and 'MESON_SUBDIR' in os.environ: + root = join(os.environ['MESON_SOURCE_ROOT'], +os.environ['MESON_SUBDIR'])You can do simpler and shorter:
Quoted 2 lines
root = join(os.getenv('MESON_SOURCE_ROOT', ''),
os.getenv('MESON_SUBDIR', ''), '.')
I think a little shorter again by putting the '.' as the default value for one of the env vars.
Quoted 6 lines
Sorry to pester you with all this, but python code in DPDK really needs some loving :-)
--
Robin
PATCH v4 1/1 build: allow using wildcards to disable drivers
In reply to Bruce RichardsonRather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
---
V4:
- get pep8/pycodestyle compliance
- simplify getting the root directory
V3:
- added check for correct number of params
- replaced list comprehension with loops for simplicity
- allow running without meson environmental vars set (for easier testing)
V2:
- fixed file suffix
- since it's being called from meson, make this python3 only
- remove use of chdir()
- use '\n' rather than ',' between entries
--- buildtools/list-dir-globs.py | 19 +++++++++++++++++++ buildtools/meson.build | 2 +- drivers/meson.build | 3 ++- 3 files changed, 22 insertions(+), 2 deletions(-) create mode 100755 buildtools/list-dir-globs.py
buildtools/list-dir-globs.py new file +19 -0
@@ -0,0 +1,19 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +import sys +import os +from glob import iglob + +if len(sys.argv) != 2: + print("Usage: {0} <path-glob>[,<path-glob>[,...]]".format(sys.argv[0])) + sys.exit(1) + +root = os.path.join(os.getenv('MESON_SOURCE_ROOT', '.'), + os.getenv('MESON_SUBDIR', '.')) + +for path in sys.argv[1].split(','): + for p in iglob(os.path.join(root, path)): + if os.path.isdir(p): + print(os.path.relpath(p))
buildtools/meson.build +1 -1
@@ -4,7 +4,7 @@ subdir('pmdinfogen') pmdinfo = find_program('gen-pmdinfo-cfile.sh') - +list_dir_globs = find_program('list-dir-globs.py') check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external
drivers/meson.build +2 -1
@@ -17,7 +17,8 @@ dpdk_driver_classes = ['common', 'event', # depends on common, bus, mempool and net. 'baseband'] # depends on common and bus. -disabled_drivers = get_option('disable_drivers').split(',') +disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), + ).stdout().split() default_cflags = machine_args if cc.has_argument('-Wno-format-truncation')
Signature · 1 line
2.20.1
2020-01-24, Bruce Richardson:
Quoted 11 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Signed-off-by Bruce Richardson <[email protected]>
Looks good to me.
Signature · 1 line
Robin
Quoted 5 lines · Bruce Richardson
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
Maybe documenting that in 'build-sdk-meson.txt' would be beneficial?
With regards
Andrzej Ostruszka
Quoted 9 lines · Andrzej Ostruszka
Quoted 5 lines · Bruce Richardson
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
Maybe documenting that in 'build-sdk-meson.txt' would be beneficial?
Yes, yes it would. Thanks for the suggestion.
/Bruce
Quoted 29 lines · Bruce Richardson
Rather than having to explicitly list each and every driver to disable in a
build, we can use a small python script and the python glob library to
expand out the wildcards. This means that we can configure meson using e.g.meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with
wildcarded drivers as required.Signed-off-by: Bruce Richardson <
[email protected]Quoted 1 line
---
V4:
- get pep8/pycodestyle compliance
- simplify getting the root directoryV3:
- added check for correct number of params
- replaced list comprehension with loops for simplicity
- allow running without meson environmental vars set (for easier testing)
Quoted 5 lines
V2:
- fixed file suffix
- since it's being called from meson, make this python3 only
- remove use ofchdir()
- use '\n' rather than ',' between entries
Signature · 2 lines
Kind regards, Luca Boccassi
PATCH v5 1/1 build: allow using wildcards to disable drivers
In reply to Bruce RichardsonRather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
---
V5:
- add a couple of lines to the doc as an example of using this
V4:
- get pep8/pycodestyle compliance
- simplify getting the root directory
V3:
- added check for correct number of params
- replaced list comprehension with loops for simplicity
- allow running without meson environmental vars set (for easier testing)
V2:
- fixed file suffix
- since it's being called from meson, make this python3 only
- remove use of chdir()
- use '\n' rather than ',' between entries
--- buildtools/list-dir-globs.py | 19 +++++++++++++++++++ buildtools/meson.build | 2 +- doc/build-sdk-meson.txt | 5 ++++- drivers/meson.build | 3 ++- 4 files changed, 26 insertions(+), 3 deletions(-) create mode 100755 buildtools/list-dir-globs.py
buildtools/list-dir-globs.py new file +19 -0
@@ -0,0 +1,19 @@ +#! /usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +import sys +import os +from glob import iglob + +if len(sys.argv) != 2: + print("Usage: {0} <path-glob>[,<path-glob>[,...]]".format(sys.argv[0])) + sys.exit(1) + +root = os.path.join(os.getenv('MESON_SOURCE_ROOT', '.'), + os.getenv('MESON_SUBDIR', '.')) + +for path in sys.argv[1].split(','): + for p in iglob(os.path.join(root, path)): + if os.path.isdir(p): + print(os.path.relpath(p))
buildtools/meson.build +1 -1
@@ -4,7 +4,7 @@ subdir('pmdinfogen') pmdinfo = find_program('gen-pmdinfo-cfile.sh') - +list_dir_globs = find_program('list-dir-globs.py') check_experimental_syms = find_program('check-experimental-syms.sh') # set up map-to-def script using python, either built-in or external
doc/build-sdk-meson.txt +4 -1
@@ -84,7 +84,10 @@ Project-specific options are passed used -Doption=value:: meson -Dmachine=default # use builder-independent baseline -march -Examples of setting the same options using meson configure:: + meson -Ddisable_drivers=event/*,net/tap # disable tap driver and all + # eventdev PMDs for a smaller build + +Examples of setting some of the same options using meson configure:: meson configure -Dwerror=true
drivers/meson.build +2 -1
@@ -17,7 +17,8 @@ dpdk_driver_classes = ['common', 'event', # depends on common, bus, mempool and net. 'baseband'] # depends on common and bus. -disabled_drivers = get_option('disable_drivers').split(',') +disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), + ).stdout().split() default_cflags = machine_args if cc.has_argument('-Wno-format-truncation')
Signature · 1 line
2.20.1
Thomas Monjalon
Applied27/01/2020 15:28, Bruce Richardson:
Quoted 6 lines
Rather than having to explicitly list each and every driver to disable in a build, we can use a small python script and the python glob library to expand out the wildcards. This means that we can configure meson using e.g.
meson -Ddisable_drivers=crypto/*,event/* build
Quoted 7 lines
to do a build omitting all the crypto and event drivers. Explicitly specified drivers e.g. net/i40e, work as before, and can be mixed with wildcarded drivers as required.
Signed-off-by Bruce Richardson <[email protected]>Reviewed-by Robin Jarry <[email protected]>Acked-by Luca Boccassi <[email protected]>
Applied, thanks