usertools/dpdk-devbind: add support for PCI wildcards
PATCH 1/1 usertools/dpdk-devbind: add support for PCI wildcards
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.*
./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
--- usertools/dpdk-devbind.py | 11 +++++++++++ 1 file changed, 11 insertions(+)
usertools/dpdk-devbind.py +11 -0
@@ -8,6 +8,7 @@ import os import getopt import subprocess +from glob import glob from os.path import exists, abspath, dirname, basename if sys.version_info.major < 3:
@@ -689,6 +690,16 @@ def parse_args(): else: b_flag = arg + # resolve any PCI globs in the args + new_args = [] + sysfs_path = "/sys/bus/pci/devices/" + for arg in args: + globbed_arg = glob(sysfs_path + arg) + glob(sysfs_path + "0000:" + arg) + if globbed_arg: + new_args.extend([a[len(sysfs_path):] for a in globbed_arg]) + else: + new_args.append(arg) + args = new_args def do_arg_actions(): '''do the actual action requested by the user'''
Signature · 1 line
2.25.1
Quoted 8 lines · Bruce Richardson
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
Quoted 8 lines
./dpdk-devbind.py -b vfio-pci 80:04.* ./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
Signed-off-by Bruce Richardson <[email protected]>
This is useful, thanks.
It can be good to document this new capability in the 'help' output (./usertools/dpdk-devbind.py --usage).
PATCH v2 1/1 usertools/dpdk-devbind: add support for PCI wildcards
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.*
./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
---
V2: added help text additions
--- usertools/dpdk-devbind.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
usertools/dpdk-devbind.py +16 -0
@@ -8,6 +8,7 @@ import os import getopt import subprocess +from glob import glob from os.path import exists, abspath, dirname, basename if sys.version_info.major < 3:
@@ -89,6 +90,8 @@ def usage(): where DEVICE1, DEVICE2 etc, are specified via PCI "domain:bus:slot.func" syntax or "bus:slot.func" syntax. For devices bound to Linux kernel drivers, they may also be referred to by Linux interface name e.g. eth0, eth1, em0, em1, etc. +If devices are specified using PCI <domain:>bus:device:func format, then +shell wildcards and ranges may be used, e.g. 80:04.*, 80:04.[0-3] Options: --help, --usage:
@@ -145,6 +148,9 @@ def usage(): To bind 0000:02:00.0 and 0000:02:00.1 to the ixgbe kernel driver %(argv0)s -b ixgbe 02:00.0 02:00.1 +To bind all funcions on device 0000:02:00 to ixgbe kernel driver + %(argv0)s -b ixgbe 02:00.* + """ % locals()) # replace items from local variables
@@ -689,6 +695,16 @@ def parse_args(): else: b_flag = arg + # resolve any PCI globs in the args + new_args = [] + sysfs_path = "/sys/bus/pci/devices/" + for arg in args: + globbed_arg = glob(sysfs_path + arg) + glob(sysfs_path + "0000:" + arg) + if globbed_arg: + new_args.extend([a[len(sysfs_path):] for a in globbed_arg]) + else: + new_args.append(arg) + args = new_args def do_arg_actions(): '''do the actual action requested by the user'''
Signature · 1 line
2.25.1
Quoted 55 lines · Bruce Richardson
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.* ./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
Signed-off-by Bruce Richardson <[email protected]>Tested-by Ferruh Yigit <[email protected]>---
V2: added help text additions--- usertools/dpdk-devbind.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)usertools/dpdk-devbind.py +6
-0@@ -8,6 +8,7 @@ import os import getopt import subprocess +from glob import glob from os.path import exists, abspath, dirname, basename if sys.version_info.major < 3:@@ -89,6 +90,8 @@ def usage(): where DEVICE1, DEVICE2 etc, are specified via PCI "domain:bus:slot.func" syntax or "bus:slot.func" syntax. For devices bound to Linux kernel drivers, they may also be referred to by Linux interface name e.g. eth0, eth1, em0, em1, etc. +If devices are specified using PCI <domain:>bus:device:func format, then +shell wildcards and ranges may be used, e.g. 80:04.*, 80:04.[0-3] Options: --help, --usage:@@ -145,6 +148,9 @@ def usage(): To bind 0000:02:00.0 and 0000:02:00.1 to the ixgbe kernel driver %(argv0)s -b ixgbe 02:00.0 02:00.1 +To bind all funcions on device 0000:02:00 to ixgbe kernel driver + %(argv0)s -b ixgbe 02:00.* + """ % locals()) # replace items from local variables
usertools/dpdk-devbind.py def parse_args(): · line 702
Quoted 8 lines
usertools/dpdk-devbind.py +5
-0@@ -689,6 +695,16 @@ def parse_args(): else: b_flag = arg + # resolve any PCI globs in the args + new_args = [] + sysfs_path = "/sys/bus/pci/devices/" + for arg in args: + globbed_arg = glob(sysfs_path + arg) + glob(sysfs_path + "0000:" + arg)
os.path.join()?
usertools/dpdk-devbind.py def parse_args(): · line 704
Quoted 2 lines
usertools/dpdk-devbind.py +2
-0+ if globbed_arg: + new_args.extend([a[len(sysfs_path):] for a in globbed_arg])
os.path.basename()?
Quoted 7 lines
usertools/dpdk-devbind.py +3
-0+ else: + new_args.append(arg) + args = new_args def do_arg_actions(): '''do the actual action requested by the user'''
Signature · 2 lines
Thanks, Anatoly
Quoted 61 lines · Anatoly
Quoted 50 lines · Bruce Richardson
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.* ./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
Signed-off-by Bruce Richardson <[email protected]>Tested-by Ferruh Yigit <[email protected]>---
V2: added help text additions--- usertools/dpdk-devbind.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)usertools/dpdk-devbind.py +6
-0@@ -8,6 +8,7 @@ import os import getopt import subprocess +from glob import glob from os.path import exists, abspath, dirname, basename if sys.version_info.major < 3:@@ -89,6 +90,8 @@ def usage(): where DEVICE1, DEVICE2 etc, are specified via PCI "domain:bus:slot.func" syntax or "bus:slot.func" syntax. For devices bound to Linux kernel drivers, they may also be referred to by Linux interface name e.g. eth0, eth1, em0, em1, etc. +If devices are specified using PCI <domain:>bus:device:func format, then +shell wildcards and ranges may be used, e.g. 80:04.*, 80:04.[0-3] Options: --help, --usage:@@ -145,6 +148,9 @@ def usage(): To bind 0000:02:00.0 and 0000:02:00.1 to the ixgbe kernel driver %(argv0)s -b ixgbe 02:00.0 02:00.1 +To bind all funcions on device 0000:02:00 to ixgbe kernel driver + %(argv0)s -b ixgbe 02:00.* + """ % locals()) # replace items from local variablesusertools/dpdk-devbind.py def parse_args(): · line 701
Quoted 7 lines
usertools/dpdk-devbind.py +5
-0@@ -689,6 +695,16 @@ def parse_args(): else: b_flag = arg + # resolve any PCI globs in the args + new_args = [] + sysfs_path = "/sys/bus/pci/devices/" + for arg in args: + globbed_arg = glob(sysfs_path + arg) + glob(sysfs_path + "0000:" + arg)
os.path.join()?
Quoted 5 lines
usertools/dpdk-devbind.py def parse_args(): · line 704
Quoted 2 lines
usertools/dpdk-devbind.py +2
-0+ if globbed_arg: + new_args.extend([a[len(sysfs_path):] for a in globbed_arg])
os.path.basename()?
These are used in V4.
Quoted 55 lines · Bruce Richardson
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.* ./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
Signed-off-by Bruce Richardson <[email protected]>Tested-by Ferruh Yigit <[email protected]>---
V2: added help text additions--- usertools/dpdk-devbind.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)usertools/dpdk-devbind.py +6
-0@@ -8,6 +8,7 @@ import os import getopt import subprocess +from glob import glob from os.path import exists, abspath, dirname, basename if sys.version_info.major < 3:@@ -89,6 +90,8 @@ def usage(): where DEVICE1, DEVICE2 etc, are specified via PCI "domain:bus:slot.func" syntax or "bus:slot.func" syntax. For devices bound to Linux kernel drivers, they may also be referred to by Linux interface name e.g. eth0, eth1, em0, em1, etc. +If devices are specified using PCI <domain:>bus:device:func format, then +shell wildcards and ranges may be used, e.g. 80:04.*, 80:04.[0-3] Options: --help, --usage:@@ -145,6 +148,9 @@ def usage(): To bind 0000:02:00.0 and 0000:02:00.1 to the ixgbe kernel driver %(argv0)s -b ixgbe 02:00.0 02:00.1 +To bind all funcions on device 0000:02:00 to ixgbe kernel driver + %(argv0)s -b ixgbe 02:00.* + """ % locals()) # replace items from local variables
usertools/dpdk-devbind.py def parse_args(): · line 702
Quoted 8 lines
usertools/dpdk-devbind.py +5
-0@@ -689,6 +695,16 @@ def parse_args(): else: b_flag = arg + # resolve any PCI globs in the args + new_args = [] + sysfs_path = "/sys/bus/pci/devices/" + for arg in args: + globbed_arg = glob(sysfs_path + arg) + glob(sysfs_path + "0000:" + arg)
Also, could be
glob_path = arg if arg.startswith("0000:") else "0000:" + arg globbed_arg = glob(os.path.join(sysfs_path, glob_path))
No need to glob twice :)
Quoted 9 lines
usertools/dpdk-devbind.py +5
-0+ if globbed_arg: + new_args.extend([a[len(sysfs_path):] for a in globbed_arg]) + else: + new_args.append(arg) + args = new_args def do_arg_actions(): '''do the actual action requested by the user'''
Signature · 2 lines
Thanks, Anatoly
Quoted 61 lines · Anatoly
Quoted 50 lines · Bruce Richardson
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.* ./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
Signed-off-by Bruce Richardson <[email protected]>Tested-by Ferruh Yigit <[email protected]>---
V2: added help text additions--- usertools/dpdk-devbind.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)usertools/dpdk-devbind.py +6
-0@@ -8,6 +8,7 @@ import os import getopt import subprocess +from glob import glob from os.path import exists, abspath, dirname, basename if sys.version_info.major < 3:@@ -89,6 +90,8 @@ def usage(): where DEVICE1, DEVICE2 etc, are specified via PCI "domain:bus:slot.func" syntax or "bus:slot.func" syntax. For devices bound to Linux kernel drivers, they may also be referred to by Linux interface name e.g. eth0, eth1, em0, em1, etc. +If devices are specified using PCI <domain:>bus:device:func format, then +shell wildcards and ranges may be used, e.g. 80:04.*, 80:04.[0-3] Options: --help, --usage:@@ -145,6 +148,9 @@ def usage(): To bind 0000:02:00.0 and 0000:02:00.1 to the ixgbe kernel driver %(argv0)s -b ixgbe 02:00.0 02:00.1 +To bind all funcions on device 0000:02:00 to ixgbe kernel driver + %(argv0)s -b ixgbe 02:00.* + """ % locals()) # replace items from local variablesusertools/dpdk-devbind.py def parse_args(): · line 701
Quoted 7 lines
usertools/dpdk-devbind.py +5
-0@@ -689,6 +695,16 @@ def parse_args(): else: b_flag = arg + # resolve any PCI globs in the args + new_args = [] + sysfs_path = "/sys/bus/pci/devices/" + for arg in args: + globbed_arg = glob(sysfs_path + arg) + glob(sysfs_path + "0000:" + arg)Also, could be
Quoted 5 lines
glob_path= arg ifarg.startswith("0000:")else "0000:" + argglobbed_arg=glob(os.path.join(sysfs_path, glob_path))No need to glob twice :)
Well, the two are not quite equivalent if one assumes that the domain part can start with something other than 0000. If the domain is e.g. FFFF:, you don't want to prefix that with 0000 - only if no domain is specified. Therefore it's safer to glob twice, especially since we are not particularly concerned about performance here (or else we wouldn't be using python!).
/Bruce
Quoted 16 lines · Ferruh Yigit
Quoted 15 lines · Bruce Richardson
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.* ./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
Quoted 7 lines
Quoted 1 line
Signed-off-by Bruce Richardson <[email protected]>This is useful, thanks.
Tested-by Ferruh Yigit <[email protected]>It can be good to document this new capability in the 'help' output (
./usertools/dpdk-devbind.py--usage).
Yep, good point, added in V2.
PATCH v3 1/1 usertools/dpdk-devbind: add support for PCI wildcards
In reply to Ferruh YigitWhen binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.*
./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
---
V3: fix typo in help text additions
V2: added help text additions
--- usertools/dpdk-devbind.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
usertools/dpdk-devbind.py +16 -0
@@ -8,6 +8,7 @@ import os import getopt import subprocess +from glob import glob from os.path import exists, abspath, dirname, basename if sys.version_info.major < 3:
@@ -89,6 +90,8 @@ def usage(): where DEVICE1, DEVICE2 etc, are specified via PCI "domain:bus:slot.func" syntax or "bus:slot.func" syntax. For devices bound to Linux kernel drivers, they may also be referred to by Linux interface name e.g. eth0, eth1, em0, em1, etc. +If devices are specified using PCI <domain:>bus:device:func format, then +shell wildcards and ranges may be used, e.g. 80:04.*, 80:04.[0-3] Options: --help, --usage:
@@ -145,6 +148,9 @@ def usage(): To bind 0000:02:00.0 and 0000:02:00.1 to the ixgbe kernel driver %(argv0)s -b ixgbe 02:00.0 02:00.1 +To bind all functions on device 0000:02:00 to ixgbe kernel driver + %(argv0)s -b ixgbe 02:00.* + """ % locals()) # replace items from local variables
@@ -689,6 +695,16 @@ def parse_args(): else: b_flag = arg + # resolve any PCI globs in the args + new_args = [] + sysfs_path = "/sys/bus/pci/devices/" + for arg in args: + globbed_arg = glob(sysfs_path + arg) + glob(sysfs_path + "0000:" + arg) + if globbed_arg: + new_args.extend([a[len(sysfs_path):] for a in globbed_arg]) + else: + new_args.append(arg) + args = new_args def do_arg_actions(): '''do the actual action requested by the user'''
Signature · 1 line
2.25.1
PATCH v4 1/1 usertools/dpdk-devbind: add support for PCI wildcards
In reply to Bruce RichardsonWhen binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.*
./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
---
V4: change path manipulation to use os.path.join and basename
separated pci_glob out into separate function with doc string [dropped Ferruh's test-by due to scope of changes]
V3: fix typo in help text additions
V2: added help text additions
--- usertools/dpdk-devbind.py | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
usertools/dpdk-devbind.py +25 -0
@@ -8,7 +8,9 @@ import os import getopt import subprocess +from glob import glob from os.path import exists, abspath, dirname, basename +from os.path import join as path_join if sys.version_info.major < 3: print("WARNING: Python 2 is deprecated for use in DPDK, and will not work in future releases.", file=sys.stderr)
@@ -89,6 +91,8 @@ def usage(): where DEVICE1, DEVICE2 etc, are specified via PCI "domain:bus:slot.func" syntax or "bus:slot.func" syntax. For devices bound to Linux kernel drivers, they may also be referred to by Linux interface name e.g. eth0, eth1, em0, em1, etc. +If devices are specified using PCI <domain:>bus:device:func format, then +shell wildcards and ranges may be used, e.g. 80:04.*, 80:04.[0-3] Options: --help, --usage:
@@ -145,6 +149,9 @@ def usage(): To bind 0000:02:00.0 and 0000:02:00.1 to the ixgbe kernel driver %(argv0)s -b ixgbe 02:00.0 02:00.1 +To bind all functions on device 0000:02:00 to ixgbe kernel driver + %(argv0)s -b ixgbe 02:00.* + """ % locals()) # replace items from local variables
@@ -648,6 +655,19 @@ def show_status(): if status_dev == "misc" or status_dev == "all": show_device_status(misc_devices, "Misc (rawdev)") + +def pci_glob(arg): + '''Returns a list containing either: + * List of PCI B:D:F matching arg, using shell wildcards e.g. 80:04.* + * Only the passed arg if matching list is empty''' + sysfs_path = "/sys/bus/pci/devices" + for _glob in [arg, '0000:' + arg]: + paths = [basename(path) for path in glob(path_join(sysfs_path, _glob))] + if paths: + return paths + return [arg] + + def parse_args(): '''Parses the command-line arguments given by the user and takes the appropriate action for each'''
@@ -689,6 +709,11 @@ def parse_args(): else: b_flag = arg + # resolve any PCI globs in the args + new_args = [] + for arg in args: + new_args.extend(pci_glob(arg)) + args = new_args def do_arg_actions(): '''do the actual action requested by the user'''
Signature · 1 line
2.25.1
Quoted 8 lines · Bruce Richardson
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
Quoted 9 lines
./dpdk-devbind.py -b vfio-pci 80:04.* ./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
Signed-off-by Bruce Richardson <[email protected]>
Thomas Monjalon
Applied28/08/2020 17:48, Ferruh Yigit:
Quoted 12 lines · Bruce Richardson
Quoted 11 lines
When binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.* ./dpdk-devbind.py -u 80:04.[2-7]
Quoted 7 lines
Quoted 5 lines
The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
Signed-off-by Bruce Richardson <[email protected]>Tested-by Ferruh Yigit <[email protected]>
Applied, thanks
Thanks,
Anatoly
Quoted 25 lines
-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Monday, August 24, 2020 6:05 PM
To: [email protected]
Cc: Yigit, Ferruh <[email protected]>; Burakov, Anatoly <[email protected]>; Richardson, Bruce <[email protected]>
Subject: [PATCH v4] usertools/dpdk-devbind: add support for PCI wildcardsWhen binding or unbinding a range of devices, it can be useful to use wildcards to specify the devices rather than repeating the same prefix multiple times. We can use the python "glob" module to give us this functionality - at least for PCI devices - by checking /sys for matching files.
Examples of use from my system:
./dpdk-devbind.py -b vfio-pci 80:04.* ./dpdk-devbind.py -u 80:04.[2-7]The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The second then unbinds six of those devices, 80:04.2..80:04.7, from any driver.
Signed-off-by Bruce Richardson <[email protected]>---