vfio: selftests: Automate finding/setting up devices for testing
Subsystems (4)
cover vfio: selftests: Automate finding/setting up devices for testing
This series introduces two improvements to make setting up devices for VFIO selftests more user-friendly.
1. Add functionality to the VFIO selftest setup script to find devices on a user's system that are compatible with the VFIO selftest framework (i.e. devices that have a VFIO selftest driver).
Example:
$ ./tools/testing/selftests/vfio/scripts/setup.sh -l
Supported devices:
0000:6a:01.0
0000:6f:01.0
0000:74:01.0
0000:79:01.0
0000:e7:01.0
0000:ec:01.0
0000:f1:01.0
0000:f6:01.02. Modify the VFIO selftest setup script to try to find and set up a compatible device automatically. "Set up the device for testing" means unbind the device from its driver, then bind it to VFIO. The user is no longer required to find a segment:bus:device.function number to pass to the script (though this functionality is still available through a -d option).
Example of automated device setup:
$ ./tools/testing/selftests/vfio/scripts/setup.sh
+ echo "0000:6a:01.0" > /sys/bus/pci/drivers/idxd/unbind
+ echo "vfio-pci" > /sys/bus/pci/devices/0000:6a:01.0/driver_override
+ echo "0000:6a:01.0" > /sys/bus/pci/drivers/vfio-pci/bind
Successfully set up 0000:6a:01.0Example of manual device setup:
$ ./tools/testing/selftests/vfio/scripts/setup.sh -d 0000:6a:01.0
+ echo "0000:6a:01.0" > /sys/bus/pci/drivers/idxd/unbind
+ echo "vfio-pci" > /sys/bus/pci/devices/0000:6a:01.0/driver_override
+ echo "0000:6a:01.0" > /sys/bus/pci/drivers/vfio-pci/bind
Successfully set up 0000:6a:01.0Josh Hilke (2):
vfio: selftests: Find devices that are compatible with VFIO selftests
vfio: selftests: Automate finding/setting up devices for testing
tools/testing/selftests/vfio/scripts/setup.sh | 63 +++++++++++++++++-- 1 file changed, 59 insertions(+), 4 deletions(-)
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
PATCH 1/2 vfio: selftests: Find devices that are compatible with VFIO selftests
Update the VFIO selftest setup script to print the segment:bus:device.function numbers of devices on the user's system that have VFIO selftest drivers (i.e. are compatible with the VFIO selftest infrastructure). This makes it easy for users to quickly find devices that are compatible with VFIO selftests.
Example of how to list compatible devices on the system:
$ ./tools/testing/selftests/vfio/scripts/setup.sh -l Supported devices: 0000:6a:01.0 0000:6f:01.0 0000:74:01.0 0000:79:01.0 0000:e7:01.0 0000:ec:01.0 0000:f1:01.0 0000:f6:01.0
Example of setting up a device:
$ ./tools/testing/selftests/vfio/scripts/setup.sh -d 0000:6a:01.0 + echo "0000:6a:01.0" > /sys/bus/pci/drivers/idxd/unbind + echo "vfio-pci" > /sys/bus/pci/devices/0000:6a:01.0/driver_override + echo "0000:6a:01.0" > /sys/bus/pci/drivers/vfio-pci/bind Successfully set up 0000:6a:01.0
This makes the script much more user friendly so that no one has to dig around in the VFIO selftest libraries to figure out which devices can run the selftests.
--- tools/testing/selftests/vfio/scripts/setup.sh | 43 +++++++++++++++++-- 1 file changed, 40 insertions(+), 3 deletions(-)
tools/testing/selftests/vfio/scripts/setup.sh +40 -3
@@ -3,18 +3,54 @@ set -e source $(dirname -- "${BASH_SOURCE[0]}")/lib.sh +# List of devices which have a VFIO selftest driver +DEVICES=( + "8086:0b25" # Intel Data Streaming Accelerator + "8086:0cf8" # Intel CBDMA +) + +function print_supported_devices() { + local vendor_device_id + local id + + for vendor_device_id in "${DEVICES[@]}"; do + read -r id <<< "${vendor_device_id}" + lspci -D -d "${id}" | awk '{ print $1 }' + done +} + +function usage() { + echo "usage: $0 [-l] [-d <segment:bus:device.function>]" >&2 + echo "" >&2 + echo " -l List segment:bus:device.function numbers of supported devices." >&2 + echo " -d segment:bus:device.function to set up." >&2 +} + function main() { local device_bdf local device_dir local numvfs local driver + local bdf_list=() + + while getopts "ld:" opt; do + case ${opt} in + l) + echo "Supported devices: " + print_supported_devices + exit 0 + ;; + d) bdf_list+=("${OPTARG}") ;; + *) usage; exit 1 ;; + esac + done - if [ $# = 0 ]; then - echo "usage: $0 segment:bus:device.function ..." >&2 + if [ ${#bdf_list[@]} -eq 0 ]; then + usage exit 1 fi - for device_bdf in "$@"; do + for device_bdf in "${bdf_list[@]}"; do test -d /sys/bus/pci/devices/${device_bdf} device_dir=${DEVICES_DIR}/${device_bdf}
@@ -42,6 +78,7 @@ function main() { bind ${device_bdf} vfio-pci touch ${device_dir}/vfio-pci + echo "Successfully set up ${device_bdf}" done }
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
David Matlack
maintainerRe: [PATCH 1/2] vfio: selftests: Find devices that are compatible with VFIO selftests
Quoted 40 lines · Josh Hilke
Update the VFIO selftest setup script to print the segment:bus:device.function numbers of devices on the user's system that have VFIO selftest drivers (i.e. are compatible with the VFIO selftest infrastructure). This makes it easy for users to quickly find devices that are compatible with VFIO selftests.
Example of how to list compatible devices on the system:
$ ./tools/testing/selftests/vfio/scripts/setup.sh -l Supported devices: 0000:6a:01.0 0000:6f:01.0 0000:74:01.0 0000:79:01.0 0000:e7:01.0 0000:ec:01.0 0000:f1:01.0 0000:f6:01.0Example of setting up a device:
$ ./tools/testing/selftests/vfio/scripts/setup.sh -d 0000:6a:01.0 + echo "0000:6a:01.0" > /sys/bus/pci/drivers/idxd/unbind + echo "vfio-pci" > /sys/bus/pci/devices/0000:6a:01.0/driver_override + echo "0000:6a:01.0" > /sys/bus/pci/drivers/vfio-pci/bind Successfully set up 0000:6a:01.0This makes the script much more user friendly so that no one has to dig around in the VFIO selftest libraries to figure out which devices can run the selftests.
Signed-off-by Josh Hilke <[email protected]>--- tools/testing/selftests/vfio/scripts/setup.sh | 43 +++++++++++++++++-- 1 file changed, 40 insertions(+), 3 deletions(-)tools/testing/selftests/vfio/scripts/setup.sh +0
-0
tools/testing/selftests/vfio/scripts/setup.sh set -e · line 8
Quoted 6 lines
tools/testing/selftests/vfio/scripts/setup.sh +3
-0@@ -3,18 +3,54 @@ set -e source $(dirname -- "${BASH_SOURCE[0]}")/lib.sh +# List of devices which have a VFIO selftest driver +DEVICES=( + "8086:0b25" # Intel Data Streaming Accelerator
Please also add the new DMR and GNR-D DSA devices that were added recently:
https://github.com/awilliam/linux-vfio/commit/c82cfe15916d33e89c2d2efeeb624e8c9c2c4ca8
tools/testing/selftests/vfio/scripts/setup.sh set -e · line 17
Quoted 9 lines
tools/testing/selftests/vfio/scripts/setup.sh +9
-0+ "8086:0cf8" # Intel CBDMA +) + +function print_supported_devices() { + local vendor_device_id + local id + + for vendor_device_id in "${DEVICES[@]}"; do + read -r id <<< "${vendor_device_id}"
Sashiko seems to think the read here is unecessary.
https://sashiko.dev/#/patchset/20260409015139.2800185-1-jrhilke%40google.com?part=1
tools/testing/selftests/vfio/scripts/setup.sh set -e · line 18
Quoted 1 line
tools/testing/selftests/vfio/scripts/setup.sh +1
-0+ lspci -D -d "${id}" | awk '{ print $1 }'
Let's print the whole line and let the user parse out the BDF with awk. That will let the user see which type of device it its rather than just raw BDFs.
Quoted 14 lines
tools/testing/selftests/vfio/scripts/setup.sh +10
-0+ done +} + +function usage() { + echo "usage: $0 [-l] [-d <segment:bus:device.function>]" >&2 + echo "" >&2 + echo " -l List segment:bus:device.function numbers of supported devices." >&2 + echo " -d segment:bus:device.function to set up." >&2 +} + function main() { local device_bdf local device_dir local numvfs
tools/testing/selftests/vfio/scripts/setup.sh set -e · line 42
Quoted 10 lines
tools/testing/selftests/vfio/scripts/setup.sh +9
-0local driver + local bdf_list=() + + while getopts "ld:" opt; do + case ${opt} in + l) + echo "Supported devices: " + print_supported_devices + exit 0 + ;;
nit: Put the echo in print_supported_devices and this can be a one-liner like the rest of the options.
while getopts "ld:" opt; do
case ${opt} in
l) print_supported_devices; exit 0 ;;
d) bdf_list+=("${OPTARG}") ;;
*) usage; exit 1 ;;
esac
donetools/testing/selftests/vfio/scripts/setup.sh set -e · line 43
Quoted 1 line
tools/testing/selftests/vfio/scripts/setup.sh +1
-0+ d) bdf_list+=("${OPTARG}") ;;
Let's keep the current semantics where the list of BDFs is passed as positional arguments. The next commit would just pick a device if there are no positional arguments.
This will keep the script backwards compatible with current usage and also keep it simple (no need to pass extra -d flag for every BDF):
./setup.sh BDF1 BDF2 BDF3
vs.
./setup.sh -d BDF1 -d BDF2 -d BDF3
Quoted 27 lines
tools/testing/selftests/vfio/scripts/setup.sh +7
-3+ *) usage; exit 1 ;; + esac + done - if [ $# = 0 ]; then - echo "usage: $0 segment:bus:device.function ..." >&2 + if [ ${#bdf_list[@]} -eq 0 ]; then + usage exit 1 fi - for device_bdf in "$@"; do + for device_bdf in "${bdf_list[@]}"; do test -d /sys/bus/pci/devices/${device_bdf} device_dir=${DEVICES_DIR}/${device_bdf}@@ -42,6 +78,7 @@ function main() { bind ${device_bdf} vfio-pci touch ${device_dir}/vfio-pci + echo "Successfully set up ${device_bdf}" done }Signature · 1 line
2.53.0.1213.gd9a14994de-goog
Re: [PATCH 1/2] vfio: selftests: Find devices that are compatible with VFIO selftests
Quoted 8 lines · David Matlack
tools/testing/selftests/vfio/scripts/setup.sh set -e · line 8
Quoted 3 lines
tools/testing/selftests/vfio/scripts/setup.sh +3
-0+# List of devices which have a VFIO selftest driver +DEVICES=( + "8086:0b25" # Intel Data Streaming AcceleratorPlease also add the new DMR and GNR-D DSA devices that were added recently:
https://github.com/awilliam/linux-vfio/commit/c82cfe15916d33e89c2d2efeeb624e8c9c2c4ca8
Ack. Will do in v3.
Quoted 10 lines
tools/testing/selftests/vfio/scripts/setup.sh set -e · line 17
Quoted 6 lines
tools/testing/selftests/vfio/scripts/setup.sh +6
-0+function print_supported_devices() { + local vendor_device_id + local id + + for vendor_device_id in "${DEVICES[@]}"; do + read -r id <<< "${vendor_device_id}"Sashiko seems to think the read here is unecessary.
https://sashiko.dev/#/patchset/20260409015139.2800185-1-jrhilke%40google.com?part=1
Ah yea, the read is a vestige of a different iteration I had of this script. I'll remove it in v3.
Quoted 5 lines
Quoted 1 line
+ lspci -D -d "${id}" | awk '{ print $1 }'
Let's print the whole line and let the user parse out the BDF with awk. That will let the user see which type of device it its rather than just raw BDFs.
Will do in v3.
Quoted 11 lines
tools/testing/selftests/vfio/scripts/setup.sh set -e · line 42
Quoted 7 lines
tools/testing/selftests/vfio/scripts/setup.sh +7
-0+ while getopts "ld:" opt; do + case ${opt} in + l) + echo "Supported devices: " + print_supported_devices + exit 0 + ;;nit: Put the echo in
print_supported_devicesand this can be a one-liner like the rest of the options.
Quoted 7 lines
while getopts "ld:" opt; do
case ${opt} in
l) print_supported_devices; exit 0 ;;
d) bdf_list+=("${OPTARG}") ;;
*) usage; exit 1 ;;
esac
doneWill do in v3.
Quoted 6 lines
Quoted 1 line
+ d) bdf_list+=("${OPTARG}") ;;
Let's keep the current semantics where the list of BDFs is passed as positional arguments. The next commit would just pick a device if there are no positional arguments.
Quoted 6 lines
This will keep the script backwards compatible with current usage and also keep it simple (no need to pass extra -d flag for every BDF):
./setup.sh BDF1 BDF2 BDF3
vs.
./setup.sh -d BDF1 -d BDF2 -d BDF3
Works for me. Will do in v3.
PATCH 2/2 vfio: selftests: Automate setting up devices for testing
In reply to Josh HilkeUpdate the VFIO selftest setup script to find a device that has a VFIO selftest driver on the system, and set it up (i.e unbind that device from its driver, and then bind that device to VFIO).
This makes the process of setting up a device to VFIO selftests much easier, as users don't have to manually figure out which device to use. Though, users can still manually setup a device by passing the -d argument with a segment:bus:device.function number.
Example of automated device setup:
$ ./tools/testing/selftests/vfio/scripts/setup.sh./setup.sh + echo "0000:6a:01.0" > /sys/bus/pci/drivers/idxd/unbind + echo "vfio-pci" > /sys/bus/pci/devices/0000:6a:01.0/driver_override + echo "0000:6a:01.0" > /sys/bus/pci/drivers/vfio-pci/bind Successfully set up 0000:6a:01.0
Example of automated device setup where no compatible devices exist:
$ . ./tools/testing/selftests/vfio/scripts/setup.sh/setup.sh No available supported devices found on the system.
Example of manual device setup:
$ ./tools/testing/selftests/vfio/scripts/setup.sh -d 0000:6a:01.0 + echo "0000:6a:01.0" > /sys/bus/pci/drivers/idxd/unbind + echo "vfio-pci" > /sys/bus/pci/devices/0000:6a:01.0/driver_override + echo "0000:6a:01.0" > /sys/bus/pci/drivers/vfio-pci/bind Successfully set up 0000:6a:01.0
--- tools/testing/selftests/vfio/scripts/setup.sh | 22 +++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)
tools/testing/selftests/vfio/scripts/setup.sh +20 -2
@@ -19,11 +19,30 @@ function print_supported_devices() { done } +function pick_device() { + local bdf + + while read -r bdf; do + if [ -n "${bdf}" ]; then + if [ -d "${DEVICES_DIR}/${bdf}" ]; then + echo "${bdf} has already been set up, exiting." >&2 + exit 0 + fi + echo "${bdf}" + return 0 + fi + done <<< "$(print_supported_devices)" + + echo "No available supported devices found on the system." >&2 + exit 1 +} + function usage() { echo "usage: $0 [-l] [-d <segment:bus:device.function>]" >&2 echo "" >&2 echo " -l List segment:bus:device.function numbers of supported devices." >&2 echo " -d segment:bus:device.function to set up." >&2 + echo " If -d is not specified, a device will be automatically picked." >&2 } function main() {
@@ -46,8 +65,7 @@ function main() { done if [ ${#bdf_list[@]} -eq 0 ]; then - usage - exit 1 + bdf_list=($(pick_device)) fi for device_bdf in "${bdf_list[@]}"; do
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
David Matlack
maintainerRe: [PATCH 2/2] vfio: selftests: Automate setting up devices for testing
Quoted 4 lines · Josh Hilke
Update the VFIO selftest setup script to find a device that has a VFIO selftest driver on the system, and set it up (i.e unbind that device from its driver, and then bind that device to VFIO).
Quoted 8 lines
This makes the process of setting up a device to VFIO selftests much easier, as users don't have to manually figure out which device to use. Though, users can still manually setup a device by passing the -d argument with a segment:bus:device.function number.
Example of automated device setup:
$ ./tools/testing/selftests/vfio/scripts/setup.sh./setup.sh
Script path here got garbled
Quoted 8 lines
+ echo "0000:6a:01.0" >
/sys/bus/pci/drivers/idxd/unbind
+ echo "vfio-pci" > /sys/bus/pci/devices/0000:6a:01.0/driver_override
+ echo "0000:6a:01.0" >/sys/bus/pci/drivers/vfio-pci/bindSuccessfully set up 0000:6a:01.0Example of automated device setup where no compatible devices exist:
$ . ./tools/testing/selftests/vfio/scripts/setup.sh/setup.sh
Here too, also the extra '.' before the path looks like a mistake.
Quoted 11 lines
No available supported devices found on the system.
Example of manual device setup:
$ ./tools/testing/selftests/vfio/scripts/setup.sh -d 0000:6a:01.0 + echo "0000:6a:01.0" > /sys/bus/pci/drivers/idxd/unbind + echo "vfio-pci" > /sys/bus/pci/devices/0000:6a:01.0/driver_override + echo "0000:6a:01.0" > /sys/bus/pci/drivers/vfio-pci/bind Successfully set up 0000:6a:01.0Signed-off-by Josh Hilke <[email protected]>
Please add Suggested-by tag for Sean and also Cc Sean on next version of the series :)
Quoted 17 lines
--- tools/testing/selftests/vfio/scripts/setup.sh | 22 +++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)tools/testing/selftests/vfio/scripts/setup.sh +5
-0@@ -19,11 +19,30 @@ function print_supported_devices() { done } +function pick_device() { + local bdf + + while read -r bdf; do + if [ -n "${bdf}" ]; then
tools/testing/selftests/vfio/scripts/setup.sh function print_supported_devices() { · line 36
Quoted 10 lines
tools/testing/selftests/vfio/scripts/setup.sh +10
-0+ if [ -d "${DEVICES_DIR}/${bdf}" ]; then + echo "${bdf} has already been set up, exiting." >&2 + exit 0 + fi + echo "${bdf}" + return 0 + fi + done <<< "$(print_supported_devices)" + + echo "No available supported devices found on the system." >&2
I think we should make it clear what "supported" means in this log. This is about finding a device that has a VFIO selftests driver, which is only required by some tests.
Same goes for the -l option in the previous commit actually and the usage message below.
Quoted 13 lines
tools/testing/selftests/vfio/scripts/setup.sh +4
-0+ exit 1 +} + function usage() { echo "usage: $0 [-l] [-d <segment:bus:device.function>]" >&2 echo "" >&2 echo " -l List segment:bus:device.function numbers of supported devices." >&2 echo " -d segment:bus:device.function to set up." >&2 + echo " If -d is not specified, a device will be automatically picked." >&2 } function main() {
tools/testing/selftests/vfio/scripts/setup.sh function main() { · line 68
Quoted 6 lines
tools/testing/selftests/vfio/scripts/setup.sh +1
-2@@ -46,8 +65,7 @@ function main() { done if [ ${#bdf_list[@]} -eq 0 ]; then - usage - exit 1 + bdf_list=($(pick_device))
Sashiko thinks this will ignore any errors from pick_device():
https://sashiko.dev/#/patchset/20260409015139.2800185-1-jrhilke%40google.com?part=2
Quoted 6 lines
tools/testing/selftests/vfio/scripts/setup.sh +0
-0fi for device_bdf in "${bdf_list[@]}"; doSignature · 1 line
2.53.0.1213.gd9a14994de-goog
David Matlack
maintainerRe: [PATCH 2/2] vfio: selftests: Automate setting up devices for testing
In reply to Josh HilkeQuoted 3 lines · Josh Hilke
Update the VFIO selftest setup script to find a device that has a VFIO selftest driver on the system, and set it up (i.e unbind that device from its driver, and then bind that device to VFIO).
I am leaning toward putting this behind an explicit flag (maybe -p for pick_device or -a for automatic device setup?) that way we are sure the user is ok with letting the script start messing with devices on their system.
Sean Christopherson
reviewerRe: [PATCH 2/2] vfio: selftests: Automate setting up devices for testing
Quoted 9 lines · David Matlack
Quoted 3 lines · Josh Hilke
Update the VFIO selftest setup script to find a device that has a VFIO selftest driver on the system, and set it up (i.e unbind that device from its driver, and then bind that device to VFIO).
I am leaning toward putting this behind an explicit flag (maybe -p for
pick_deviceor -a for automatic device setup?) that way we are sure the user is ok with letting the script start messing with devices on their system.
In the spirit of "do one thing and do it well", I would rather put "discovery" in a separate script entirely, and then tweak setup.sh as necessary to allow piping in a BDF from the discovery script.
The mechanics of configuring a device for VFIO (setup.sh) are largely generic, i.e. not unique to the selftests, whereas print_supported_devices() is very much tightly coupled to selftests.
And with respect to making sure the user actually wants the script to mess with their system, the permissions required for each script can and should be different. lspci, and discovery in general, typically doesn't require root, whereas most systems will be configured to have things like "sriov_numvfs" writable only from root.
Re: [PATCH 2/2] vfio: selftests: Automate setting up devices for testing
Quoted 20 lines · Sean Christopherson
Quoted 9 lines · David Matlack
Quoted 3 lines · Josh Hilke
Update the VFIO selftest setup script to find a device that has a VFIO selftest driver on the system, and set it up (i.e unbind that device from its driver, and then bind that device to VFIO).
I am leaning toward putting this behind an explicit flag (maybe -p for
pick_deviceor -a for automatic device setup?) that way we are sure the user is ok with letting the script start messing with devices on their system.In the spirit of "do one thing and do it well", I would rather put "discovery" in a separate script entirely, and then tweak
setup.shas necessary to allow piping in a BDF from the discovery script.The mechanics of configuring a device for VFIO (
setup.sh) are largely generic, i.e. not unique to the selftests, whereasprint_supported_devices()is very much tightly coupled to selftests.
Quoted 5 lines
And with respect to making sure the user actually wants the script to mess with their system, the permissions required for each script can and should be different.
lspci, and discovery in general, typically doesn't require root, whereas most systems will be configured to have things like "sriov_numvfs" writable only from root.
I'll put device discovery in a separate script for v3.
Regarding David's feedback, I'll also use -p for automatic device setup, and make the phrasing for "supported devices" more precise.