vfio: selftest: Add SR-IOV UAPI test
Subsystems (4)
cover vfio: selftest: Add SR-IOV UAPI test
Hello,
This series adds a vfio selftest, vfio_pci_sriov_uapi_test.c, to get some coverage on SR-IOV UAPI handling. Specifically, it includes the following cases that iterates over all the iommu modes:
- Setting correct/incorrect/NULL tokens during device init.
- Close the PF device immediately after setting the token.
- Change/override the PF's token after device init.
The test takes care of creating/setting up the VF device, and hence, it can be executed like any other test, simply by passing the PF's BDF to run.sh. For example,
./run.sh -d 0000:16:00.1 -- ./vfio_pci_sriov_uapi_test
+ echo "0" > /sys/bus/pci/devices/0000:16:00.1/sriov_numvfsdddd
+ echo "vfio-pci" > /sys/bus/pci/devices/0000:16:00.1/driver_override
+ echo "0000:16:00.1" > /sys/bus/pci/drivers/vfio-pci/bind
TAP version 13
1..45
Starting 45 tests from 15 test cases.
RUN vfio_pci_sriov_uapi_test.vfio_type1_iommu_same_uuid.init_token_match
OK vfio_pci_sriov_uapi_test.vfio_type1_iommu_same_uuid.init_token_match
ok 1 vfio_pci_sriov_uapi_test.vfio_type1_iommu_same_uuid.init_token_match
RUN vfio_pci_sriov_uapi_test.vfio_type1_iommu_same_uuid.pf_early_close
OK vfio_pci_sriov_uapi_test.vfio_type1_iommu_same_uuid.pf_early_close
ok 2 vfio_pci_sriov_uapi_test.vfio_type1_iommu_same_uuid.pf_early_close
RUN vfio_pci_sriov_uapi_test.vfio_type1_iommu_same_uuid.override_token
OK vfio_pci_sriov_uapi_test.vfio_type1_iommu_same_uuid.override_token
[...]
RUN vfio_pci_sriov_uapi_test.iommufd_null_uuid.override_token ...
OK vfio_pci_sriov_uapi_test.iommufd_null_uuid.override_token
ok 45 vfio_pci_sriov_uapi_test.iommufd_null_uuid.override_token
PASSED: 45 / 45 tests passed.
The series this dependent on another series that provides fixes in the IOMMUFD's vf_token handling [1].
Thank you.
Raghavendra
[1]: https://lore.kernel.org/all/[email protected]/
Raghavendra Rao Ananta (4):
vfio: selftests: Add support for passing vf_token in device init
vfio: selftests: Export vfio_pci_device functions
vfio: selftests: Add helper to set/override a vf_token
vfio: selftests: Add tests to validate SR-IOV UAPI
tools/testing/selftests/vfio/Makefile | 1 + .../selftests/vfio/lib/include/vfio_util.h | 19 +- tools/testing/selftests/vfio/lib/libvfio.mk | 4 +- .../selftests/vfio/lib/vfio_pci_device.c | 151 ++++++++++-- .../selftests/vfio/vfio_dma_mapping_test.c | 2 +- .../selftests/vfio/vfio_pci_device_test.c | 4 +- .../selftests/vfio/vfio_pci_driver_test.c | 4 +- .../selftests/vfio/vfio_pci_sriov_uapi_test.c | 220 ++++++++++++++++++ 8 files changed, 377 insertions(+), 28 deletions(-) create mode 100644 tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c
base-commit: 211ddde0823f1442e4ad052a2f30f050145ccada
Signature · 1 line
2.51.2.997.g839fc31de9-goog
PATCH 1/4 vfio: selftests: Add support for passing vf_token in device init
A UUID is normally set as a vf_token to correspond the VFs with the PFs, if they are both bound by the vfio-pci driver. This is true for iommufd-based approach and container-based approach. The token can be set either during device creation (VFIO_GROUP_GET_DEVICE_FD) in container-based approach or during iommu bind (VFIO_DEVICE_BIND_IOMMUFD) in the iommu-fd case. Hence, extend the vfio_pci_device_init() helper to accept vf_token during device setup.
The tests depending on vfio_pci_device_init() are adjusted accordingly and no functional changes are expected. A later patch will add tests that passes actual token to test the UAPI.
--- .../selftests/vfio/lib/include/vfio_util.h | 4 +- tools/testing/selftests/vfio/lib/libvfio.mk | 4 +- .../selftests/vfio/lib/vfio_pci_device.c | 60 ++++++++++++++++--- .../selftests/vfio/vfio_dma_mapping_test.c | 2 +- .../selftests/vfio/vfio_pci_device_test.c | 4 +- .../selftests/vfio/vfio_pci_driver_test.c | 4 +- 6 files changed, 62 insertions(+), 16 deletions(-)
tools/testing/selftests/vfio/lib/include/vfio_util.h +3 -1
@@ -202,7 +202,9 @@ const char *vfio_pci_get_cdev_path(const char *bdf);extern const char *default_iommu_mode; -struct vfio_pci_device *vfio_pci_device_init (const char *bdf,const char *iommu_mode); +struct vfio_pci_device *vfio_pci_device_init (const char *bdf, +const char *iommu_mode, +const char *vf_token);void vfio_pci_device_cleanup (struct vfio_pci_device *device);void vfio_pci_device_reset (struct vfio_pci_device *device);
tools/testing/selftests/vfio/lib/libvfio.mk +3 -1
@@ -18,7 +18,9 @@ $(shell mkdir -p $(LIBVFIO_O_DIRS)) CFLAGS += -I$(VFIO_DIR)/lib/include +LDLIBS += -luuid + $(LIBVFIO_O): $(OUTPUT)/%.o : $(VFIO_DIR)/%.c - $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@ + $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< $(LDLIBS) -o $@ EXTRA_CLEAN += $(LIBVFIO_O)
tools/testing/selftests/vfio/lib/vfio_pci_device.c +51 -9
118 lines not on this page · raw message
tools/testing/selftests/vfio/vfio_dma_mapping_test.c +1 -1
@@ -114,7 +114,7 @@ FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES(anonymous_hugetlb_1gb, SZ_1G, MAP_HUGETLB |FIXTURE_SETUP (vfio_dma_mapping_test) { - self->device =vfio_pci_device_init (device_bdf, variant->iommu_mode); + self->device =vfio_pci_device_init (device_bdf, variant->iommu_mode,NULL ); }FIXTURE_TEARDOWN (vfio_dma_mapping_test)
tools/testing/selftests/vfio/vfio_pci_device_test.c +2 -2
@@ -28,7 +28,7 @@ FIXTURE(vfio_pci_device_test) {FIXTURE_SETUP (vfio_pci_device_test) { - self->device =vfio_pci_device_init (device_bdf, default_iommu_mode); + self->device =vfio_pci_device_init (device_bdf, default_iommu_mode,NULL ); }FIXTURE_TEARDOWN (vfio_pci_device_test)
@@ -116,7 +116,7 @@ FIXTURE_VARIANT_ADD(vfio_pci_irq_test, msix) {FIXTURE_SETUP (vfio_pci_irq_test) { - self->device =vfio_pci_device_init (device_bdf, default_iommu_mode); + self->device =vfio_pci_device_init (device_bdf, default_iommu_mode,NULL ); }FIXTURE_TEARDOWN (vfio_pci_irq_test)
tools/testing/selftests/vfio/vfio_pci_driver_test.c +2 -2
@@ -71,7 +71,7 @@ FIXTURE_SETUP(vfio_pci_driver_test) {struct vfio_pci_driver *driver; - self->device =vfio_pci_device_init (device_bdf, variant->iommu_mode); + self->device =vfio_pci_device_init (device_bdf, variant->iommu_mode,NULL ); driver = &self->device->driver;
@@ -233,7 +233,7 @@ int main(int argc, char *argv[]) device_bdf =vfio_selftests_get_bdf (&argc, argv); - device =vfio_pci_device_init (device_bdf, default_iommu_mode); + device =vfio_pci_device_init (device_bdf, default_iommu_mode,NULL );if (!device->driver.ops) {fprintf (stderr,"No driver found for device %s\n" , device_bdf);return KSFT_SKIP ;
Signature · 1 line
2.51.2.997.g839fc31de9-goog
David Matlack
maintainerRe: [PATCH 1/4] vfio: selftests: Add support for passing vf_token in device init
Review
Quoted 4 lines · Raghavendra Rao Ananta
(quoted diff) +3
-1-struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode); +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, + const char *iommu_mode, + const char *vf_token);
Vipin is also looking at adding an optional parameter to vfio_pci_device_init():
https://lore.kernel.org/kvm/[email protected]/
I am wondering if we should support an options struct for such parameters. e.g. something like this
tools/testing/selftests/vfio/lib/include/vfio_util.h +16 -3
@@ -160,6 +160,10 @@ struct vfio_pci_driver {int msi; }; +struct vfio_pci_device_options { +const char *vf_token; +}; +struct vfio_pci_device {int fd;
@@ -202,9 +206,18 @@ const char *vfio_pci_get_cdev_path(const char *bdf);extern const char *default_iommu_mode; -struct vfio_pci_device *vfio_pci_device_init (const char *bdf, -const char *iommu_mode, -const char *vf_token); +struct vfio_pci_device *__vfio_pci_device_init (const char *bdf, +const char *iommu_mode, +const struct vfio_pci_device_options *options); + +static inline struct vfio_pci_device *vfio_pci_device_init (const char *bdf, +const char *iommu_mode) +{ +static const struct vfio_pci_device_options default_options = {}; + +return __vfio_pci_device_init (bdf, iommu_mode, &default_options); +} +
This will avoid you having to update every test.
You can create a helper function in vfio_pci_sriov_uapi_test.c to call __vfio_pci_device_init() and abstract away the options stuff from your test.
Review
Quoted 2 lines
(quoted diff) +1
-1-static void vfio_pci_container_setup(struct vfio_pci_device *device, const char *bdf) +static void vfio_pci_container_get_device_fd(struct vfio_pci_device *device,
Let's name this vfio_pci_group_get_device_fd() since it's getting the device FD from the group using ioctl(VFIO_GROUP_GET_DEVICE_FD).
Review
Quoted 2 lines
(quoted diff) +2
-0+ const char *bdf, + const char *vf_token)
There's an extra space before these arguments. Align them all vertically with the first argument.
I noticed this throughout this patch, so please fix throughout the whole series in v2.
Also please run checkpatch.pl. It will catch things like this for you.
CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #78: FILE: tools/testing/selftests/vfio/lib/vfio_pci_device.c:335:
+static void vfio_pci_container_get_device_fd(struct vfio_pci_device *device,
+ const char *bdf,
Review
Quoted 2 lines
(quoted diff) +2
-0+{ + char *arg = (char *) bdf;
No space necessary after a cast. This is another one checkpatch.pl will catch for you.
CHECK:SPACING: No space is necessary after a cast
#81: FILE: tools/testing/selftests/vfio/lib/vfio_pci_device.c:338:
+ char *arg = (char *) bdf;
Quoted 5 lines
(quoted diff) +5
-0+ + /* + * If a vf_token exists, argument to VFIO_GROUP_GET_DEVICE_FD + * will be in the form of the following example: + * "0000:04:10.0 vf_token=bd8d9d2b-5a5f-4f5a-a211-f591514ba1f3"
Review
Quoted 10 lines
(quoted diff) +10
-0+ */ + if (vf_token) { + size_t sz = strlen(bdf) + strlen(" "VF_TOKEN_ARG) + + strlen(vf_token) + 1; + + arg = calloc(1, sz); + VFIO_ASSERT_NOT_NULL(arg); + + snprintf(arg, sz, "%s %s%s", bdf, VF_TOKEN_ARG, vf_token); + }
UUIDs are 16 bytes so I think we could create a pretty reasonably fixed size array to hold the argument and simplify this code, make it more self-documenting, and eliminate VF_TOKEN_ARG. This is test code, so we can make the array bigger in the future if we need to. Keeping the code simple is more important IMO.
static void vfio_pci_container_get_device_fd(struct vfio_pci_device *device,
const char *bdf, const char *vf_token)
{
char arg[64];
if (vf_token)
snprintf(arg, ARRAY_SIZE(arg), "%s vf_token=%s", bdf, vf_token);
else
snprintf(arg, ARRAY_SIZE(arg), "%s", bdf);
device->fd = ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, arg);
VFIO_ASSERT_GE(device->fd, 0);
}And to protect against writing off the end of arg, we can introduce a snprintf_assert() in a separate commit. There are actually a few snprintf() calls in vfio_pci_device.c that would be nice to convert to snprintf_assert().
#define snprintf_assert(_s, _size, _fmt, ...) do { \
int __ret = snprintf(_s, _size, _fmt, ##__VA_ARGS__); \
VFIO_ASSERT_LT(__ret, _size); \} while (0)
snprintf_assert() could be added in a precursor commit to this one.
Review
Quoted 11 lines
(quoted diff) +6
-0+static void vfio_device_bind_iommufd(int device_fd, int iommufd, const char *vf_token) { struct vfio_device_bind_iommufd args = { .argsz = sizeof(args), .iommufd = iommufd, }; + uuid_t token_uuid = {0}; + + if (vf_token) { + VFIO_ASSERT_EQ(uuid_parse(vf_token, token_uuid), 0); + args.flags = VFIO_DEVICE_BIND_FLAG_TOKEN;
Maybe make this |= instead of = ? I had to double-check that this wasn't overwriting any flags set above this. Obviously it's not since this is a small function, but |= would make that super obvious.
David Matlack
maintainerRe: [PATCH 1/4] vfio: selftests: Add support for passing vf_token in device init
Quoted 11 lines · David Matlack
Review
Quoted 4 lines · Raghavendra Rao Ananta
(quoted diff) +3
-1-struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode); +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, + const char *iommu_mode, + const char *vf_token);Vipin is also looking at adding an optional parameter to
vfio_pci_device_init():
https://lore.kernel.org/kvm/[email protected]/
Quoted 2 lines
I am wondering if we should support an options struct for such parameters. e.g. something like this
Wait, patch 4 doesn't even use vfio_pci_device_init(). Do we need this commit? It seems like we just need some of the inner functions to have support for vf_token.
Re: [PATCH 1/4] vfio: selftests: Add support for passing vf_token in device init
Quoted 13 lines · David Matlack
Quoted 11 lines · David Matlack
Review
Quoted 4 lines · Raghavendra Rao Ananta
(quoted diff) +3
-1-struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode); +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, + const char *iommu_mode, + const char *vf_token);Vipin is also looking at adding an optional parameter to
vfio_pci_device_init():
https://lore.kernel.org/kvm/[email protected]/
Quoted 6 lines
Quoted 2 lines
I am wondering if we should support an options struct for such parameters. e.g. something like this
Wait, patch 4 doesn't even use
vfio_pci_device_init(). Do we need this commit? It seems like we just need some of the inner functions to have support forvf_token.
Gah, that's my bad. I changed the approach later but forgot to revert the API. I'll fix it in v2.
Re: [PATCH 1/4] vfio: selftests: Add support for passing vf_token in device init
In reply to David MatlackHi David,
Quoted 52 lines · David Matlack
Review
Quoted 4 lines · Raghavendra Rao Ananta
(quoted diff) +3
-1-struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode); +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, + const char *iommu_mode, + const char *vf_token);Vipin is also looking at adding an optional parameter to
vfio_pci_device_init():
https://lore.kernel.org/kvm/[email protected]/I am wondering if we should support an options struct for such parameters. e.g. something like this
tools/testing/selftests/vfio/lib/include/vfio_util.h +16
-3@@ -160,6 +160,10 @@ struct vfio_pci_driver {int msi; }; +struct vfio_pci_device_options { +const char *vf_token; +}; +struct vfio_pci_device {int fd;@@ -202,9 +206,18 @@ const char *vfio_pci_get_cdev_path(const char *bdf);extern const char *default_iommu_mode; -struct vfio_pci_device *vfio_pci_device_init (const char *bdf, -const char *iommu_mode, -const char *vf_token); +struct vfio_pci_device *__vfio_pci_device_init (const char *bdf, +const char *iommu_mode, +const struct vfio_pci_device_options *options); + +static inline struct vfio_pci_device *vfio_pci_device_init (const char *bdf, +const char *iommu_mode) +{ +static const struct vfio_pci_device_options default_options = {}; + +return __vfio_pci_device_init (bdf, iommu_mode, &default_options); +} +This will avoid you having to update every test.
Quoted 4 lines
You can create a helper function in
vfio_pci_sriov_uapi_test.cto call__vfio_pci_device_init()and abstract away the options stuff from your test.
I like the idea of an optional expandable struct. I'll implement this in v2.
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +1
-1-static void vfio_pci_container_setup(struct vfio_pci_device *device, const char *bdf) +static void vfio_pci_container_get_device_fd(struct vfio_pci_device *device,Let's name this
vfio_pci_group_get_device_fd()since it's getting the device FD from the group usingioctl(VFIO_GROUP_GET_DEVICE_FD).
Quoted 9 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ const char *bdf, + const char *vf_token)There's an extra space before these arguments. Align them all vertically with the first argument.
I noticed this throughout this patch, so please fix throughout the whole series in v2.
I may have to fix my editor to display this right. Thanks for catching this.
Quoted 10 lines
Also please run checkpatch.pl. It will catch things like this for you.
CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #78: FILE:
tools/testing/selftests/vfio/lib/vfio_pci_device.c:335:
+static void vfio_pci_container_get_device_fd(structvfio_pci_device*device,
+ const char *bdf,Review
Quoted 2 lines
(quoted diff) +2
-0+{ + char *arg = (char *) bdf;
Quoted 7 lines
No space necessary after a cast. This is another one checkpatch.pl will catch for you.
CHECK:SPACING: No space is necessary after a cast
#81: FILE:tools/testing/selftests/vfio/lib/vfio_pci_device.c:338:
+ char *arg = (char *) bdf;
Actually, I did run checkpatch.pl on the entire series as:
.$ ./scripts/checkpatch.pl *.patch
I didn't see any of these warnings. Are there any other options to consider?
Quoted 25 lines
Quoted 5 lines
(quoted diff) +5
-0+ + /* + * If a vf_token exists, argument to VFIO_GROUP_GET_DEVICE_FD + * will be in the form of the following example: + * "0000:04:10.0 vf_token=bd8d9d2b-5a5f-4f5a-a211-f591514ba1f3"Review
Quoted 10 lines
(quoted diff) +10
-0+ */ + if (vf_token) { + size_t sz = strlen(bdf) + strlen(" "VF_TOKEN_ARG) + + strlen(vf_token) + 1; + + arg = calloc(1, sz); + VFIO_ASSERT_NOT_NULL(arg); + + snprintf(arg, sz, "%s %s%s", bdf, VF_TOKEN_ARG, vf_token); + }UUIDs are 16 bytes so I think we could create a pretty reasonably fixed size array to hold the argument and simplify this code, make it more self-documenting, and eliminate
VF_TOKEN_ARG. This is test code, so we can make the array bigger in the future if we need to. Keeping the code simple is more important IMO.static void vfio_pci_container_get_device_fd(struct
vfio_pci_device*device,
const char *bdf, const char*vf_token)
Quoted 12 lines
{
char arg[64];
if (vf_token)
snprintf(arg, ARRAY_SIZE(arg), "%s vf_token=%s", bdf, vf_token);
else
snprintf(arg, ARRAY_SIZE(arg), "%s", bdf);
device->fd = ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, arg);
VFIO_ASSERT_GE(device->fd, 0);
}Yeah, this is a good idea! I'll implement it in v2.
Quoted 5 lines
And to protect against writing off the end of arg, we can introduce a
snprintf_assert()in a separate commit. There are actually a fewsnprintf()calls invfio_pci_device.cthat would be nice to convert tosnprintf_assert().
Quoted 7 lines
#define snprintf_assert(_s, _size, _fmt, ...) do { \ int __ret = snprintf(_s, _size, _fmt, ##__VA_ARGS__); \ VFIO_ASSERT_LT(__ret, _size); \} while (0)
snprintf_assert()could be added in a precursor commit to this one.
Sounds good. I'll add a commit for this and convert all the snprintf()s to snprintf_assert()s.
Quoted 7 lines
Quoted 7 lines
(quoted diff) +2
-0+static void vfio_device_bind_iommufd(int device_fd, int iommufd, const char *vf_token) { struct vfio_device_bind_iommufd args = { .argsz = sizeof(args), .iommufd = iommufd, }; + uuid_t token_uuid = {0};
Quoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ + if (vf_token) { + VFIO_ASSERT_EQ(uuid_parse(vf_token, token_uuid), 0); + args.flags = VFIO_DEVICE_BIND_FLAG_TOKEN;Maybe make this |= instead of = ? I had to double-check that this wasn't overwriting any flags set above this. Obviously it's not since this is a small function, but |= would make that super obvious.
Oh yes, absolutely!
Thank you.
Raghavendra
David Matlack
maintainerRe: [PATCH 1/4] vfio: selftests: Add support for passing vf_token in device init
Quoted 50 lines · Raghavendra Rao Ananta
Quoted 49 lines · David Matlack
Review
Quoted 4 lines · Raghavendra Rao Ananta
(quoted diff) +3
-1-struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode); +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, + const char *iommu_mode, + const char *vf_token);Vipin is also looking at adding an optional parameter to
vfio_pci_device_init():
https://lore.kernel.org/kvm/[email protected]/I am wondering if we should support an options struct for such parameters. e.g. something like this
tools/testing/selftests/vfio/lib/include/vfio_util.h +16
-3@@ -160,6 +160,10 @@ struct vfio_pci_driver {int msi; }; +struct vfio_pci_device_options { +const char *vf_token; +}; +struct vfio_pci_device {int fd;@@ -202,9 +206,18 @@ const char *vfio_pci_get_cdev_path(const char *bdf);extern const char *default_iommu_mode; -struct vfio_pci_device *vfio_pci_device_init (const char *bdf, -const char *iommu_mode, -const char *vf_token); +struct vfio_pci_device *__vfio_pci_device_init (const char *bdf, +const char *iommu_mode, +const struct vfio_pci_device_options *options); + +static inline struct vfio_pci_device *vfio_pci_device_init (const char *bdf, +const char *iommu_mode) +{ +static const struct vfio_pci_device_options default_options = {}; + +return __vfio_pci_device_init (bdf, iommu_mode, &default_options); +} +
Quoted 7 lines
Quoted 6 lines
This will avoid you having to update every test.
You can create a helper function in
vfio_pci_sriov_uapi_test.cto call__vfio_pci_device_init()and abstract away the options stuff from your test.I like the idea of an optional expandable struct. I'll implement this in v2.
Just to make sure we're on the same page: I don't think you need to add this in v2 since you don't need to call vfio_pci_device_init(). For the inner functions that you want to call from your test, passing vf_token directly makes more sense IMO. vfio_pci_device_init() will just pass in NULL to those functions for vf_token by default.
If/when we want to pass vf_token to vfio_pci_device_init() we can add the options struct.
Quoted 11 lines
Quoted 7 lines
No space necessary after a cast. This is another one checkpatch.pl will catch for you.
CHECK:SPACING: No space is necessary after a cast
#81: FILE:tools/testing/selftests/vfio/lib/vfio_pci_device.c:338:
+ char *arg = (char *) bdf;Actually, I did run checkpatch.pl on the entire series as:
.$./scripts/checkpatch.pl*.patchI didn't see any of these warnings. Are there any other options to consider?
Ah, I run with a few additional options. That's probably why we are seeing different output. Here's what I have in my .bashrc:
function checkpatch() {
scripts/checkpatch.pl \
-q \
--strict \
--codespell \
--no-signoff \
--show-types \
--ignore gerrit_change_id,FILE_PATH_CHANGES,NOT_UNIFIED_DIFF \
--no-summary \
"$@"
}Re: [PATCH 1/4] vfio: selftests: Add support for passing vf_token in device init
Quoted 60 lines · David Matlack
Quoted 50 lines · Raghavendra Rao Ananta
Quoted 49 lines · David Matlack
Review
Quoted 4 lines · Raghavendra Rao Ananta
(quoted diff) +3
-1-struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode); +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, + const char *iommu_mode, + const char *vf_token);Vipin is also looking at adding an optional parameter to
vfio_pci_device_init():
https://lore.kernel.org/kvm/[email protected]/I am wondering if we should support an options struct for such parameters. e.g. something like this
tools/testing/selftests/vfio/lib/include/vfio_util.h +16
-3@@ -160,6 +160,10 @@ struct vfio_pci_driver {int msi; }; +struct vfio_pci_device_options { +const char *vf_token; +}; +struct vfio_pci_device {int fd;@@ -202,9 +206,18 @@ const char *vfio_pci_get_cdev_path(const char *bdf);extern const char *default_iommu_mode; -struct vfio_pci_device *vfio_pci_device_init (const char *bdf, -const char *iommu_mode, -const char *vf_token); +struct vfio_pci_device *__vfio_pci_device_init (const char *bdf, +const char *iommu_mode, +const struct vfio_pci_device_options *options); + +static inline struct vfio_pci_device *vfio_pci_device_init (const char *bdf, +const char *iommu_mode) +{ +static const struct vfio_pci_device_options default_options = {}; + +return __vfio_pci_device_init (bdf, iommu_mode, &default_options); +} +Quoted 7 lines
Quoted 6 lines
This will avoid you having to update every test.
You can create a helper function in
vfio_pci_sriov_uapi_test.cto call__vfio_pci_device_init()and abstract away the options stuff from your test.I like the idea of an optional expandable struct. I'll implement this in v2.
Quoted 9 lines
Just to make sure we're on the same page: I don't think you need to add this in v2 since you don't need to call
vfio_pci_device_init(). For the inner functions that you want to call from your test, passingvf_tokendirectly makes more sense IMO.vfio_pci_device_init()will just pass in NULL to those functions forvf_tokenby default.If/when we want to pass
vf_tokentovfio_pci_device_init()we can add the options struct.
Yes, of course. I'll avoid it until we have a need.
Quoted 15 lines
Quoted 11 lines
Quoted 7 lines
No space necessary after a cast. This is another one checkpatch.pl will catch for you.
CHECK:SPACING: No space is necessary after a cast
#81: FILE:tools/testing/selftests/vfio/lib/vfio_pci_device.c:338:
+ char *arg = (char *) bdf;Actually, I did run checkpatch.pl on the entire series as:
.$./scripts/checkpatch.pl*.patchI didn't see any of these warnings. Are there any other options to consider?
Ah, I run with a few additional options. That's probably why we are seeing different output. Here's what I have in my .bashrc:
Quoted 11 lines
function checkpatch() {
scripts/checkpatch.pl \
-q \
--strict \
--codespell \
--no-signoff \
--show-types \
--ignore gerrit_change_id,FILE_PATH_CHANGES,NOT_UNIFIED_DIFF \
--no-summary \
"$@"
}Thanks! I'll give this a shot!
David Matlack
maintainerRe: [PATCH 1/4] vfio: selftests: Add support for passing vf_token in device init
In reply to Raghavendra Rao Anantatools/testing/selftests/vfio/lib/libvfio.mk $(shell mkdir -p $(LIBVFIO_O_DIRS)) · line 21
Quoted 9 lines · Raghavendra Rao Ananta
tools/testing/selftests/vfio/lib/libvfio.mk +1
-0@@ -18,7 +18,9 @@ $(shell mkdir -p $(LIBVFIO_O_DIRS)) CFLAGS += -I$(VFIO_DIR)/lib/include +LDLIBS += -luuid
I wonder if we really need this dependency. VFIO and IOMMUFD just expect a 16 byte character array. That is easy enough to represent. The other part we use is uuid_parse(), but I don't know if selftests need to do that validation. We can let VFIO and IOMMUFD validate the UUID as they see fit and return an error if they aren't happy with it. i.e. We do not need to duplicate validation in the test.
Re: [PATCH 1/4] vfio: selftests: Add support for passing vf_token in device init
Quoted 11 lines · David Matlack
Quoted 8 lines · Raghavendra Rao Ananta
tools/testing/selftests/vfio/lib/libvfio.mk +0
-0@@ -18,7 +18,9 @@ $(shell mkdir -p $(LIBVFIO_O_DIRS)) CFLAGS += -I$(VFIO_DIR)/lib/include
Quoted 8 lines
tools/testing/selftests/vfio/lib/libvfio.mk $(shell mkdir -p $(LIBVFIO_O_DIRS)) · line 21
Quoted 1 line
tools/testing/selftests/vfio/lib/libvfio.mk +1
-0+LDLIBS += -luuidI wonder if we really need this dependency. VFIO and IOMMUFD just expect a 16 byte character array. That is easy enough to represent. The other part we use is
uuid_parse(), but I don't know if selftests need to do that validation. We can let VFIO and IOMMUFD validate the UUID as they see fit and return an error if they aren't happy with it. i.e. We do not need to duplicate validation in the test.
Unfortunately, VFIO interface accepts UUID in multiple formats. For VFIO_DEVICE_FEATURE and VFIO_DEVICE_BIND_IOMMUFD it accepts a 'u8[16]', but for VFIO_GROUP_GET_DEVICE_FD, we must present it as a string. Is there an issue with the inclusion of an external library (I think I've seen others in tools/ use it).
Thank you.
Raghavendra
David Matlack
maintainerRe: [PATCH 1/4] vfio: selftests: Add support for passing vf_token in device init
Quoted 21 lines · Raghavendra Rao Ananta
Quoted 11 lines · David Matlack
Quoted 8 lines · Raghavendra Rao Ananta
tools/testing/selftests/vfio/lib/libvfio.mk +0
-0@@ -18,7 +18,9 @@ $(shell mkdir -p $(LIBVFIO_O_DIRS)) CFLAGS += -I$(VFIO_DIR)/lib/includeQuoted 8 lines
tools/testing/selftests/vfio/lib/libvfio.mk $(shell mkdir -p $(LIBVFIO_O_DIRS)) · line 21
Quoted 1 line
tools/testing/selftests/vfio/lib/libvfio.mk +1
-0+LDLIBS += -luuidI wonder if we really need this dependency. VFIO and IOMMUFD just expect a 16 byte character array. That is easy enough to represent. The other part we use is
uuid_parse(), but I don't know if selftests need to do that validation. We can let VFIO and IOMMUFD validate the UUID as they see fit and return an error if they aren't happy with it. i.e. We do not need to duplicate validation in the test.
Quoted 5 lines
Unfortunately, VFIO interface accepts UUID in multiple formats. For
VFIO_DEVICE_FEATUREandVFIO_DEVICE_BIND_IOMMUFDit accepts a 'u8[16]', but forVFIO_GROUP_GET_DEVICE_FD, we must present it as a string. Is there an issue with the inclusion of an external library (I think I've seen others intools/use it).
Ack. In that case, depending on libuuid SGTM.
I mistakenly thought VFIO always took the UUID as a string and uuid_parse() was just being used to sanity check the format. Thanks for clarifying.
PATCH 2/4 vfio: selftests: Export vfio_pci_device functions
In reply to Raghavendra Rao AnantaRefactor and make the functions called under device initialization public. A later patch adds a test that calls these functions to validate the UAPI of SR-IOV devices. Opportunistically, to test the success and failure cases of the UAPI, split the functions dealing with VFIO_GROUP_GET_DEVICE_FD and VFIO_DEVICE_BIND_IOMMUFD into a core function and another one that asserts the ioctl. The former will be used for testing the SR-IOV UAPI, hence only export these.
--- .../selftests/vfio/lib/include/vfio_util.h | 13 ++++ .../selftests/vfio/lib/vfio_pci_device.c | 75 +++++++++++++------ 2 files changed, 67 insertions(+), 21 deletions(-)
tools/testing/selftests/vfio/lib/include/vfio_util.h +13 -0
@@ -294,4 +294,17 @@ void vfio_pci_driver_memcpy_start(struct vfio_pci_device *device,int vfio_pci_driver_memcpy_wait (struct vfio_pci_device *device);void vfio_pci_driver_send_msi (struct vfio_pci_device *device); +const char *iommu_mode_container_path (const char *iommu_mode); +const struct vfio_iommu_mode *lookup_iommu_mode (const char *iommu_mode); + +void vfio_container_open (struct vfio_pci_device *device); +void vfio_pci_group_setup (struct vfio_pci_device *device,const char *bdf); +void vfio_container_set_iommu (struct vfio_pci_device *device); +void __vfio_container_get_device_fd (struct vfio_pci_device *device, +const char *bdf,const char *vf_token); + +void vfio_pci_iommufd_cdev_open (struct vfio_pci_device *device,const char *bdf); +void vfio_pci_iommufd_iommudev_open (struct vfio_pci_device *device); +int __vfio_device_bind_iommufd (int device_fd,int iommufd,const char *vf_token); +#endif /* SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H */
tools/testing/selftests/vfio/lib/vfio_pci_device.c +54 -21
146 lines not on this page · raw message
Signature · 1 line
2.51.2.997.g839fc31de9-goog
David Matlack
maintainerRe: [PATCH 2/4] vfio: selftests: Export vfio_pci_device functions
Quoted 7 lines · Raghavendra Rao Ananta
Refactor and make the functions called under device initialization public. A later patch adds a test that calls these functions to validate the UAPI of SR-IOV devices. Opportunistically, to test the success and failure cases of the UAPI, split the functions dealing with
VFIO_GROUP_GET_DEVICE_FDandVFIO_DEVICE_BIND_IOMMUFDinto a core function and another one that asserts the ioctl. The former will be used for testing the SR-IOV UAPI, hence only export these.
I have a series that separates the IOMMU initialization and fields from struct vfio_pci_device. I suspect that will make what you are trying to do a lot easier.
https://lore.kernel.org/kvm/[email protected]/
Can you take a look at it and see if it would simplifying things for you? Reviews would be very appreciated if so :)
Re: [PATCH 2/4] vfio: selftests: Export vfio_pci_device functions
Quoted 10 lines · David Matlack
Quoted 7 lines · Raghavendra Rao Ananta
Refactor and make the functions called under device initialization public. A later patch adds a test that calls these functions to validate the UAPI of SR-IOV devices. Opportunistically, to test the success and failure cases of the UAPI, split the functions dealing with
VFIO_GROUP_GET_DEVICE_FDandVFIO_DEVICE_BIND_IOMMUFDinto a core function and another one that asserts the ioctl. The former will be used for testing the SR-IOV UAPI, hence only export these.
Quoted 6 lines
I have a series that separates the IOMMU initialization and fields from struct
vfio_pci_device. I suspect that will make what you are trying to do a lot easier.
Nice! I'll take a look at it. By the way, how do we normally deal with dependencies among series? Do we simply mention it in the cover-letter?
Quoted 2 lines
Can you take a look at it and see if it would simplifying things for you? Reviews would be very appreciated if so :)
Absolutely! Sorry, I have it on my TODO list to review the changes, but didn't get a chance. I'll get to it soon. Thanks for the reminder :)
David Matlack
maintainerRe: [PATCH 2/4] vfio: selftests: Export vfio_pci_device functions
Quoted 15 lines · Raghavendra Rao Ananta
Quoted 14 lines · David Matlack
Quoted 7 lines · Raghavendra Rao Ananta
Refactor and make the functions called under device initialization public. A later patch adds a test that calls these functions to validate the UAPI of SR-IOV devices. Opportunistically, to test the success and failure cases of the UAPI, split the functions dealing with
VFIO_GROUP_GET_DEVICE_FDandVFIO_DEVICE_BIND_IOMMUFDinto a core function and another one that asserts the ioctl. The former will be used for testing the SR-IOV UAPI, hence only export these.I have a series that separates the IOMMU initialization and fields from struct
vfio_pci_device. I suspect that will make what you are trying to do a lot easier.
Quoted 5 lines
Quoted 2 lines
Nice! I'll take a look at it. By the way, how do we normally deal with dependencies among series? Do we simply mention it in the cover-letter?
What I usually do is:
- Mention in the cover letter that this series applies on top of another series, and provide a lore link to the exact series version it's on top of.
- Upload your series (with the dependent series) somewhere pulic like GitHub so that it's easy for reviewers to check out your code without having to apply multiple different series of patches. Include a link to this in your cover letter too.
See this series from Vipin for an example of doing this:
https://lore.kernel.org/kvm/[email protected]/
Quoted 5 lines
Quoted 2 lines
Can you take a look at it and see if it would simplifying things for you? Reviews would be very appreciated if so :)
Absolutely! Sorry, I have it on my TODO list to review the changes, but didn't get a chance. I'll get to it soon. Thanks for the reminder :)
Thanks!
PATCH 3/4 vfio: selftests: Add helper to set/override a vf_token
In reply to Raghavendra Rao AnantaNot only at init, but a vf_token can also be set via the VFIO_DEVICE_FEATURE ioctl, by setting the VFIO_DEVICE_FEATURE_PCI_VF_TOKEN flag. Add an API to utilize this functionality from the test code.
--- .../selftests/vfio/lib/include/vfio_util.h | 2 ++ .../selftests/vfio/lib/vfio_pci_device.c | 34 +++++++++++++++++++ 2 files changed, 36 insertions(+)
tools/testing/selftests/vfio/lib/include/vfio_util.h +2 -0
@@ -307,4 +307,6 @@ void vfio_pci_iommufd_cdev_open(struct vfio_pci_device *device, const char *bdf)void vfio_pci_iommufd_iommudev_open (struct vfio_pci_device *device);int __vfio_device_bind_iommufd (int device_fd,int iommufd,const char *vf_token); +void vfio_device_set_vf_token (int fd,const char *vf_token); +#endif /* SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H */
tools/testing/selftests/vfio/lib/vfio_pci_device.c +34 -0
@@ -144,6 +144,40 @@ static void vfio_pci_irq_get(struct vfio_pci_device *device, u32 index,ioctl_assert (device->fd,VFIO_DEVICE_GET_IRQ_INFO , irq_info); } +static int vfio_device_feature_ioctl (int fd,u32 flags,void *data, +size_t data_size) +{ +u8 buffer[sizeof (struct vfio_device_feature ) + data_size] = {}; +struct vfio_device_feature *feature = (void *)buffer; + +memcpy (feature->data, data, data_size); + + feature->argsz =sizeof (buffer); + feature->flags = flags; + +return ioctl (fd,VFIO_DEVICE_FEATURE , feature); +} + +static void vfio_device_feature_set (int fd,u16 feature,void *data,size_t data_size) +{ +u32 flags =VFIO_DEVICE_FEATURE_SET | feature; +int ret; + + ret =vfio_device_feature_ioctl (fd, flags, data, data_size); +VFIO_ASSERT_EQ (ret,0 ,"Failed to set feature %u\n" , feature); +} + +void vfio_device_set_vf_token (int fd,const char *vf_token) +{ +uuid_t token_uuid = {0 }; + +VFIO_ASSERT_NOT_NULL (vf_token,"vf_token is NULL" ); +VFIO_ASSERT_EQ (uuid_parse (vf_token, token_uuid),0 ); + +vfio_device_feature_set (fd,VFIO_DEVICE_FEATURE_PCI_VF_TOKEN , + token_uuid,sizeof (uuid_t)); +} +static void vfio_iommu_dma_map (struct vfio_pci_device *device,struct vfio_dma_region *region) {
Signature · 1 line
2.51.2.997.g839fc31de9-goog
David Matlack
maintainerRe: [PATCH 3/4] vfio: selftests: Add helper to set/override a vf_token
Quoted 4 lines · Raghavendra Rao Ananta
Not only at init, but a
vf_tokencan also be set via theVFIO_DEVICE_FEATUREioctl, by setting theVFIO_DEVICE_FEATURE_PCI_VF_TOKENflag. Add an API to utilize this functionality from the test code.
Say what the commit does first. Then add context (e.g. compare/contrast to other ways of setting the VF token).
Also please add a sentence about how this will be used in a subsequent commit, since there are no callers in this commit.
tools/testing/selftests/vfio/lib/vfio_pci_device.c static void vfio_pci_irq_get(struct vfio_pci_device *device, u32 index, · line 174
Quoted 5 lines
tools/testing/selftests/vfio/lib/vfio_pci_device.c +5
-0+void vfio_device_set_vf_token (int fd,const char *vf_token) +{ +uuid_t token_uuid = {0 }; + +VFIO_ASSERT_NOT_NULL (vf_token,"vf_token is NULL" );
nit: The help message here is not needed. It will be very obvious that vf_token is NULL if this assert fires :)
Re: [PATCH 3/4] vfio: selftests: Add helper to set/override a vf_token
Quoted 13 lines · David Matlack
Quoted 4 lines · Raghavendra Rao Ananta
Not only at init, but a
vf_tokencan also be set via theVFIO_DEVICE_FEATUREioctl, by setting theVFIO_DEVICE_FEATURE_PCI_VF_TOKENflag. Add an API to utilize this functionality from the test code.Say what the commit does first. Then add context (e.g. compare/contrast to other ways of setting the VF token).
Also please add a sentence about how this will be used in a subsequent commit, since there are no callers in this commit.
Quoted 8 lines
tools/testing/selftests/vfio/lib/vfio_pci_device.c static void vfio_pci_irq_get(struct vfio_pci_device *device, u32 index, · line 174
Quoted 5 lines
tools/testing/selftests/vfio/lib/vfio_pci_device.c +5
-0+void vfio_device_set_vf_token (int fd,const char *vf_token) +{ +uuid_t token_uuid = {0 }; + +VFIO_ASSERT_NOT_NULL (vf_token,"vf_token is NULL" );nit: The help message here is not needed. It will be very obvious that
vf_tokenis NULL if this assert fires :)
I'll apply these suggestions in v2.
Thank you.
Raghavendra
PATCH 4/4 vfio: selftests: Add tests to validate SR-IOV UAPI
In reply to Raghavendra Rao AnantaAdd a test to validate the SR-IOV UAPI, including the following cases, iterating over all the IOMMU modes currently supported:
- Setting correct/incorrect/NULL tokens during device init.
- Close the PF device immediately after setting the token.
- Change/override the PF's token after device init.
--- tools/testing/selftests/vfio/Makefile | 1 + .../selftests/vfio/vfio_pci_sriov_uapi_test.c | 220 ++++++++++++++++++ 2 files changed, 221 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c
tools/testing/selftests/vfio/Makefile +1 -0
@@ -3,6 +3,7 @@ TEST_GEN_PROGS += vfio_dma_mapping_test TEST_GEN_PROGS += vfio_iommufd_setup_test TEST_GEN_PROGS += vfio_pci_device_test TEST_GEN_PROGS += vfio_pci_driver_test +TEST_GEN_PROGS += vfio_pci_sriov_uapi_test TEST_PROGS_EXTENDED := run.sh include ../lib.mk include lib/libvfio.mk
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +220 -0
220 lines not on this page · raw message
Signature · 1 line
2.51.2.997.g839fc31de9-goog
David Matlack
maintainerRe: [PATCH 4/4] vfio: selftests: Add tests to validate SR-IOV UAPI
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 18
Quoted 2 lines · Raghavendra Rao Ananta
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +2
-0+static const char *pf_dev_bdf; +static char vf_dev_bdf[16 ];
vf_dev_bdf can be part of the test fixture instead of a global variable. pf_dev_bdf should be the only global variable since we have to get it from main() into the text fixture.
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 21
Quoted 3 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +3
-0+ +struct vfio_pci_device *pf_device; +struct vfio_pci_device *vf_device;
These can be local variables in the places they are used.
Quoted 25 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +25
-0+ +static void test_vfio_pci_container_setup (struct vfio_pci_device *device, +const char *bdf, +const char *vf_token) +{ +vfio_container_open (device); +vfio_pci_group_setup (device, bdf); +vfio_container_set_iommu (device); +__vfio_container_get_device_fd (device, bdf, vf_token); +} + +static int test_vfio_pci_iommufd_setup (struct vfio_pci_device *device, +const char *bdf,const char *vf_token) +{ +vfio_pci_iommufd_cdev_open (device, bdf); +vfio_pci_iommufd_iommudev_open (device); +return __vfio_device_bind_iommufd (device->fd, device->iommufd, vf_token); +} + +static struct vfio_pci_device *test_vfio_pci_device_init (const char *bdf, +const char *iommu_mode, +const char *vf_token, +int *out_ret) +{ +struct vfio_pci_device *device;
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 56
Quoted 10 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +10
-0+ + device =calloc (1 ,sizeof (*device)); +VFIO_ASSERT_NOT_NULL (device); + + device->iommu_mode =lookup_iommu_mode (iommu_mode); + +if (iommu_mode_container_path (iommu_mode)) { +test_vfio_pci_container_setup (device, bdf, vf_token); +/* The device fd will be -1 in case of mismatched tokens */ + *out_ret = (device->fd <0 );
Maybe just return device->fd from test_vfio_pci_container_setup() so this can be:
*out_ret = test_vfio_pci_container_setup(device, bdf, vf_token);
and then you can drop the curly braces.
Quoted 25 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +25
-0+ }else { + *out_ret =test_vfio_pci_iommufd_setup (device, bdf, vf_token); + } + +return device; +} + +static void test_vfio_pci_device_cleanup (struct vfio_pci_device *device) +{ +if (device->fd >0 ) +VFIO_ASSERT_EQ (close (device->fd),0 ); + +if (device->iommufd) { +VFIO_ASSERT_EQ (close (device->iommufd),0 ); + }else { +VFIO_ASSERT_EQ (close (device->group_fd),0 ); +VFIO_ASSERT_EQ (close (device->container_fd),0 ); + } + +free (device); +} + +FIXTURE (vfio_pci_sriov_uapi_test) {}; + +FIXTURE_SETUP (vfio_pci_sriov_uapi_test)
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 91
Quoted 10 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +10
-0+{ +char vf_path[PATH_MAX ] = {0 }; +char path[PATH_MAX ] = {0 }; +unsigned int nr_vfs; +char buf[32 ] = {0 }; +int ret; +int fd; + +/* Check if SR-IOV is supported by the device */ +snprintf (path,PATH_MAX ,"%s/%s/sriov_totalvfs" ,PCI_SYSFS_PATH , pf_dev_bdf);
nit: Personally I would just hard-code the sysfs path instead of using PCI_SYSFS_PATH. I think the code is more readable and more succinct that way. And sysfs should be a stable ABI.
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 95
Quoted 4 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +4
-0+ fd =open (path,O_RDONLY ); +if (fd <0 ) { +fprintf (stderr,"SR-IOV may not be supported by the device\n" ); +exit (KSFT_SKIP );
Use SKIP() for this:
if (fd < 0)
SKIP(return, "SR-IOV is not supported by the device\n");
Ditto below.
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 106
Quoted 11 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +11
-0+ } + +ASSERT_GT (read (fd, buf,ARRAY_SIZE (buf)),0 ); +ASSERT_EQ (close (fd),0 ); + nr_vfs =strtoul (buf,NULL ,0 ); +if (nr_vfs <0 ) { +fprintf (stderr,"SR-IOV may not be supported by the device\n" ); +exit (KSFT_SKIP ); + } + +/* Setup VFs, if already not done */
Before creating VFs, should we disable auto-probing so the VFs don't get bound to some other random driver (write 0 to sriov_drivers_autoprobe)?
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 111
Quoted 5 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +5
-0+snprintf (path,PATH_MAX ,"%s/%s/sriov_numvfs" ,PCI_SYSFS_PATH , pf_dev_bdf); +ASSERT_GT (fd =open (path,O_RDWR ),0 ); +ASSERT_GT (read (fd, buf,ARRAY_SIZE (buf)),0 ); + nr_vfs =strtoul (buf,NULL ,0 ); +if (nr_vfs ==0 )
If VFs are already enabled, shouldn't the test fail or skip?
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 120
Quoted 9 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +9
-0+ASSERT_EQ (write (fd,"1" ,1 ),1 ); +ASSERT_EQ (close (fd),0 ); + +/* Get the BDF of the first VF */ +snprintf (path,PATH_MAX ,"%s/%s/virtfn0" ,PCI_SYSFS_PATH , pf_dev_bdf); + ret =readlink (path, vf_path,PATH_MAX ); +ASSERT_NE (ret,-1 ); + ret =sscanf (basename (vf_path),"%s" , vf_dev_bdf); +ASSERT_EQ (ret,1 );
What ensures the created VF is bound to vfio-pci?
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 125
Quoted 5 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +5
-0+} + +FIXTURE_TEARDOWN (vfio_pci_sriov_uapi_test) +{ +}
FIXTURE_TEARDOWN() should undo what FIXTURE_SETUP() did, i.e. write 0 to sriov_numvfs. Otherwise running this test will leave behind SR-IOV enabled on the PF.
You could also make this the users problem (the user has to provide a PF with 1 VF where both PF and VF are bound to vfio-pci). But I think it would be nice to make the test work automatically given a PF if we can.
Re: [PATCH 4/4] vfio: selftests: Add tests to validate SR-IOV UAPI
Quoted 10 lines · David Matlack
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 18
Quoted 2 lines · Raghavendra Rao Ananta
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +2
-0+static const char *pf_dev_bdf; +static char vf_dev_bdf[16 ];
vf_dev_bdfcan be part of the test fixture instead of a global variable.pf_dev_bdfshould be the only global variable since we have to get it frommain()into the text fixture.
My understading is placing vars in FIXTURE() is helpful to get an access across various other FIXTURE_*() and TEST*() functions. Out of curiosity, is there an advantage here vs having them global?
Quoted 6 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 21
Quoted 3 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +3
-0+ +struct vfio_pci_device *pf_device; +struct vfio_pci_device *vf_device;These can be local variables in the places they are used.
I was a bit greedy to save a few lines, as they are reassigned in every TEST_F() anyway. Is there any advantage by making them local?
Quoted 39 lines
Quoted 25 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +25
-0+ +static void test_vfio_pci_container_setup (struct vfio_pci_device *device, +const char *bdf, +const char *vf_token) +{ +vfio_container_open (device); +vfio_pci_group_setup (device, bdf); +vfio_container_set_iommu (device); +__vfio_container_get_device_fd (device, bdf, vf_token); +} + +static int test_vfio_pci_iommufd_setup (struct vfio_pci_device *device, +const char *bdf,const char *vf_token) +{ +vfio_pci_iommufd_cdev_open (device, bdf); +vfio_pci_iommufd_iommudev_open (device); +return __vfio_device_bind_iommufd (device->fd, device->iommufd, vf_token); +} + +static struct vfio_pci_device *test_vfio_pci_device_init (const char *bdf, +const char *iommu_mode, +const char *vf_token, +int *out_ret) +{ +struct vfio_pci_device *device;tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 56
Quoted 10 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +10
-0+ + device =calloc (1 ,sizeof (*device)); +VFIO_ASSERT_NOT_NULL (device); + + device->iommu_mode =lookup_iommu_mode (iommu_mode); + +if (iommu_mode_container_path (iommu_mode)) { +test_vfio_pci_container_setup (device, bdf, vf_token); +/* The device fd will be -1 in case of mismatched tokens */ + *out_ret = (device->fd <0 );Maybe just return
device->fdfromtest_vfio_pci_container_setup()so this can be:
Quoted 4 lines
*out_ret = test_vfio_pci_container_setup(device, bdf, vf_token);and then you can drop the curly braces.
Makes sense. I'll do it in v2.
Quoted 31 lines
Quoted 31 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +31
-0+ }else { + *out_ret =test_vfio_pci_iommufd_setup (device, bdf, vf_token); + } + +return device; +} + +static void test_vfio_pci_device_cleanup (struct vfio_pci_device *device) +{ +if (device->fd >0 ) +VFIO_ASSERT_EQ (close (device->fd),0 ); + +if (device->iommufd) { +VFIO_ASSERT_EQ (close (device->iommufd),0 ); + }else { +VFIO_ASSERT_EQ (close (device->group_fd),0 ); +VFIO_ASSERT_EQ (close (device->container_fd),0 ); + } + +free (device); +} + +FIXTURE (vfio_pci_sriov_uapi_test) {}; + +FIXTURE_SETUP (vfio_pci_sriov_uapi_test) +{ +char vf_path[PATH_MAX ] = {0 }; +char path[PATH_MAX ] = {0 }; +unsigned int nr_vfs; +char buf[32 ] = {0 }; +int ret;
Quoted 9 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 91
Quoted 4 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +4
-0+int fd; + +/* Check if SR-IOV is supported by the device */ +snprintf (path,PATH_MAX ,"%s/%s/sriov_totalvfs" ,PCI_SYSFS_PATH , pf_dev_bdf);nit: Personally I would just hard-code the sysfs path instead of using
PCI_SYSFS_PATH. I think the code is more readable and more succinct that way. And sysfs should be a stable ABI.
Sure.
Quoted 7 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 95
Quoted 4 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +4
-0+ fd =open (path,O_RDONLY ); +if (fd <0 ) { +fprintf (stderr,"SR-IOV may not be supported by the device\n" ); +exit (KSFT_SKIP );Use
SKIP()for this:
Sure.
Quoted 5 lines
if (fd < 0)
SKIP(return, "SR-IOV is not supported by the device\n");Ditto below.
Quoted 15 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 106
Quoted 11 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +11
-0+ } + +ASSERT_GT (read (fd, buf,ARRAY_SIZE (buf)),0 ); +ASSERT_EQ (close (fd),0 ); + nr_vfs =strtoul (buf,NULL ,0 ); +if (nr_vfs <0 ) { +fprintf (stderr,"SR-IOV may not be supported by the device\n" ); +exit (KSFT_SKIP ); + } + +/* Setup VFs, if already not done */ Before creating VFs, should we disable auto-probing so the VFs don't get bound to some other random driver (write 0 to
sriov_drivers_autoprobe)?
Good idea. I'll make this change in v2.
Quoted 8 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 111
Quoted 5 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +5
-0+snprintf (path,PATH_MAX ,"%s/%s/sriov_numvfs" ,PCI_SYSFS_PATH , pf_dev_bdf); +ASSERT_GT (fd =open (path,O_RDWR ),0 ); +ASSERT_GT (read (fd, buf,ARRAY_SIZE (buf)),0 ); + nr_vfs =strtoul (buf,NULL ,0 ); +if (nr_vfs ==0 )If VFs are already enabled, shouldn't the test fail or skip?
My idea was to simply "steal" the device that was already created and use it. Do we want to skip it, as you suggested?
Quoted 10 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 120
Quoted 9 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +9
-0+ASSERT_EQ (write (fd,"1" ,1 ),1 ); +ASSERT_EQ (close (fd),0 ); + +/* Get the BDF of the first VF */ +snprintf (path,PATH_MAX ,"%s/%s/virtfn0" ,PCI_SYSFS_PATH , pf_dev_bdf); + ret =readlink (path, vf_path,PATH_MAX ); +ASSERT_NE (ret,-1 ); + ret =sscanf (basename (vf_path),"%s" , vf_dev_bdf); +ASSERT_EQ (ret,1 );
Quoted 2 lines
What ensures the created VF is bound to vfio-pci?
Good point. I'll explicitly bind it to vfio-pci, if it isn't already.
Quoted 10 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 125
Quoted 5 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +5
-0+} + +FIXTURE_TEARDOWN (vfio_pci_sriov_uapi_test) +{ +}
FIXTURE_TEARDOWN()should undo whatFIXTURE_SETUP()did, i.e. write 0 tosriov_numvfs. Otherwise running this test will leave behind SR-IOV enabled on the PF.
I had this originally, but then realized that run.sh aready resets the sriov_numvfs to its original value. We can do it here too, if you'd like to keep the symmetry and make the test self-sufficient. With some of your other suggestions, I may have to do some more cleanup here now.
Quoted 3 lines
You could also make this the users problem (the user has to provide a PF with 1 VF where both PF and VF are bound to vfio-pci). But I think it would be nice to make the test work automatically given a PF if we can.
Let's go with the latter, assuming it doesn't get too complicated (currently, the setup part seems bigger than the actual test :) )
David Matlack
maintainerRe: [PATCH 4/4] vfio: selftests: Add tests to validate SR-IOV UAPI
Quoted 11 lines · Raghavendra Rao Ananta
Quoted 10 lines · David Matlack
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 18
Quoted 2 lines · Raghavendra Rao Ananta
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +2
-0+static const char *pf_dev_bdf; +static char vf_dev_bdf[16 ];
vf_dev_bdfcan be part of the test fixture instead of a global variable.pf_dev_bdfshould be the only global variable since we have to get it frommain()into the text fixture.
Quoted 3 lines
My understading is placing vars in
FIXTURE()is helpful to get an access across various other FIXTURE_*() and TEST*() functions. Out of curiosity, is there an advantage here vs having them global?
Global variables are just generally a bad design pattern. IMO, only variables that truly need to be global should be global.
The only variable that needs to be global is pf_dev_bdf.
Since vf_dev_bdf needs to be accessed within FIXTURE_SETUP(), FIXTURE_TEARDOWN(), and TEST_F(), then FIXTURE() is the right home for it. The whole point of FIXTURE() is to hold state for each TEST_F().
Quoted 9 lines
Quoted 7 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 21
Quoted 4 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +3
-0+ +struct vfio_pci_device *pf_device; +struct vfio_pci_device *vf_device;These can be local variables in the places they are used.
I was a bit greedy to save a few lines, as they are reassigned in every
TEST_F()anyway. Is there any advantage by making them local?
It's easy to mess up global variables. And also when reading the code it is confusing to see a global variable that does not need to be global. It makes me think I must be missing something.
As a general practice I think it's good to limit the scope of variables to the minimum scope they are needed.
Quoted 10 lines
Quoted 8 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 111
Quoted 5 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +5
-0+snprintf (path,PATH_MAX ,"%s/%s/sriov_numvfs" ,PCI_SYSFS_PATH , pf_dev_bdf); +ASSERT_GT (fd =open (path,O_RDWR ),0 ); +ASSERT_GT (read (fd, buf,ARRAY_SIZE (buf)),0 ); + nr_vfs =strtoul (buf,NULL ,0 ); +if (nr_vfs ==0 )If VFs are already enabled, shouldn't the test fail or skip?
My idea was to simply "steal" the device that was already created and use it. Do we want to skip it, as you suggested?
If a VF already exists it might be bound to a different driver, and may be in use by something else. I think the only safe thing to do is to bail if a VF already exists. If the test creates the VF, then it knows that it owns it.
Quoted 8 lines
Quoted 8 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 125
Quoted 3 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +3
-0+FIXTURE_TEARDOWN (vfio_pci_sriov_uapi_test) +{ +}
FIXTURE_TEARDOWN()should undo whatFIXTURE_SETUP()did, i.e. write 0 tosriov_numvfs. Otherwise running this test will leave behind SR-IOV enabled on the PF.
Quoted 5 lines
I had this originally, but then realized that
run.shaready resets thesriov_numvfsto its original value. We can do it here too, if you'd like to keep the symmetry and make the test self-sufficient. With some of your other suggestions, I may have to do some more cleanup here now.
I think the test should return the PF back to the state it was in at the start of the test. That way the test doesn't "leak" changes it made. The best way to do that is to clean up in FIXTURE_TEARDOWN(). There might be some other test that wants to run using the PF before run.sh does its cleanup work.
Quoted 5 lines
Quoted 3 lines
You could also make this the users problem (the user has to provide a PF with 1 VF where both PF and VF are bound to vfio-pci). But I think it would be nice to make the test work automatically given a PF if we can.
Let's go with the latter, assuming it doesn't get too complicated (currently, the setup part seems bigger than the actual test :) )
Let's create helpers for all the sysfs operations under lib.
e.g. tools/testing/selftests/vfio/lib/sysfs.c:
int sysfs_get_sriov_totalvfs(const char *bdf); void sysfs_set_sriov_numvfs(const char *bdfs, int numvfs); ...
That will greatly simplify the amount of code in this test, and I think it's highly likely we re-use those functions in other tests. And even if we don't, it's nice to encapsulate all the sysfs code in one place for readability and maintainability.
If you do this I think there's also some sysfs stuff in vfio_pci_device.c that you can also pull out into this helper file.
Re: [PATCH 4/4] vfio: selftests: Add tests to validate SR-IOV UAPI
Quoted 17 lines · David Matlack
Quoted 11 lines · Raghavendra Rao Ananta
Quoted 10 lines · David Matlack
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 18
Quoted 2 lines · Raghavendra Rao Ananta
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +2
-0+static const char *pf_dev_bdf; +static char vf_dev_bdf[16 ];
vf_dev_bdfcan be part of the test fixture instead of a global variable.pf_dev_bdfshould be the only global variable since we have to get it frommain()into the text fixture.Quoted 3 lines
My understading is placing vars in
FIXTURE()is helpful to get an access across various other FIXTURE_*() and TEST*() functions. Out of curiosity, is there an advantage here vs having them global?
Quoted 9 lines
Global variables are just generally a bad design pattern. IMO, only variables that truly need to be global should be global.
The only variable that needs to be global is
pf_dev_bdf.Since
vf_dev_bdfneeds to be accessed withinFIXTURE_SETUP(),FIXTURE_TEARDOWN(), andTEST_F(), thenFIXTURE()is the right home for it. The whole point ofFIXTURE()is to hold state for eachTEST_F().
Sounds good. I'll move them into FIXTURE().
Quoted 10 lines
Quoted 9 lines
Quoted 7 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 21
Quoted 4 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +3
-0+ +struct vfio_pci_device *pf_device; +struct vfio_pci_device *vf_device;These can be local variables in the places they are used.
I was a bit greedy to save a few lines, as they are reassigned in every
TEST_F()anyway. Is there any advantage by making them local?
Quoted 7 lines
It's easy to mess up global variables. And also when reading the code it is confusing to see a global variable that does not need to be global. It makes me think I must be missing something.
As a general practice I think it's good to limit the scope of variables to the minimum scope they are needed.
Agreed. I prefer min scope too, but I guess my habit of using global variables in other tests and avoiding passing pointers around led me to use it here. I'll move it to a local scope.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 111
Quoted 5 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +5
-0+snprintf (path,PATH_MAX ,"%s/%s/sriov_numvfs" ,PCI_SYSFS_PATH , pf_dev_bdf); +ASSERT_GT (fd =open (path,O_RDWR ),0 ); +ASSERT_GT (read (fd, buf,ARRAY_SIZE (buf)),0 ); + nr_vfs =strtoul (buf,NULL ,0 ); +if (nr_vfs ==0 )If VFs are already enabled, shouldn't the test fail or skip?
Quoted 8 lines
Quoted 2 lines
My idea was to simply "steal" the device that was already created and use it. Do we want to skip it, as you suggested?
If a VF already exists it might be bound to a different driver, and may be in use by something else. I think the only safe thing to do is to bail if a VF already exists. If the test creates the VF, then it knows that it owns it.
Makes sense. Let's skip in that case.
Quoted 14 lines
Quoted 8 lines
Quoted 8 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c line 125
Quoted 3 lines
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c new file +3
-0+FIXTURE_TEARDOWN (vfio_pci_sriov_uapi_test) +{ +}
FIXTURE_TEARDOWN()should undo whatFIXTURE_SETUP()did, i.e. write 0 tosriov_numvfs. Otherwise running this test will leave behind SR-IOV enabled on the PF.Quoted 5 lines
I had this originally, but then realized that
run.shaready resets thesriov_numvfsto its original value. We can do it here too, if you'd like to keep the symmetry and make the test self-sufficient. With some of your other suggestions, I may have to do some more cleanup here now.
Quoted 6 lines
I think the test should return the PF back to the state it was in at the start of the test. That way the test doesn't "leak" changes it made. The best way to do that is to clean up in
FIXTURE_TEARDOWN(). There might be some other test that wants to run using the PF beforerun.shdoes its cleanup work.
Sure, I'll clean up everything that the test does in FIXTURE_SETUP().
Quoted 14 lines
Quoted 5 lines
Quoted 3 lines
You could also make this the users problem (the user has to provide a PF with 1 VF where both PF and VF are bound to vfio-pci). But I think it would be nice to make the test work automatically given a PF if we can.
Let's go with the latter, assuming it doesn't get too complicated (currently, the setup part seems bigger than the actual test :) )
Let's create helpers for all the sysfs operations under lib.
e.g.
tools/testing/selftests/vfio/lib/sysfs.c:int sysfs_get_sriov_totalvfs(const char *bdf); void sysfs_set_sriov_numvfs(const char *bdfs, int numvfs); ...
Quoted 7 lines
That will greatly simplify the amount of code in this test, and I think it's highly likely we re-use those functions in other tests. And even if we don't, it's nice to encapsulate all the sysfs code in one place for readability and maintainability.
If you do this I think there's also some sysfs stuff in
vfio_pci_device.cthat you can also pull out into this helper file.
Good idea. I'll create this lib.
Thank you.
Raghavendra