vfio: selftests: Add peer-to-peer DMA selftest
Subsystems (6)
David Matlack
maintainercover vfio: selftests: Add peer-to-peer DMA selftest
This series can be found on GitHub:
https://github.com/dmatlack/linux/tree/vfio/selftests/p2pdma/rfc/v1
This series adds a peer-to-peer (P2P) DMA selftest to the VFIO selftests. The test pairs an initiator device supported by the VFIO selftests driver framework (dsa, igb, ioat, mlx5, or nv_falcon) with a target device that exposes a memory BAR, maps the target's BAR into the IOMMU, and uses the initiator to perform DMA reads and writes targeting the BAR.
No changes to struct vfio_pci_driver_ops are needed. Because vfio_pci_driver_memcpy() already takes source and destination IOVAs, any existing selftests driver can drive P2P DMA once a peer's BAR is mapped into its IOMMU address space.
There are two motivations for adding P2P DMA support to VFIO selftests:
1. Exercise the P2P DMA paths in VFIO and iommufd end-to-end, including mapping BAR mmaps via VFIO_IOMMU_MAP_DMA in legacy type1/type1v2 modes and exporting BARs via VFIO_DEVICE_FEATURE_DMA_BUF + IOMMU_IOAS_MAP_FILE (and dma-buf revocation) in native iommufd.
2. Lay the groundwork for testing uninterrupted P2P DMA across a Live Update in the future, building on VFIO PCI Live Update support [1] and the PCI core changes to keep preserved devices accessible across kexec [2].
Patch 1 comes from v3 of Mina's huge_fault series [3] and is included here for convenience. It adds vfio_pci_find_mem_bar() to locate a BAR range that behaves like ordinary memory, starting with QEMU's pci-testdev.
This series applies on top of vfio/next (vfio-v7.4-rc1).
[1] https://lore.kernel.org/kvm/[email protected]/
[2] https://lore.kernel.org/linux-pci/[email protected]/
[3] https://lore.kernel.org/kvm/[email protected]/
Series Overview
---------------
This series implements the following:
1. Add vfio_pci_find_mem_bar() to locate a BAR range that can be safely read and written like memory, starting with QEMU's pci-testdev, and verify that writes stick before returning it so that unbacked pci-testdev BARs are skipped (patches 1-2).
2. Allow VFIO_SELFTESTS_BDF to contain multiple whitespace-separated BDFs so multi-device tests can run via run_kselftest.sh while single-device tests continue to use the first BDF (patch 3).
3. Add vfio_pci_dmabuf_create() and iommu_map_file() helpers to export a VFIO PCI region range via VFIO_DEVICE_FEATURE_DMA_BUF and map it into an iommufd IOAS via IOMMU_IOAS_MAP_FILE (patches 4-5).
4. Add vfio_pci_find_p2p_pair() to find a compatible (initiator, target) pair from an array of VFIO PCI devices, providing a central place to check P2P topology reachability in the future (patch 6).
5. Add vfio_pci_p2pdma_test, covering RAM-to-BAR, BAR-to-RAM, BAR-to-BAR, and RAM-BAR-RAM roundtrip copies across all supported IOMMU modes, plus iommufd dma-buf revocation when PCI_COMMAND_MEMORY is cleared on the target (patch 7).
Open Questions
--------------
- Re-mapping after dma-buf revocation: Clearing PCI_COMMAND_MEMORY on the target device calls vfio_pci_dma_buf_move(vdev, true), which notifies iommufd via iopt_revoke_notify() and unmaps the BAR from the IOMMU domain. Re-enabling PCI_COMMAND_MEMORY calls vfio_pci_dma_buf_move(vdev, false), allowing the dma-buf to be mapped again, but dma-buf has no callback to re-populate iommufd, so the existing IOAS mapping stays unmapped until userspace calls IOMMU_IOAS_UNMAP and IOMMU_IOAS_MAP_FILE again.
Is requiring userspace to re-issue IOMMU_IOAS_MAP_FILE the expected uAPI behavior? VFIO_DEVICE_RESET (and runtime PM transitions) on the target device also bracket the reset with vfio_pci_dma_buf_move(vdev, true) and vfio_pci_dma_buf_move(vdev, false), tearing down any active IOMMU_IOAS_MAP_FILE mapping of that device's BAR even though PCI_COMMAND_MEMORY remains enabled when VFIO_DEVICE_RESET returns.
- Physical target devices and P2P topology checks: mem_bar_descs[] currently only matches QEMU's pci-testdev, and vfio_pci_has_p2p_path() does not yet check PCI topology or ACS to verify that P2P traffic can flow between two physical endpoints.
Suggestions on physical target BARs to add to mem_bar_descs[] and on checking P2P reachability from userspace are welcome.
Testing
-------
All patches were build-tested individually. The series was tested on x86_64 in QEMU 11.1 (membar-backed=on requires QEMU 10.1+) using an emulated Intel 82576 NIC (igb) and a pci-testdev with a 1G RAM-backed BAR2 behind an emulated Intel IOMMU:
qemu-system-x86_64 \
-M q35,accel=kvm,kernel-irqchip=split \
-device intel-iommu,intremap=on,caching-mode=on,device-iotlb=on \
-netdev user,id=net0 \
-device pcie-root-port,id=rp0,slot=2 \
-device igb,bus=rp0,addr=0.0,netdev=net0 \
-device pci-testdev,membar=1G,membar-backed=on \
...Inside the guest, bind both devices to vfio-pci and run the test (BDF order does not matter):
# tools/testing/selftests/vfio/scripts/setup.sh 0000:01:00.0 0000:00:0d.0
# tools/testing/selftests/vfio/scripts/run.sh \
tools/testing/selftests/vfio/vfio_pci_p2pdma_testOr via run_kselftest.sh:
# VFIO_SELFTESTS_BDF="0000:01:00.0 0000:00:0d.0" \
./run_kselftest.sh -t vfio:vfio_pci_p2pdma_test
David Matlack (6):
vfio: selftests: Verify memory behavior in vfio_pci_find_mem_bar()
vfio: selftests: Allow multiple BDFs in VFIO_SELFTESTS_BDF
vfio: selftests: Add vfio_pci_dmabuf_create() helper
vfio: selftests: Add iommu_map_file() helper
vfio: selftests: Add vfio_pci_find_p2p_pair() helper
vfio: selftests: Add P2P DMA selftest
Mina Almasry (1):
vfio: selftests: Add vfio_pci_find_mem_bar() helper
tools/testing/selftests/vfio/Makefile | 1 + .../vfio/lib/include/libvfio/iommu.h | 9 + .../lib/include/libvfio/vfio_pci_device.h | 23 + tools/testing/selftests/vfio/lib/iommu.c | 29 ++ tools/testing/selftests/vfio/lib/libvfio.c | 53 ++- .../selftests/vfio/lib/vfio_pci_device.c | 181 ++++++++ .../selftests/vfio/vfio_pci_p2pdma_test.c | 435 ++++++++++++++++++ 7 files changed, 717 insertions(+), 14 deletions(-) create mode 100644 tools/testing/selftests/vfio/vfio_pci_p2pdma_test.c
base-commit: 076d7e989cb69f6ded7d60f2a080eb909930061b
Signature · 1 line
2.56.0.567.gdb80ae8cd5-goog
David Matlack
maintainerRFC PATCH 1/7 vfio: selftests: Add vfio_pci_find_mem_bar() helper
From: Mina Almasry <[email protected]>
Add vfio_pci_find_mem_bar() to libvfio to locate a BAR (or range within a BAR) of at least min_size bytes that behaves like ordinary memory and can be safely read and written without device-specific side effects.
Detection is backed by a table of known devices, starting with QEMU's pci-testdev (1b36:0005) BAR 2 when configured with membar=<size> and membar-backed=on.
--- .../lib/include/libvfio/vfio_pci_device.h | 10 +++ .../selftests/vfio/lib/vfio_pci_device.c | 67 +++++++++++++++++++ 2 files changed, 77 insertions(+)
tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +10 -0
@@ -15,6 +15,13 @@ struct vfio_pci_bar {void *vaddr; }; +struct vfio_pci_mem_bar { +struct vfio_pci_bar *bar; +void *vaddr; +u64 offset; +u64 size; +}; +struct vfio_pci_device {const char *bdf;int fd;
@@ -162,6 +169,9 @@ static inline bool vfio_pci_device_match(struct vfio_pci_device *device, (device_id ==vfio_pci_config_readw (device,PCI_DEVICE_ID )); } +bool vfio_pci_find_mem_bar (struct vfio_pci_device *device,u64 min_size, +struct vfio_pci_mem_bar *mem_bar); +const char *vfio_pci_get_cdev_path (const char *bdf);void vfio_pci_group_setup (struct vfio_pci_device *device,const char *bdf);
tools/testing/selftests/vfio/lib/vfio_pci_device.c +67 -0
79 lines not on this page · raw message
Signature · 1 line
2.56.0.567.gdb80ae8cd5-goog
[email protected]
AI reviewRe: [RFC PATCH 1/7] vfio: selftests: Add vfio_pci_find_mem_bar() helper
Sashiko has reviewed this patch and found no issues.
Quoted 10 lines
From: Mina Almasry <[email protected]>
Add
vfio_pci_find_mem_bar()to libvfio to locate a BAR (or range within a BAR) of at leastmin_sizebytes that behaves like ordinary memory and can be safely read and written without device-specific side effects.Detection is backed by a table of known devices, starting with QEMU's pci-testdev (1b36:0005) BAR 2 when configured with membar=<size> and membar-backed=on.
Quoted 7 lines
Suggested-by David Matlack <[email protected]>Cc David Matlack <[email protected]>Cc Matt Evans <[email protected]>Signed-off-by Mina Almasry <[email protected]>Reviewed-by David Matlack <[email protected]>Signed-off-by David Matlack <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
David Matlack
maintainerRFC PATCH 2/7 vfio: selftests: Verify memory behavior in vfio_pci_find_mem_bar()
In reply to David MatlackWrite and read back test patterns before returning a BAR from vfio_pci_find_mem_bar(), restoring the original contents and PCI_COMMAND_MEMORY state afterward.
QEMU's pci-testdev exposes a readable, writable, mmap-capable BAR2 whenever membar=<size> is passed, even if membar-backed=on was omitted and accesses are dropped as unimplemented MMIO. Probing the BAR upfront lets callers skip cleanly instead of failing later DMA or fault checks.
--- .../selftests/vfio/lib/vfio_pci_device.c | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+)
tools/testing/selftests/vfio/lib/vfio_pci_device.c +33 -0
@@ -13,6 +13,7 @@#include <sys/mman.h> #include <linux/align.h> +#include <linux/io.h> #include <linux/iommufd.h> #include <linux/kernel.h> #include <linux/limits.h>
@@ -265,6 +266,33 @@ static const struct vfio_pci_mem_bar_desc mem_bar_descs[] = { }, }; +static bool vfio_pci_mem_bar_verify (struct vfio_pci_device *device,void *vaddr) +{ +const u64 pattern =0x55aa55aa55aa55aaULL ; +u16 orig_cmd; +u64 orig; +bool ok; + + orig_cmd =vfio_pci_config_readw (device,PCI_COMMAND ); +if (!(orig_cmd &PCI_COMMAND_MEMORY )) +vfio_pci_config_writew (device,PCI_COMMAND , + orig_cmd |PCI_COMMAND_MEMORY ); + + orig =readq (vaddr); +writeq (pattern, vaddr); + ok = (readq (vaddr) == pattern); +if (ok) { +writeq (~pattern, vaddr); + ok = (readq (vaddr) == ~pattern); + } +writeq (orig, vaddr); + +if (!(orig_cmd &PCI_COMMAND_MEMORY )) +vfio_pci_config_writew (device,PCI_COMMAND , orig_cmd); + +return ok; +} +bool vfio_pci_find_mem_bar (struct vfio_pci_device *device,u64 min_size,struct vfio_pci_mem_bar *mem_bar) {
@@ -273,6 +301,8 @@ bool vfio_pci_find_mem_bar(struct vfio_pci_device *device, u64 min_size,VFIO_REGION_INFO_FLAG_WRITE ;size_t i; + min_size =max_t (u64, min_size,sizeof (u64)); +for (i =0 ; i <ARRAY_SIZE (mem_bar_descs); i++) {const struct vfio_pci_mem_bar_desc *desc = &mem_bar_descs[i];struct vfio_pci_bar *bar;
@@ -300,6 +330,9 @@ bool vfio_pci_find_mem_bar(struct vfio_pci_device *device, u64 min_size,if (size < min_size)continue ; +if (!vfio_pci_mem_bar_verify (device, bar->vaddr + desc->offset)) +continue ; + *mem_bar = (struct vfio_pci_mem_bar ){ .bar = bar, .vaddr = bar->vaddr + desc->offset,
Signature · 1 line
2.56.0.567.gdb80ae8cd5-goog
[email protected]
AI reviewRe: [RFC PATCH 2/7] vfio: selftests: Verify memory behavior in vfio_pci_find_mem_bar()
Sashiko has reviewed this patch and found no issues.
Quoted 11 lines
Write and read back test patterns before returning a BAR from
vfio_pci_find_mem_bar(), restoring the original contents andPCI_COMMAND_MEMORYstate afterward.QEMU's pci-testdev exposes a readable, writable, mmap-capable BAR2 whenever membar=<size> is passed, even if membar-backed=on was omitted and accesses are dropped as unimplemented MMIO. Probing the BAR upfront lets callers skip cleanly instead of failing later DMA or fault checks.
Suggested-by Alex Williamson <[email protected]>Signed-off-by David Matlack <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
David Matlack
maintainerRFC PATCH 3/7 vfio: selftests: Allow multiple BDFs in VFIO_SELFTESTS_BDF
In reply to David MatlackParse whitespace-separated PCI BDFs from VFIO_SELFTESTS_BDF in vfio_selftests_get_bdfs() instead of requiring a single BDF.
Multi-device tests such as P2P DMA tests need two or more devices even when invoked through run_kselftest.sh without command-line arguments. Single-device callers using vfio_selftests_get_bdf() continue to receive the first BDF.
--- tools/testing/selftests/vfio/lib/libvfio.c | 53 ++++++++++++++++------ 1 file changed, 39 insertions(+), 14 deletions(-)
tools/testing/selftests/vfio/lib/libvfio.c +39 -14
@@ -2,6 +2,7 @@#include <stdio.h> #include <stdlib.h> +#include <string.h> #include <sys/mman.h> #include <linux/align.h>
@@ -32,38 +33,62 @@ static char **get_bdfs_cmdline(int *argc, char *argv[], int *nr_bdfs)return *nr_bdfs ? &argv[i] :NULL ; } -static char *get_bdf_env (void ) +static char **get_bdfs_env (int *nr_bdfs) { -char *bdf; +static char **bdfs; +static int count; +char *env, *tok, *saveptr; +int cap =0 ; - bdf =getenv ("VFIO_SELFTESTS_BDF" ); -if (!bdf) +if (bdfs) { + *nr_bdfs = count; +return bdfs; + } + + env =getenv ("VFIO_SELFTESTS_BDF" ); +if (!env)return NULL ; -VFIO_ASSERT_TRUE (is_bdf (bdf),"Invalid BDF: %s\n" , bdf); -return bdf; + env =strdup (env); +VFIO_ASSERT_NOT_NULL (env); + +for (tok =strtok_r (env," \t\n" , &saveptr); tok; + tok =strtok_r (NULL ," \t\n" , &saveptr)) { +VFIO_ASSERT_TRUE (is_bdf (tok),"Invalid BDF: %s\n" , tok); +if (count == cap) { + cap = cap ? cap *2 :4 ; + bdfs =realloc (bdfs, cap *sizeof (*bdfs)); +VFIO_ASSERT_NOT_NULL (bdfs); + } + bdfs[count++] = tok; + } + +if (!count) { +free (env); +return NULL ; + } + + *nr_bdfs = count; +return bdfs; }char **vfio_selftests_get_bdfs (int *argc,char *argv[],int *nr_bdfs) { -static char *env_bdf;char **bdfs; bdfs =get_bdfs_cmdline (argc, argv, nr_bdfs);if (bdfs)return bdfs; - env_bdf =get_bdf_env (); -if (env_bdf) { - *nr_bdfs =1 ; -return &env_bdf; - } + bdfs =get_bdfs_env (nr_bdfs); +if (bdfs) +return bdfs;fprintf (stderr,"Unable to determine which device(s) to use, skipping test.\n" );fprintf (stderr,"\n" ); -fprintf (stderr,"To pass the device address via environment variable:\n" ); +fprintf (stderr,"To pass the device address(es) via environment variable:\n" );fprintf (stderr,"\n" ); -fprintf (stderr," export VFIO_SELFTESTS_BDF=\"segment:bus:device.function\"\n" ); +fprintf (stderr," export VFIO_SELFTESTS_BDF=\"segment:bus:device.function ...\"\n" );fprintf (stderr," %s [options]\n" , argv[0 ]);fprintf (stderr,"\n" );fprintf (stderr,"To pass the device address(es) via argv:\n" );
Signature · 1 line
2.56.0.567.gdb80ae8cd5-goog
[email protected]
AI reviewRe: [RFC PATCH 3/7] vfio: selftests: Allow multiple BDFs in VFIO_SELFTESTS_BDF
Sashiko has reviewed this patch and found no issues.
Quoted 9 lines
Parse whitespace-separated PCI BDFs from
VFIO_SELFTESTS_BDFinvfio_selftests_get_bdfs()instead of requiring a single BDF.Multi-device tests such as P2P DMA tests need two or more devices even when invoked through
run_kselftest.shwithout command-line arguments. Single-device callers usingvfio_selftests_get_bdf()continue to receive the first BDF.Signed-off-by David Matlack <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
David Matlack
maintainerRFC PATCH 4/7 vfio: selftests: Add vfio_pci_dmabuf_create() helper
In reply to David MatlackAdd vfio_pci_dmabuf_create() and __vfio_pci_dmabuf_create() to export a VFIO PCI region range as a dma-buf fd via VFIO_DEVICE_FEATURE_DMA_BUF, and preserve -errno on failure in vfio_device_feature_ioctl().
Native iommufd maps device MMIO BARs for peer-to-peer DMA via dma-buf file descriptors rather than VM_PFNMAP virtual addresses.
--- .../lib/include/libvfio/vfio_pci_device.h | 5 +++ .../selftests/vfio/lib/vfio_pci_device.c | 33 +++++++++++++++++++ 2 files changed, 38 insertions(+)
tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +5 -0
@@ -172,6 +172,11 @@ static inline bool vfio_pci_device_match(struct vfio_pci_device *device,bool vfio_pci_find_mem_bar (struct vfio_pci_device *device,u64 min_size,struct vfio_pci_mem_bar *mem_bar); +int __vfio_pci_dmabuf_create (struct vfio_pci_device *device,u32 region_index, +u64 offset,u64 length,u32 open_flags); +int vfio_pci_dmabuf_create (struct vfio_pci_device *device,u32 region_index, +u64 offset,u64 length,u32 open_flags); +const char *vfio_pci_get_cdev_path (const char *bdf);void vfio_pci_group_setup (struct vfio_pci_device *device,const char *bdf);
tools/testing/selftests/vfio/lib/vfio_pci_device.c +33 -0
@@ -153,11 +153,44 @@ static int vfio_device_feature_ioctl(int fd, u32 flags, void *data, feature->flags = flags; ret =ioctl (fd,VFIO_DEVICE_FEATURE , feature); +if (ret <0 ) + ret = -errno;free (feature);return ret; } +int __vfio_pci_dmabuf_create (struct vfio_pci_device *device,u32 region_index, +u64 offset,u64 length,u32 open_flags) +{ +u8 buf[sizeof (struct vfio_device_feature_dma_buf ) + +sizeof (struct vfio_region_dma_range )] = {}; +struct vfio_device_feature_dma_buf *dma_buf = (void *)buf; + + dma_buf->region_index = region_index; + dma_buf->open_flags = open_flags; + dma_buf->nr_ranges =1 ; + dma_buf->dma_ranges[0 ].offset = offset; + dma_buf->dma_ranges[0 ].length = length; + +return vfio_device_feature_ioctl (device->fd, +VFIO_DEVICE_FEATURE_GET | +VFIO_DEVICE_FEATURE_DMA_BUF , + buf,sizeof (buf)); +} + +int vfio_pci_dmabuf_create (struct vfio_pci_device *device,u32 region_index, +u64 offset,u64 length,u32 open_flags) +{ +int fd; + + fd =__vfio_pci_dmabuf_create (device, region_index, offset, length, + open_flags); +VFIO_ASSERT_GE (fd,0 ,"VFIO_DEVICE_FEATURE_DMA_BUF failed: %d\n" , fd); + +return fd; +} +static void vfio_device_feature_set (int fd,u16 feature,void *data,size_t data_size) {u32 flags =VFIO_DEVICE_FEATURE_SET | feature;
Signature · 1 line
2.56.0.567.gdb80ae8cd5-goog
[email protected]
AI reviewRe: [RFC PATCH 4/7] vfio: selftests: Add vfio_pci_dmabuf_create() helper
Sashiko has reviewed this patch and found no issues.
Quoted 8 lines
Add
vfio_pci_dmabuf_create()and__vfio_pci_dmabuf_create()to export a VFIO PCI region range as a dma-buf fd viaVFIO_DEVICE_FEATURE_DMA_BUF, and preserve -errno on failure invfio_device_feature_ioctl().Native iommufd maps device MMIO BARs for peer-to-peer DMA via dma-buf file descriptors rather than
VM_PFNMAPvirtual addresses.Signed-off-by David Matlack <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
David Matlack
maintainerRFC PATCH 5/7 vfio: selftests: Add iommu_map_file() helper
In reply to David MatlackAdd iommu_map_file() and __iommu_map_file() to map a file descriptor (such as a memfd or VFIO PCI dma-buf) into an iommufd IOAS via IOMMU_IOAS_MAP_FILE. Also skip regions without a host virtual address in __iommu_hva2iova(), since file-backed DMA mappings do not require one.
--- .../vfio/lib/include/libvfio/iommu.h | 9 ++++++ tools/testing/selftests/vfio/lib/iommu.c | 29 +++++++++++++++++++ 2 files changed, 38 insertions(+)
tools/testing/selftests/vfio/lib/include/libvfio/iommu.h +9 -0
@@ -42,6 +42,15 @@ static inline void iommu_map(struct iommu *iommu, struct dma_region *region)VFIO_ASSERT_EQ (__iommu_map (iommu, region),0 ); } +int __iommu_map_file (struct iommu *iommu,int fd,u64 file_offset, +struct dma_region *region); + +static inline void iommu_map_file (struct iommu *iommu,int fd,u64 file_offset, +struct dma_region *region) +{ +VFIO_ASSERT_EQ (__iommu_map_file (iommu, fd, file_offset, region),0 ); +} +int __iommu_unmap (struct iommu *iommu,struct dma_region *region,u64 *unmapped);static inline void iommu_unmap (struct iommu *iommu,struct dma_region *region)
tools/testing/selftests/vfio/lib/iommu.c +29 -0
@@ -71,6 +71,9 @@ int __iommu_hva2iova(struct iommu *iommu, void *vaddr, iova_t *iova)struct dma_region *region;list_for_each_entry (region, &iommu->dma_regions, link) { +if (!region->vaddr) +continue ; +if (vaddr < region->vaddr)continue ;
@@ -149,6 +152,32 @@ int __iommu_map(struct iommu *iommu, struct dma_region *region)return 0 ; } +int __iommu_map_file (struct iommu *iommu,int fd,u64 file_offset, +struct dma_region *region) +{ +struct iommu_ioas_map_file args = { + .size =sizeof (args), + .flags =IOMMU_IOAS_MAP_READABLE | +IOMMU_IOAS_MAP_WRITEABLE | +IOMMU_IOAS_MAP_FIXED_IOVA , + .ioas_id = iommu->ioas_id, + .fd = fd, + .start = file_offset, + .length = region->size, + .iova = region->iova, + }; + +if (!iommu->iommufd) +return -EOPNOTSUPP ; + +if (ioctl (iommu->iommufd,IOMMU_IOAS_MAP_FILE , &args)) +return -errno; + +list_add (®ion->link, &iommu->dma_regions); + +return 0 ; +} +static int __vfio_iommu_unmap (int fd,u64 iova,u64 size,u32 flags,u64 *unmapped) {struct vfio_iommu_type1_dma_unmap args = {
Signature · 1 line
2.56.0.567.gdb80ae8cd5-goog
[email protected]
AI reviewRe: [RFC PATCH 5/7] vfio: selftests: Add iommu_map_file() helper
Sashiko has reviewed this patch and found no issues.
Quoted 6 lines
Add
iommu_map_file()and__iommu_map_file()to map a file descriptor (such as a memfd or VFIO PCI dma-buf) into an iommufd IOAS viaIOMMU_IOAS_MAP_FILE. Also skip regions without a host virtual address in__iommu_hva2iova(), since file-backed DMA mappings do not require one.Signed-off-by David Matlack <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
David Matlack
maintainerRFC PATCH 6/7 vfio: selftests: Add vfio_pci_find_p2p_pair() helper
In reply to David MatlackAdd vfio_pci_find_p2p_pair() to locate an initiator device with memcpy driver support and a distinct target device with a memory BAR of at least min_bar_size bytes from an array of VFIO PCI devices.
Factor out vfio_pci_has_p2p_path() as the central place to validate PCI topology and P2P DMA reachability between the two devices in the future.
--- .../lib/include/libvfio/vfio_pci_device.h | 8 ++++ .../selftests/vfio/lib/vfio_pci_device.c | 48 +++++++++++++++++++ 2 files changed, 56 insertions(+)
tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +8 -0
@@ -42,6 +42,12 @@ struct vfio_pci_device {struct vfio_pci_driver driver; }; +struct vfio_pci_p2p_pair { +struct vfio_pci_device *initiator; +struct vfio_pci_device *target; +struct vfio_pci_mem_bar mem_bar; +}; +#define dev_info (_dev, _fmt, ...) printf("%s: " _fmt, (_dev)->bdf, ##__VA_ARGS__)#define dev_err (_dev, _fmt, ...) fprintf(stderr, "%s: " _fmt, (_dev)->bdf, ##__VA_ARGS__)
@@ -171,6 +177,8 @@ static inline bool vfio_pci_device_match(struct vfio_pci_device *device,bool vfio_pci_find_mem_bar (struct vfio_pci_device *device,u64 min_size,struct vfio_pci_mem_bar *mem_bar); +bool vfio_pci_find_p2p_pair (struct vfio_pci_device **devices,int nr_devices, +u64 min_bar_size,struct vfio_pci_p2p_pair *pair);int __vfio_pci_dmabuf_create (struct vfio_pci_device *device,u32 region_index,u64 offset,u64 length,u32 open_flags);
tools/testing/selftests/vfio/lib/vfio_pci_device.c +48 -0
54 lines not on this page · raw message
Signature · 1 line
2.56.0.567.gdb80ae8cd5-goog
[email protected]
AI reviewRe: [RFC PATCH 6/7] vfio: selftests: Add vfio_pci_find_p2p_pair() helper
Sashiko has reviewed this patch and found no issues.
Quoted 8 lines
Add
vfio_pci_find_p2p_pair()to locate an initiator device with memcpy driver support and a distinct target device with a memory BAR of at leastmin_bar_sizebytes from an array of VFIO PCI devices.Factor out
vfio_pci_has_p2p_path()as the central place to validate PCI topology and P2P DMA reachability between the two devices in the future.Signed-off-by David Matlack <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
David Matlack
maintainerRFC PATCH 7/7 vfio: selftests: Add P2P DMA selftest
In reply to David MatlackAdd vfio_pci_p2pdma_test to validate peer-to-peer DMA between an initiator device driven by the VFIO selftests driver framework and a target device exposing a memory BAR located via vfio_pci_find_p2p_pair().
Map the target BAR into the IOMMU across all supported IOMMU modes (using VFIO_IOMMU_MAP_DMA on the BAR mmap for type1/type1v2 and VFIO_DEVICE_FEATURE_DMA_BUF + IOMMU_IOAS_MAP_FILE for iommufd) and test RAM-to-BAR writes, BAR-to-RAM reads, BAR-to-BAR copies, RAM-BAR-RAM roundtrips, and iommufd dmabuf revocation when PCI_COMMAND_MEMORY is cleared on the target.
--- tools/testing/selftests/vfio/Makefile | 1 + .../selftests/vfio/vfio_pci_p2pdma_test.c | 435 ++++++++++++++++++ 2 files changed, 436 insertions(+) create mode 100644 tools/testing/selftests/vfio/vfio_pci_p2pdma_test.c
tools/testing/selftests/vfio/Makefile +1 -0
@@ -12,6 +12,7 @@ TEST_GEN_PROGS += vfio_iommufd_setup_test TEST_GEN_PROGS += vfio_pci_device_test TEST_GEN_PROGS += vfio_pci_device_init_perf_test TEST_GEN_PROGS += vfio_pci_driver_test +TEST_GEN_PROGS += vfio_pci_p2pdma_test TEST_GEN_PROGS += vfio_pci_sriov_uapi_test TEST_FILES += scripts/cleanup.sh
tools/testing/selftests/vfio/vfio_pci_p2pdma_test.c new file +435 -0
435 lines not on this page · raw message
Signature · 1 line
2.56.0.567.gdb80ae8cd5-goog
[email protected]
AI reviewRe: [RFC PATCH 7/7] vfio: selftests: Add P2P DMA selftest
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] vfio: selftests: missing -ENOTTY skip check for __vfio_pci_dmabuf_create() --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] vfio: selftests: missing -ENOTTY skip check for __vfio_pci_dmabuf_create()
--
commit 3d8e1bca27abb87842a5dfc06a0cde5ad88fe0fb
Author: David Matlack <[email protected]>
vfio: selftests: Add P2P DMA selftest
This commit adds vfio_pci_p2pdma_test to validate peer-to-peer DMA between an initiator device and a target device's memory BAR across supported IOMMU modes, including dmabuf revocation.
Quoted 5 lines
tools/testing/selftests/vfio/vfio_pci_p2pdma_test.c new file +0
-0
[ ... ]
Quoted 2 lines
tools/testing/selftests/vfio/vfio_pci_p2pdma_test.c new file +2
-0+FIXTURE_SETUP (vfio_pci_p2pdma_test) +{
[ ... ]
tools/testing/selftests/vfio/vfio_pci_p2pdma_test.c line 229
Quoted 11 lines
tools/testing/selftests/vfio/vfio_pci_p2pdma_test.c new file +11
-0+if (self->iommu->iommufd) { +u32 bar_idx = self->mem_bar.bar->info.index; + + self->dmabuf_fd =__vfio_pci_dmabuf_create (self->target, + bar_idx, + self->mem_bar.offset, + bar_map_size, +O_RDWR |O_CLOEXEC ); +if (self->dmabuf_fd == -EOPNOTSUPP ) +SKIP (return,"Kernel does not support VFIO_DEVICE_FEATURE_DMA_BUF" ); +ASSERT_GE (self->dmabuf_fd,0 );
[Severity: Medium]
Should this check also skip when self->dmabuf_fd is -ENOTTY? When CONFIG_VFIO_PCI_DMABUF is disabled (or on kernels predating VFIO_DEVICE_FEATURE_DMA_BUF), the vfio_pci_core_feature_dma_buf() stub and vfio_pci_core_ioctl_feature() return -ENOTTY rather than -EOPNOTSUPP, causing the iommufd test variants to fail at ASSERT_GE() instead of skipping.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Hey David,
Quoted 7 lines · David Matlack
Is requiring userspace to re-issue
IOMMU_IOAS_MAP_FILEthe expected uAPI behavior?VFIO_DEVICE_RESET(and runtime PM transitions) on the target device also bracket the reset withvfio_pci_dma_buf_move(vdev, true)andvfio_pci_dma_buf_move(vdev, false), tearing down any activeIOMMU_IOAS_MAP_FILEmapping of that device's BAR even thoughPCI_COMMAND_MEMORYremains enabled whenVFIO_DEVICE_RESETreturns.
There was some discussion about this in the thread below. My understanding of Jason's point is that the current behavior, and differences between what revoking and move-capable importers see in regards to vfio-exported dma-buf is somewhat incidental and not necessarily desirable.
https://lore.kernel.org/all/[email protected]/
Quoted 7 lines
- Physical target devices and P2P topology checks: mem_bar_descs[] currently only matches QEMU's pci-testdev, and
vfio_pci_has_p2p_path()does not yet check PCI topology or ACS to verify that P2P traffic can flow between two physical endpoints.Suggestions on physical target BARs to add to mem_bar_descs[] and on checking P2P reachability from userspace are welcome.
FWIW, you can also exercise the "P2P" path with just a single device if you IOMMU map a device's BAR into it's own IOAS, and have the device DMA itself. For eventually validating aspects of Live Update, maybe you don't necessarily care if an actual peer device is being DMA'd?
Alex
David Matlack
maintainerQuoted 12 lines · Alex Mastro
Hey David,
Quoted 7 lines · David Matlack
Is requiring userspace to re-issue
IOMMU_IOAS_MAP_FILEthe expected uAPI behavior?VFIO_DEVICE_RESET(and runtime PM transitions) on the target device also bracket the reset withvfio_pci_dma_buf_move(vdev, true)andvfio_pci_dma_buf_move(vdev, false), tearing down any activeIOMMU_IOAS_MAP_FILEmapping of that device's BAR even thoughPCI_COMMAND_MEMORYremains enabled whenVFIO_DEVICE_RESETreturns.
Quoted 4 lines
There was some discussion about this in the thread below. My understanding of Jason's point is that the current behavior, and differences between what revoking and move-capable importers see in regards to vfio-exported dma-buf is somewhat incidental and not necessarily desirable.
Thanks, I will read through that thread. It seemed like a surprising uAPI.
Quoted 8 lines
https://lore.kernel.org/all/[email protected]/
Quoted 5 lines
- Physical target devices and P2P topology checks: mem_bar_descs[] currently only matches QEMU's pci-testdev, and
vfio_pci_has_p2p_path()does not yet check PCI topology or ACS to verify that P2P traffic can flow between two physical endpoints.
Quoted 7 lines
Quoted 2 lines
Suggestions on physical target BARs to add to mem_bar_descs[] and on checking P2P reachability from userspace are welcome.
FWIW, you can also exercise the "P2P" path with just a single device if you IOMMU map a device's BAR into it's own IOAS, and have the device DMA itself. For eventually validating aspects of Live Update, maybe you don't necessarily care if an actual peer device is being DMA'd?
The way I designed the P2P selftest support requires 2 things:
(A) initiator: A device that has a VFIO selftests driver.
(B) target: A device that can provide a memory BAR (vfio_pci_find_mem_bar()).
So far I haven't found a single device that can do both. But you are right that theoretically a single device could satisfy both requirements.
I can relax vfio_pci_has_p2p_path() to allow initiator == target.
Quoted 7 lines · David Matlack
Quoted 6 lines · Alex Mastro
Hey David,
Quoted 2 lines · David Matlack
Is requiring userspace to re-issue
IOMMU_IOAS_MAP_FILEthe expected uAPI behavior?
Yes. Even if we wanted to the locking is doesn't allow trying to do it automatically.
Quoted 8 lines
The way I designed the P2P selftest support requires 2 things:
(A) initiator: A device that has a VFIO selftests driver.
(B) target: A device that can provide a memory BAR (vfio_pci_find_mem_bar()).So far I haven't found a single device that can do both. But you are right that theoretically a single device could satisfy both requirements.
mlx5 can do both
Jason
David Matlack
maintainerQuoted 15 lines · Jason Gunthorpe
Quoted 7 lines · David Matlack
Quoted 6 lines · Alex Mastro
Hey David,
Quoted 2 lines · David Matlack
Is requiring userspace to re-issue
IOMMU_IOAS_MAP_FILEthe expected uAPI behavior?Yes. Even if we wanted to the locking is doesn't allow trying to do it automatically.
Quoted 2 lines
The way I designed the P2P selftest support requires 2 things:
Quoted 8 lines
Quoted 6 lines
(A) initiator: A device that has a VFIO selftests driver.
(B) target: A device that can provide a memory BAR (vfio_pci_find_mem_bar()).So far I haven't found a single device that can do both. But you are right that theoretically a single device could satisfy both requirements.
mlx5 can do both
Looks like GPUs and NVMe can do both as well.
How would we implement vfio_pci_find_mem_bar() for mlx5? BTW for the target device, by "a memory BAR" I meant "a region of a memory BAR that behaves like normal RAM". Sorry that was not clear.
Quoted 9 lines · David Matlack
Quoted 9 lines
Quoted 8 lines
The way I designed the P2P selftest support requires 2 things:
(A) initiator: A device that has a VFIO selftests driver.
(B) target: A device that can provide a memory BAR (vfio_pci_find_mem_bar()).So far I haven't found a single device that can do both. But you are right that theoretically a single device could satisfy both requirements.
Quoted 7 lines
Quoted 1 line
mlx5 can do both
Looks like GPUs and NVMe can do both as well.
How would we implement
vfio_pci_find_mem_bar()for mlx5? BTW for the target device, by "a memory BAR" I meant "a region of a memory BAR that behaves like normal RAM". Sorry that was not clear.
It's a litlle complicated, you have to allocate a "Device memory" (DM) from the FW, then you get a chunk of BAR that is like RAM.
There may also be some device boot time configuration that has to be done to turn it on, I forget.
Jason
Quoted 19 lines · David Matlack
Quoted 18 lines · Jason Gunthorpe
Quoted 7 lines · David Matlack
Quoted 6 lines · Alex Mastro
Hey David,
Quoted 2 lines · David Matlack
Is requiring userspace to re-issue
IOMMU_IOAS_MAP_FILEthe expected uAPI behavior?Yes. Even if we wanted to the locking is doesn't allow trying to do it automatically.
Quoted 5 lines
The way I designed the P2P selftest support requires 2 things:
(A) initiator: A device that has a VFIO selftests driver.
(B) target: A device that can provide a memory BAR (vfio_pci_find_mem_bar()).
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
So far I haven't found a single device that can do both. But you are right that theoretically a single device could satisfy both requirements.
mlx5 can do both
Looks like GPUs and NVMe can do both as well.
AFAIU the memory BARs for GPUs would need initialization. The internal VRAM needs to be exposed (mapped) to the BARs.
Do you think we should have a P2P device "setup" step that can be implemented for various types of devices?
Thanks,
Sami
David Matlack
maintainerQuoted 8 lines · Samiullah Khawaja
Quoted 1 line · David Matlack
Looks like GPUs and NVMe can do both as well.
AFAIU the memory BARs for GPUs would need initialization. The internal VRAM needs to be exposed (mapped) to the BARs.
Do you think we should have a P2P device "setup" step that can be implemented for various types of devices?
Yes, I think we need something like that.
I've been prototyping a new optional op in struct vfio_pci_driver_ops:
/*
* Set up a range of device memory, accessible through a BAR,
* that other devices can target with P2P DMA and that the CPU
* can access through mem_bar->vaddr.
*/
int (*mem_bar_setup)(struct vfio_pci_device *device, u64 min_size,
struct vfio_pci_mem_bar *mem_bar);
vfio_pci_find_mem_bar() falls back to calling it when the device does not match any entry in mem_bar_descs[], sanity checks the returned range, and then runs the existing memory verification on it. mem_bar_descs[] stays around for devices like pci-testdev that need no setup.
For NVIDIA GPUs, I had an LLM work on it and the setup was simpler than I expected, at least on Blackwell. BAR1 comes out of reset in physical mode, where BAR1 offset N maps directly to VRAM physical address N without going through the GPU MMU. The LLM confirmed this on by writing VRAM through BAR1 and reading it back through the BAR0 PRAMIN window (and vice versa) at offsets from 256MiB up to 178GiB.
The nv_falcon implementation only has to:
- Wait for FSP to finish booting, which takes a few seconds after the reset VFIO does when the device is opened.
- Make sure BAR1 is in physical mode.
- Return the range of BAR1 that maps VRAM (the size comes from NV_USABLE_FB_SIZE_IN_MB), staying clear of firmware reservations at either end.
With that the P2P test passes with a ConnectX-7 (mlx5 driver) as the initiator and B200 VRAM as the target.
A couple of open questions:
1. The prototype allows .mem_bar_setup() to be called before .init(). That is enough for GPUs, but as Jason described, mlx5 device memory has to be allocated from firmware, which needs the command interface and therefore init(). Supporting mlx5 as a target would mean running .mem_bar_setup() after .init().
2. nv_falcon does not support Falcon DMA on Blackwell yet, so a B200 can only be a target for now. That requires a driver to be able to probe a device that cannot do memcpy. I am not sure I want to go down that road.