Skip to content

vfio: selftest: Add NVIDIA GPU Falcon DMA test driver

Rubin Du kvm linux-kselftest lkml 5 versions – Applied

Status from this message: Applied to vfio next branch for v7.3. Thanks,

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v14 Latest4 patches 2 repliesApplied
  2. v124 patches 6 repliesSuperseded
  3. v113 patches 1 replySuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v102 patches4 repliesSuperseded
v92 patches5 repliesSuperseded

Changes in v14 from the cover letter

- Re-tested with vfio_pci_driver_test on K2200, M60, P100, P4000, V100, T4, RTX 3090, RTX 4000 Ada and H100.
- Alex prepared this respin addressing David's review of v13.
- Patch 1 & 3: No change.
- Patch 2: R-b collected.
- Patch 4: address David's review -
- Include <linux/types.h>
- Factor out get_elapsed_ms()
- Rename NV_*_BOOT_COMPLETE_MASK to _SUCCESS
- Drop the redundant falcon_enable() and move the no_outside_reset check to the top of falcon_enable() so falcon_reset() is unconditional
- Fold dev_err() and size_to_dma_encoding() into nv_falcon_dma()
- Drop gpu->memcpy_count and the redundant GPU_ARCH_UNKNOWN assert
- Alignment fixes

Patches in v14 Applied

#Subject and filesSizeRepliesApplied
1/4vfio: selftests: Add memcpy chunking to vfio_pci_driver_memcpy()tools/testing/selftests/vfio/lib/vfio_pci_driver.c tools/testing/selftests/vfio/vfio_pci_driver_test.c+26 -100 replies
2/4vfio: selftests: Add generic PCI command register helperstools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h+14 -00 replies
3/4vfio: selftests: Allow drivers without send_msi() supporttools/testing/selftests/vfio/vfio_pci_driver_test.c+26 -130 replies
4/4vfio: selftests: Add NVIDIA Falcon driver for DMA testingtools/testing/selftests/vfio/lib/drivers/nv_falcon/hw.h tools/testing/selftests/vfio/lib/drivers/nv_falcon/nv_falcon.c tools/testing/selftests/vfio/lib/libvfio.mk +1 more+1140 -01 reply

Reviews trailers per patch

All 4 patches have a Reviewed-by or Acked-by.

PatchDavid Matlack
1/4R-b
2/4R-b
3/4R-b
4/4A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v14 thread

Cover letter [PATCH v14 0/4] vfio: selftest: Add NVIDIA GPU Falcon DMA test driver

Patch 1:

Add a chunking loop to vfio_pci_driver_memcpy() so that it handles arbitrarily sized memcpy requests by breaking them into max_memcpy_size-sized chunks. This allows tests to request any memcpy size. Update the test to use a size of 4x max_memcpy_size to exercise the chunking logic.

Patch 2:

Add vfio_pci_cmd_set()/vfio_pci_cmd_clear() macros for PCI_COMMAND operations.

Patch 3:

Allow drivers that cannot trigger MSI interrupts to leave the send_msi callback NULL. Tests check ops->send_msi directly and gracefully skip MSI-related operations when it is absent.

Patch 4:

Introduce the nv_falcon plugin driver, which extracts and adapts relevant functionality from NVIDIA's gpu-admin-tools project [1] and integrates it into the VFIO selftest framework. As a result, any system equipped with a PCIe slot and a supported NVIDIA GPU can now run VFIO DMA selftests using commonly available hardware.

Falcons are general-purpose microcontrollers present on NVIDIA GPUs that can perform DMA operations between system memory and device memory.

[1] https://github.com/NVIDIA/gpu-admin-tools

Changes for v14:
- Re-tested with vfio_pci_driver_test on K2200, M60, P100, P4000, V100, T4, RTX 3090, RTX 4000 Ada and H100.
- Alex prepared this respin addressing David's review of v13.
- Patch 1 & 3: No change.
- Patch 2: R-b collected.
- Patch 4: address David's review -
- Include <linux/types.h>
- Factor out get_elapsed_ms()
- Rename NV_*_BOOT_COMPLETE_MASK to _SUCCESS
- Drop the redundant falcon_enable() and move the no_outside_reset check to the top of falcon_enable() so falcon_reset() is unconditional
- Fold dev_err() and size_to_dma_encoding() into nv_falcon_dma()
- Drop gpu->memcpy_count and the redundant GPU_ARCH_UNKNOWN assert
- Alignment fixes

Note on version numbering: v1 through v9 were internal review iterations that were mistakenly carried over to the upstream submission. Apologies for the confusion, the internal changelog has been dropped.

Rubin Du (4):
vfio: selftests: Add memcpy chunking to vfio_pci_driver_memcpy()
vfio: selftests: Add generic PCI command register helpers
vfio: selftests: Allow drivers without send_msi() support
vfio: selftests: Add NVIDIA Falcon driver for DMA testing

 .../selftests/vfio/lib/drivers/nv_falcon/hw.h | 352 ++++++++
 .../vfio/lib/drivers/nv_falcon/nv_falcon.c    | 783 ++++++++++++++++++
 .../lib/include/libvfio/vfio_pci_device.h     |  14 +
 tools/testing/selftests/vfio/lib/libvfio.mk   |   2 +
 .../selftests/vfio/lib/vfio_pci_driver.c      |  21 +-
 .../selftests/vfio/vfio_pci_driver_test.c     |  57 +-
 6 files changed, 1206 insertions(+), 23 deletions(-)
 create mode 100644 tools/testing/selftests/vfio/lib/drivers/nv_falcon/hw.h
 create mode 100644 tools/testing/selftests/vfio/lib/drivers/nv_falcon/nv_falcon.c

base-commit: d0c3bcd5b8976159d835a897254048e078f447e6 --
2.43.0