Skip to contentSkip to conversation

pds_vfio driver

PATCH Superseded Brett Creeley 4 participants 24 messages Apr 22, 2023 – Jun 2, 2023

Subsystems (9)

v9 of 14 Superseded Latest is v14 →

lore
Share & tools
lore

Brett Creeley

author maintainer
#0 lore raw

v9 cover pds_vfio driver

This is a patchset for a new vendor specific VFIO driver (pds_vfio) for use with the AMD/Pensando Distributed Services Card (DSC). This driver makes use of the newly accepted pds_core driver, which can be referenced at:

https://lore.kernel.org/netdev/168206343035.30967.14987352827118974870.git-patchwork-notify@kernel.org/

This driver will use the pds_core device's adminq as the VFIO control path to the DSC. In order to make adminq calls, the VFIO instance makes use of functions exported by the pds_core driver.

In order to receive events from pds_core, the pds_vfio driver registers to a private notifier. This is needed for various events that come from the device.

An ASCII diagram of a VFIO instance looks something like this and can be used with the VFIO subsystem to provide the VF device VFIO and live migration support.

                               .------.  .-----------------------.
                               | QEMU |--|  VM  .-------------.  |
                               '......'  |      |   Eth VF    |  |
                                  |      |      .-------------.  |
                                  |      |      |  SR-IOV VF  |  |
                                  |      |      '-------------'  |
                                  |      '------------||---------'
                               .--------------.       ||
                               |/dev/<vfio_fd>|       ||
                               '--------------'       ||
Host Userspace                         |              ||
===================================================   ||
Host Kernel                            |              ||
                                  .--------.          ||
                                  |vfio-pci|          ||
                                  '--------'          ||
       .------------------.           ||              ||
       |   | exported API |<----+     ||              ||
       |   '--------------|     |     ||              ||
       |                  |    .-------------.        ||
       |     pds_core     |--->|   pds_vfio  |        ||
       '------------------' |  '-------------'        ||
               ||           |         ||              ||
             09:00.0     notifier    09:00.1          ||
== PCI ===============================================||=====
               ||                     ||              ||
          .----------.          .----------.          ||
    ,-----|    PF    |----------|    VF    |-------------------,
    |     '----------'          '----------'  |       VF       |
    |                     DSC                 |  data/control  |
    |                                         |      path      |
    -----------------------------------------------------------

The pds_vfio driver is targeted to reside in drivers/vfio/pci/pds. It makes use of and introduces new files in the common include/linux/pds include directory.

Changes:

v9:
- Various fixes/suggestions by Alex Williamson
- Fix how ID is generated in client registration
- Add helper functions to get the VF's struct device and struct pci_dev pointers instead of caching the struct pci dev
- Remove redundant pds_vfio_lm_state() function and remove any places this was being called
- Fix multi-line comments to follow standard convention
- Remove confusing comments in pds_vfio_step_device_state_locked() since the driver's migration states align with the VFIO documentation
- Validate pdsc returned from pdsc_get_pf_struct()
- Various fixes/suggestions by Jason Gunthorpe
- Use struct pdsc instead of void *
- Use {} instead of {0} for structure initialization
- Use unions on the stack instead of casting to the union when sending AQ commands, which required including pds_lm.h in pds_adminq.h
- Replace use of dma_alloc_coherent() when creating the sgl DMA entries for the LM file
- Remove cached struct device *coredev and instead use pci_physfn() to get the pds_core's struct device pointer
- Drop the recovery work item and call pds_vfio_recovery() directly from the notifier callback
- Remove unnecessary #define for "pds_vfio_lm" and just use the string inline to the anon_inode_getfile() argument
- Fix LM file reference counting
- Move initialization of some struct members to when the struct is being initialized for AQ commands
- Make use of GFP_KERNEL_ACCOUNT where it makes sense
- Replace PDS_VFIO_DRV_NAME with KBUILD_MODNAME
- Update to latest pds_core exported functions
- Remove duplicated prototypes for pds_vfio_dma_logging_[start|stop|report] from lm.h
- Hold pds_vfio->state_mutex while starting, stopping, and reporting dirty page tracking in pds_vfio_dma_logging_[start|stop|report]
- Remove duplicate PDS_DEV_TYPE_LM_STR define from pds_lm.h that's already included in pds_common.h
- Replace use of dma_alloc_coherent() when creating the sgl DMA entries for the dirty bitmaps

v8:
https://lore.kernel.org/netdev/[email protected]/
- provide default iommufd callbacks for bind_iommufd, unbind_iommufd, and attach_ioas for the VFIO device as suggested by Shameerali Kolothum Thodi

v7:
https://lore.kernel.org/netdev/[email protected]/
- Disable and clean up dirty page tracking when the VFIO device is closed
- Various improvements suggested by Simon Horman:
- Fix RCT in vfio_combine_iova_ranges()
- Simplify function exit paths by removing unnecessary goto labels
- Cleanup pds_vifo_print_guest_region_info() by adding a goto label for freeing memory, which allowed for reduced indentation on a for loop
- Where possible use C99 style for loops

v6:
https://lore.kernel.org/netdev/[email protected]/
- As suggested by Alex Williamson, use pci_domain_nr() macro to make sure the pds_vfio client's devname is unique
- Remove unnecessary forward declaration and include
- Fix copyright comment to use correct company name
- Remove "." from struct documentation for consistency

v5:
https://lore.kernel.org/netdev/[email protected]/
- Fix SPDX comments in .h files
- Remove adminqcq argument from pdsc_post_adminq() uses
- Unregister client on vfio_pci_core_register_device() failure
- Other minor checkpatch issues

v4:
https://lore.kernel.org/netdev/[email protected]/
- Update cover letter ASCII diagram to reflect new driver architecture
- Remove auxiliary driver implementation
- Use pds_core's exported functions to communicate with the device
- Implement and register notifier for events from the device/pds_core
- Use module_pci_driver() macro since auxiliary driver configuration is no longer needed in __init/__exit

v3:
https://lore.kernel.org/netdev/[email protected]/
- Update copyright year to 2023 and use "Advanced Micro Devices, Inc." for the company name
- Clarify the fact that AMD/Pensando's VFIO solution is device type agnostic, which aligns with other current VFIO solutions
- Add line in drivers/vfio/pci/Makefile to build pds_vfio
- Move documentation to amd sub-directory
- Remove some dead code due to the pds_core implementation of listening to BIND/UNBIND events
- Move a dev_dbg() to a previous patch in the series
- Add implementation for vfio_migration_ops.migration_get_data_size to return the maximum possible device state size

RFC to v2:
https://lore.kernel.org/all/[email protected]/
- Implement state transitions for VFIO_MIGRATION_P2P flag
- Improve auxiliary driver probe by returning EPROBE_DEFER when the PCI driver is not set up correctly
- Add pointer to docs in Documentation/networking/device_drivers/ethernet/index.rst

RFC:
https://lore.kernel.org/all/[email protected]/

Brett Creeley (7):
vfio: Commonize combine_ranges for use in other VFIO drivers vfio/pds: Initial support for pds_vfio VFIO driver vfio/pds: register with the pds_core PF
vfio/pds: Add VFIO live migration support vfio/pds: Add support for dirty page tracking vfio/pds: Add support for firmware recovery vfio/pds: Add Kconfig and documentation

 .../device_drivers/ethernet/amd/pds_vfio.rst  |  79 +++
 .../device_drivers/ethernet/index.rst         |   1 +
 MAINTAINERS                                   |   7 +
 drivers/vfio/pci/Kconfig                      |   2 +
 drivers/vfio/pci/Makefile                     |   2 +
 drivers/vfio/pci/mlx5/cmd.c                   |  48 +-
 drivers/vfio/pci/pds/Kconfig                  |  20 +
 drivers/vfio/pci/pds/Makefile                 |  11 +
 drivers/vfio/pci/pds/cmds.c                   | 487 +++++++++++++++
 drivers/vfio/pci/pds/cmds.h                   |  25 +
 drivers/vfio/pci/pds/dirty.c                  | 562 ++++++++++++++++++
 drivers/vfio/pci/pds/dirty.h                  |  45 ++
 drivers/vfio/pci/pds/lm.c                     | 436 ++++++++++++++
 drivers/vfio/pci/pds/lm.h                     |  41 ++
 drivers/vfio/pci/pds/pci_drv.c                | 214 +++++++
 drivers/vfio/pci/pds/pci_drv.h                |   9 +
 drivers/vfio/pci/pds/vfio_dev.c               | 248 ++++++++
 drivers/vfio/pci/pds/vfio_dev.h               |  46 ++
 drivers/vfio/vfio_main.c                      |  47 ++
 include/linux/pds/pds_adminq.h                |  17 +
 include/linux/pds/pds_lm.h                    | 390 ++++++++++++
 include/linux/vfio.h                          |   3 +
 22 files changed, 2693 insertions(+), 47 deletions(-)
 create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst
 create mode 100644 drivers/vfio/pci/pds/Kconfig
 create mode 100644 drivers/vfio/pci/pds/Makefile
 create mode 100644 drivers/vfio/pci/pds/cmds.c
 create mode 100644 drivers/vfio/pci/pds/cmds.h
 create mode 100644 drivers/vfio/pci/pds/dirty.c
 create mode 100644 drivers/vfio/pci/pds/dirty.h
 create mode 100644 drivers/vfio/pci/pds/lm.c
 create mode 100644 drivers/vfio/pci/pds/lm.h
 create mode 100644 drivers/vfio/pci/pds/pci_drv.c
 create mode 100644 drivers/vfio/pci/pds/pci_drv.h
 create mode 100644 drivers/vfio/pci/pds/vfio_dev.c
 create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
 create mode 100644 include/linux/pds/pds_lm.h
Signature · 1 line
2.17.1
author maintainer
#1 lore raw

PATCH v9 1/7 vfio: Commonize combine_ranges for use in other VFIO drivers

Currently only Mellanox uses the combine_ranges function. The new pds_vfio driver also needs this function. So, move it to a common location for other vendor drivers to use.

Also, Simon Harmon noticed that RCT ordering was not followed for vfio_combin_iova_ranges(), so fix that.

---
 drivers/vfio/pci/mlx5/cmd.c | 48 +------------------------------------
 drivers/vfio/vfio_main.c    | 47 ++++++++++++++++++++++++++++++++++++
 include/linux/vfio.h        |  3 +++
 3 files changed, 51 insertions(+), 47 deletions(-)
drivers/vfio/pci/mlx5/cmd.c +1 -47
index deed156e6165..7f6c51992a15 100644
@@ -732,52 +732,6 @@ void mlx5fv_cmd_clean_migf_resources(struct mlx5_vf_migration_file *migf)
 	mlx5vf_cmd_dealloc_pd(migf);
 }
 
-static void combine_ranges(struct rb_root_cached *root, u32 cur_nodes,
-			   u32 req_nodes)
-{
-	struct interval_tree_node *prev, *curr, *comb_start, *comb_end;
-	unsigned long min_gap;
-	unsigned long curr_gap;
-
-	/* Special shortcut when a single range is required */
-	if (req_nodes == 1) {
-		unsigned long last;
-
-		curr = comb_start = interval_tree_iter_first(root, 0, ULONG_MAX);
-		while (curr) {
-			last = curr->last;
-			prev = curr;
-			curr = interval_tree_iter_next(curr, 0, ULONG_MAX);
-			if (prev != comb_start)
-				interval_tree_remove(prev, root);
-		}
-		comb_start->last = last;
-		return;
-	}
-
-	/* Combine ranges which have the smallest gap */
-	while (cur_nodes > req_nodes) {
-		prev = NULL;
-		min_gap = ULONG_MAX;
-		curr = interval_tree_iter_first(root, 0, ULONG_MAX);
-		while (curr) {
-			if (prev) {
-				curr_gap = curr->start - prev->last;
-				if (curr_gap < min_gap) {
-					min_gap = curr_gap;
-					comb_start = prev;
-					comb_end = curr;
-				}
-			}
-			prev = curr;
-			curr = interval_tree_iter_next(curr, 0, ULONG_MAX);
-		}
-		comb_start->last = comb_end->last;
-		interval_tree_remove(comb_end, root);
-		cur_nodes--;
-	}
-}
-
 static int mlx5vf_create_tracker(struct mlx5_core_dev *mdev,
 				 struct mlx5vf_pci_core_device *mvdev,
 				 struct rb_root_cached *ranges, u32 nnodes)
@@ -800,7 +754,7 @@ static int mlx5vf_create_tracker(struct mlx5_core_dev *mdev,
 	int i;
 
 	if (num_ranges > max_num_range) {
-		combine_ranges(ranges, nnodes, max_num_range);
+		vfio_combine_iova_ranges(ranges, nnodes, max_num_range);
 		num_ranges = max_num_range;
 	}
 
drivers/vfio/vfio_main.c +47 -0

53 lines not on this page · raw message

include/linux/vfio.h +3 -0
index 93134b023968..10db5aa3e985 100644
@@ -241,6 +241,9 @@ int vfio_mig_get_next_state(struct vfio_device *device,
 			    enum vfio_device_mig_state new_fsm,
 			    enum vfio_device_mig_state *next_fsm);
 
+void vfio_combine_iova_ranges(struct rb_root_cached *root, u32 cur_nodes,
+			      u32 req_nodes);
+
 /*
  * External user API
  */
Signature · 1 line
2.17.1
author maintainer
#2 lore raw

PATCH v9 2/7 vfio/pds: Initial support for pds_vfio VFIO driver

In reply to Brett Creeley

This is the initial framework for the new pds_vfio device driver. This does the very basics of registering the PDS PCI device and configuring it as a VFIO PCI device.

With this change, the VF device can be bound to the pds_vfio driver on the host and presented to the VM as the VF's device type.

---
 drivers/vfio/pci/Makefile       |  2 +
 drivers/vfio/pci/pds/Makefile   |  8 ++++
 drivers/vfio/pci/pds/pci_drv.c  | 72 ++++++++++++++++++++++++++++++
 drivers/vfio/pci/pds/vfio_dev.c | 77 +++++++++++++++++++++++++++++++++
 drivers/vfio/pci/pds/vfio_dev.h | 20 +++++++++
 5 files changed, 179 insertions(+)
 create mode 100644 drivers/vfio/pci/pds/Makefile
 create mode 100644 drivers/vfio/pci/pds/pci_drv.c
 create mode 100644 drivers/vfio/pci/pds/vfio_dev.c
 create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
drivers/vfio/pci/Makefile +2 -0
index 24c524224da5..45167be462d8 100644
@@ -11,3 +11,5 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
 obj-$(CONFIG_MLX5_VFIO_PCI)           += mlx5/
 
 obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/
+
+obj-$(CONFIG_PDS_VFIO_PCI) += pds/
drivers/vfio/pci/pds/Makefile new file +8 -0
new file mode 100644
index 000000000000..e1a55ae0f079
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Advanced Micro Devices, Inc.
+
+obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o
+
+pds_vfio-y := \
+	pci_drv.o	\
+	vfio_dev.o
drivers/vfio/pci/pds/pci_drv.c new file +72 -0

72 lines not on this page · raw message

drivers/vfio/pci/pds/vfio_dev.c new file +77 -0

77 lines not on this page · raw message

drivers/vfio/pci/pds/vfio_dev.h new file +20 -0
new file mode 100644
index 000000000000..66cfcab5b5bf
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc. */
+
+#ifndef _VFIO_DEV_H_
+#define _VFIO_DEV_H_
+
+#include <linux/pci.h>
+#include <linux/vfio_pci_core.h>
+
+struct pds_vfio_pci_device {
+	struct vfio_pci_core_device vfio_coredev;
+
+	int vf_id;
+	int pci_id;
+};
+
+const struct vfio_device_ops *pds_vfio_ops_info(void);
+struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev);
+
+#endif /* _VFIO_DEV_H_ */
Signature · 1 line
2.17.1
#3 lore raw

Re: [PATCH v9 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver

drivers/vfio/pci/pds/pci_drv.c line 21

Quoted 4 lines · Brett Creeley
drivers/vfio/pci/pds/pci_drv.c new file +4 -0
+static int
+pds_vfio_pci_probe(struct pci_dev *pdev,
+		   const struct pci_device_id *id)
+{

This indenting scheme is not kernel style. I generally suggest people run their code through clang-format and go through and take most of the changes. Most of what it sugges for this series is good

This GNU style of left aligning the function name should not be in the kernel.

Jason

maintainer
#4 lore raw

Re: [PATCH v9 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver

Quoted 6 lines · On Thu, 4 May 2023 14:26:43 -0300 Jason Gunthorpe wrote:

This indenting scheme is not kernel style. I generally suggest people run their code through clang-format and go through and take most of the changes. Most of what it sugges for this series is good

This GNU style of left aligning the function name should not be in the kernel.

FTR that's not a kernel-wide rule. Please scope your coding style suggestions to your subsystem, you may confuse people.

#5 lore raw

Re: [PATCH v9 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver

Quoted 10 lines · Jakub Kicinski
Quoted 6 lines · On Thu, 4 May 2023 14:26:43 -0300 Jason Gunthorpe wrote:

This indenting scheme is not kernel style. I generally suggest people run their code through clang-format and go through and take most of the changes. Most of what it sugges for this series is good

This GNU style of left aligning the function name should not be in the kernel.

FTR that's not a kernel-wide rule. Please scope your coding style suggestions to your subsystem, you may confuse people.

It is what Documentation/process/coding-style.rst expects.

It is good advice for new submitters to follow the common style guide consistently. The places that want different can ask for their differences during the first review, we don't need to confuse people with the reality that everything is an exception to someone somewhere.

Jason

maintainer
#6 lore raw

Re: [PATCH v9 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver

Quoted 9 lines · On Thu, 4 May 2023 19:41:54 -0300 Jason Gunthorpe wrote:
Quoted 6 lines · Jakub Kicinski
Quoted 2 lines · On Thu, 4 May 2023 14:26:43 -0300 Jason Gunthorpe wrote:

This GNU style of left aligning the function name should not be in the kernel.

FTR that's not a kernel-wide rule. Please scope your coding style suggestions to your subsystem, you may confuse people.

It is what Documentation/process/coding-style.rst expects.

A reference to the section? You mean the vague mentions of the GNU coding style? That's just a tiny part of the GNU style, the separation of arg types is probably the most offensive.

If the function declaration does not fit in 80 chars breaking the type off as a separate line is often a very reasonable choice.

Anyway, I shouldn't complain, networking still has its odd rules. Probably why people making up rules for no strong reason is on my mind.

Quoted 4 lines

It is good advice for new submitters to follow the common style guide consistently. The places that want different can ask for their differences during the first review, we don't need to confuse people with the reality that everything is an exception to someone somewhere.

Yeah.

#7 lore raw

Re: [PATCH v9 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver

Quoted 6 lines · Jakub Kicinski
Quoted 5 lines · On Thu, 4 May 2023 19:41:54 -0300 Jason Gunthorpe wrote:
Quoted 4 lines · Jakub Kicinski
Quoted 2 lines · On Thu, 4 May 2023 14:26:43 -0300 Jason Gunthorpe wrote:

This GNU style of left aligning the function name should not be in the kernel.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

FTR that's not a kernel-wide rule. Please scope your coding style suggestions to your subsystem, you may confuse people.

It is what Documentation/process/coding-style.rst expects.

A reference to the section? You mean the vague mentions of the GNU coding style?

Here I was looking at the "left indent the function name" - that is a GNUism. IIRC the justification is it makes it easy to find the function with 'grep "^func"'

Coding style says:

Descendants are always substantially shorter than the parent and are placed substantially to the right. A very commonly used style is to align descendants to a function open parenthesis.

So this patch had things like this:

+static void
+pds_vfio_pci_remove(struct pci_dev *pdev)

The first line is shorter than the second and the second is left not right placed. It doesn't even need wrapping.

I know some people like to do this in some parts of the tree regardless of coding-style. The GNU idea is sort of reasonable after all.

Quoted 2 lines

If the function declaration does not fit in 80 chars breaking the type off as a separate line is often a very reasonable choice.

Reasonable yes, but not "common" :)

Quoted 2 lines

Anyway, I shouldn't complain, networking still has its odd rules. Probably why people making up rules for no strong reason is on my mind.

I usually try to ignore most of the style details, but this one stood out. When I checked the series against clang-format it was pretty good otherwise. A few minor fine tunings on some line wrapping :)

Jason

#8 lore raw

Re: [PATCH v9 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver

In reply to Jason Gunthorpe
Quoted 14 lines · Jason Gunthorpe

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

drivers/vfio/pci/pds/pci_drv.c line 21

Quoted 4 lines · Brett Creeley
drivers/vfio/pci/pds/pci_drv.c new file +4 -0
+static int
+pds_vfio_pci_probe(struct pci_dev *pdev,
+                const struct pci_device_id *id)
+{

This indenting scheme is not kernel style. I generally suggest people run their code through clang-format and go through and take most of the changes. Most of what it sugges for this series is good

Quoted 4 lines

This GNU style of left aligning the function name should not be in the kernel.

Jason

I will align and fix it in the next revision. Thanks for the review.

Brett

#9 lore raw

Re: [PATCH v9 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver

In reply to Brett Creeley
Quoted 14 lines · Brett Creeley
drivers/vfio/pci/pds/vfio_dev.c new file +14 -0
+static const struct vfio_device_ops
+pds_vfio_ops = {
+	.name = "pds-vfio",
+	.init = pds_vfio_init_device,
+	.release = vfio_pci_core_release_dev,
+	.open_device = pds_vfio_open_device,
+	.close_device = vfio_pci_core_close_device,
+	.ioctl = vfio_pci_core_ioctl,
+	.device_feature = vfio_pci_core_ioctl_feature,
+	.read = vfio_pci_core_read,
+	.write = vfio_pci_core_write,
+	.mmap = vfio_pci_core_mmap,
+	.request = vfio_pci_core_request,
+	.match = vfio_pci_core_match,

drivers/vfio/pci/pds/vfio_dev.c line 77

Quoted 10 lines
drivers/vfio/pci/pds/vfio_dev.c new file +10 -0
+	.bind_iommufd = vfio_iommufd_physical_bind,
+	.unbind_iommufd = vfio_iommufd_physical_unbind,
+	.attach_ioas = vfio_iommufd_physical_attach_ioas,
+};
+
+const struct vfio_device_ops *
+pds_vfio_ops_info(void)
+{
+	return &pds_vfio_ops;
+}

No reason for a function like this

It is a bit strange to split up the driver files so the registration is in a different file than the ops implementation.

Jason

#10 lore raw

Re: [PATCH v9 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver

Quoted 32 lines · Jason Gunthorpe

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 14 lines · Brett Creeley
drivers/vfio/pci/pds/vfio_dev.c new file +14 -0
+static const struct vfio_device_ops
+pds_vfio_ops = {
+     .name = "pds-vfio",
+     .init = pds_vfio_init_device,
+     .release = vfio_pci_core_release_dev,
+     .open_device = pds_vfio_open_device,
+     .close_device = vfio_pci_core_close_device,
+     .ioctl = vfio_pci_core_ioctl,
+     .device_feature = vfio_pci_core_ioctl_feature,
+     .read = vfio_pci_core_read,
+     .write = vfio_pci_core_write,
+     .mmap = vfio_pci_core_mmap,
+     .request = vfio_pci_core_request,
+     .match = vfio_pci_core_match,

drivers/vfio/pci/pds/vfio_dev.c line 77

Quoted 10 lines
drivers/vfio/pci/pds/vfio_dev.c new file +10 -0
+     .bind_iommufd = vfio_iommufd_physical_bind,
+     .unbind_iommufd = vfio_iommufd_physical_unbind,
+     .attach_ioas = vfio_iommufd_physical_attach_ioas,
+};
+
+const struct vfio_device_ops *
+pds_vfio_ops_info(void)
+{
+     return &pds_vfio_ops;
+}

No reason for a function like this

Quoted 4 lines

It is a bit strange to split up the driver files so the registration is in a different file than the ops implementation.

Jason

I will fix this in the next revision. Thanks for the review.

Brett

#11 lore raw

Re: [PATCH v9 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver

In reply to Jason Gunthorpe
Quoted 32 lines · Jason Gunthorpe

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 14 lines · Brett Creeley
drivers/vfio/pci/pds/vfio_dev.c new file +14 -0
+static const struct vfio_device_ops
+pds_vfio_ops = {
+     .name = "pds-vfio",
+     .init = pds_vfio_init_device,
+     .release = vfio_pci_core_release_dev,
+     .open_device = pds_vfio_open_device,
+     .close_device = vfio_pci_core_close_device,
+     .ioctl = vfio_pci_core_ioctl,
+     .device_feature = vfio_pci_core_ioctl_feature,
+     .read = vfio_pci_core_read,
+     .write = vfio_pci_core_write,
+     .mmap = vfio_pci_core_mmap,
+     .request = vfio_pci_core_request,
+     .match = vfio_pci_core_match,

drivers/vfio/pci/pds/vfio_dev.c line 77

Quoted 10 lines
drivers/vfio/pci/pds/vfio_dev.c new file +10 -0
+     .bind_iommufd = vfio_iommufd_physical_bind,
+     .unbind_iommufd = vfio_iommufd_physical_unbind,
+     .attach_ioas = vfio_iommufd_physical_attach_ioas,
+};
+
+const struct vfio_device_ops *
+pds_vfio_ops_info(void)
+{
+     return &pds_vfio_ops;
+}

No reason for a function like this

Quoted 2 lines

It is a bit strange to split up the driver files so the registration is in a different file than the ops implementation.

The reason I did this was to separate the pci functionality from the vfio device functionality. There are other similar examples of uses like this. I ended up not changing this for v10 because it was intentional due to the reason I stated above.

Quoted 2 lines

Jason

author maintainer
#12 lore raw

PATCH v9 3/7 vfio/pds: register with the pds_core PF

In reply to Brett Creeley

The pds_core driver will supply adminq services, so find the PF and register with the DSC services.

Use the following commands to enable a VF:
echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs

---
 drivers/vfio/pci/pds/Makefile   |  1 +
 drivers/vfio/pci/pds/cmds.c     | 47 +++++++++++++++++++++++++++++++++
 drivers/vfio/pci/pds/cmds.h     | 10 +++++++
 drivers/vfio/pci/pds/pci_drv.c  | 19 +++++++++++++
 drivers/vfio/pci/pds/pci_drv.h  |  9 +++++++
 drivers/vfio/pci/pds/vfio_dev.c | 11 ++++++++
 drivers/vfio/pci/pds/vfio_dev.h |  6 +++++
 include/linux/pds/pds_lm.h      | 11 ++++++++
 8 files changed, 114 insertions(+)
 create mode 100644 drivers/vfio/pci/pds/cmds.c
 create mode 100644 drivers/vfio/pci/pds/cmds.h
 create mode 100644 drivers/vfio/pci/pds/pci_drv.h
 create mode 100644 include/linux/pds/pds_lm.h
drivers/vfio/pci/pds/Makefile +1 -0
index e1a55ae0f079..87581111fa17 100644
@@ -4,5 +4,6 @@
 obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o
 
 pds_vfio-y := \
+	cmds.o		\
 	pci_drv.o	\
 	vfio_dev.o
drivers/vfio/pci/pds/cmds.c new file +47 -0

47 lines not on this page · raw message

drivers/vfio/pci/pds/cmds.h new file +10 -0
new file mode 100644
index 000000000000..4c592afccf89
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc. */
+
+#ifndef _CMDS_H_
+#define _CMDS_H_
+
+int pds_vfio_register_client_cmd(struct pds_vfio_pci_device *pds_vfio);
+void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio);
+
+#endif /* _CMDS_H_ */
drivers/vfio/pci/pds/pci_drv.c +19 -0
index 3856a4e78c8d..dca7d457bd8c 100644
@@ -8,9 +8,13 @@
 #include <linux/types.h>
 #include <linux/vfio.h>
 
+#include <linux/pds/pds_common.h>
 #include <linux/pds/pds_core_if.h>
+#include <linux/pds/pds_adminq.h>
 
 #include "vfio_dev.h"
+#include "pci_drv.h"
+#include "cmds.h"
 
 #define PDS_VFIO_DRV_DESCRIPTION	"AMD/Pensando VFIO Device Driver"
 #define PCI_VENDOR_ID_PENSANDO		0x1dd8
@@ -28,13 +32,27 @@ pds_vfio_pci_probe(struct pci_dev *pdev,
 		return PTR_ERR(pds_vfio);
 
 	dev_set_drvdata(&pdev->dev, &pds_vfio->vfio_coredev);
+	pds_vfio->pdsc = pdsc_get_pf_struct(pdev);
+	if (IS_ERR_OR_NULL(pds_vfio->pdsc)) {
+		err = PTR_ERR(pds_vfio->pdsc) ?: -ENODEV;
+		goto out_put_vdev;
+	}
 
 	err = vfio_pci_core_register_device(&pds_vfio->vfio_coredev);
 	if (err)
 		goto out_put_vdev;
 
+	err = pds_vfio_register_client_cmd(pds_vfio);
+	if (err) {
+		dev_err(&pdev->dev, "failed to register as client: %pe\n",
+			ERR_PTR(err));
+		goto out_unregister_coredev;
+	}
+
 	return 0;
 
+out_unregister_coredev:
+	vfio_pci_core_unregister_device(&pds_vfio->vfio_coredev);
 out_put_vdev:
 	vfio_put_device(&pds_vfio->vfio_coredev.vdev);
 	return err;
@@ -45,6 +63,7 @@ pds_vfio_pci_remove(struct pci_dev *pdev)
 {
 	struct pds_vfio_pci_device *pds_vfio = pds_vfio_pci_drvdata(pdev);
 
+	pds_vfio_unregister_client_cmd(pds_vfio);
 	vfio_pci_core_unregister_device(&pds_vfio->vfio_coredev);
 	vfio_put_device(&pds_vfio->vfio_coredev.vdev);
 }
drivers/vfio/pci/pds/pci_drv.h new file +9 -0
new file mode 100644
index 000000000000..e79bed12ed14
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc. */
+
+#ifndef _PCI_DRV_H
+#define _PCI_DRV_H
+
+#include <linux/pci.h>
+
+#endif /* _PCI_DRV_H */
drivers/vfio/pci/pds/vfio_dev.c +11 -0
index 0f70efec01e1..228b27a0aa60 100644
@@ -6,6 +6,12 @@
 
 #include "vfio_dev.h"
 
+struct pci_dev *
+pds_vfio_to_pci_dev(struct pds_vfio_pci_device *pds_vfio)
+{
+	return  pds_vfio->vfio_coredev.pdev;
+}
+
 struct pds_vfio_pci_device *
 pds_vfio_pci_drvdata(struct pci_dev *pdev)
 {
@@ -31,6 +37,11 @@ pds_vfio_init_device(struct vfio_device *vdev)
 	pds_vfio->vf_id = pci_iov_vf_id(pdev);
 	pds_vfio->pci_id = PCI_DEVID(pdev->bus->number, pdev->devfn);
 
+	dev_dbg(&pdev->dev, "%s: PF %#04x VF %#04x (%d) vf_id %d domain %d pds_vfio %p\n",
+		__func__, pci_dev_id(pdev->physfn),
+		pds_vfio->pci_id, pds_vfio->pci_id, pds_vfio->vf_id,
+		pci_domain_nr(pdev->bus), pds_vfio);
+
 	return 0;
 }
 
drivers/vfio/pci/pds/vfio_dev.h +6 -0
index 66cfcab5b5bf..92e8ff241ca8 100644
@@ -7,14 +7,20 @@
 #include <linux/pci.h>
 #include <linux/vfio_pci_core.h>
 
+struct pdsc;
+
 struct pds_vfio_pci_device {
 	struct vfio_pci_core_device vfio_coredev;
+	struct pdsc *pdsc;
 
 	int vf_id;
 	int pci_id;
+	u16 client_id;
 };
 
 const struct vfio_device_ops *pds_vfio_ops_info(void);
 struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev);
 
+struct pci_dev *pds_vfio_to_pci_dev(struct pds_vfio_pci_device *pds_vfio);
+
 #endif /* _VFIO_DEV_H_ */
include/linux/pds/pds_lm.h new file +11 -0
new file mode 100644
index 000000000000..26d17efeff28
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#ifndef _PDS_LM_H_
+#define _PDS_LM_H_
+
+#include "pds_common.h"
+
+#define PDS_LM_DEV_NAME		PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_LM_STR
+
+#endif /* _PDS_LM_H_ */
Signature · 1 line
2.17.1
author maintainer
#13 lore raw

PATCH v9 4/7 vfio/pds: Add VFIO live migration support

In reply to Brett Creeley

Add live migration support via the VFIO subsystem. The migration implementation aligns with the definition from uapi/vfio.h and uses the pds_core PF's adminq for device configuration.

The ability to suspend, resume, and transfer VF device state data is included along with the required admin queue command structures and implementations.

PDS_LM_CMD_SUSPEND and PDS_LM_CMD_SUSPEND_STATUS are added to support the VF device suspend operation.

PDS_LM_CMD_RESUME is added to support the VF device resume operation.

PDS_LM_CMD_STATUS is added to determine the exact size of the VF device state data.

PDS_LM_CMD_SAVE is added to get the VF device state data.

PDS_LM_CMD_RESTORE is added to restore the VF device with the previously saved data from PDS_LM_CMD_SAVE.

PDS_LM_CMD_HOST_VF_STATUS is added to notify the device when a migration is in/not-in progress from the host's perspective.

---
 drivers/vfio/pci/pds/Makefile   |   1 +
 drivers/vfio/pci/pds/cmds.c     | 316 +++++++++++++++++++++++
 drivers/vfio/pci/pds/cmds.h     |   8 +-
 drivers/vfio/pci/pds/lm.c       | 437 ++++++++++++++++++++++++++++++++
 drivers/vfio/pci/pds/lm.h       |  41 +++
 drivers/vfio/pci/pds/pci_drv.c  |  15 ++
 drivers/vfio/pci/pds/vfio_dev.c | 127 +++++++++-
 drivers/vfio/pci/pds/vfio_dev.h |  11 +
 include/linux/pds/pds_adminq.h  |  12 +
 include/linux/pds/pds_lm.h      | 206 +++++++++++++++
 10 files changed, 1172 insertions(+), 2 deletions(-)
 create mode 100644 drivers/vfio/pci/pds/lm.c
 create mode 100644 drivers/vfio/pci/pds/lm.h
drivers/vfio/pci/pds/Makefile +1 -0
index 87581111fa17..dbaf613d3794 100644
@@ -5,5 +5,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o
 
 pds_vfio-y := \
 	cmds.o		\
+	lm.o		\
 	pci_drv.o	\
 	vfio_dev.o
drivers/vfio/pci/pds/cmds.c +316 -0

331 lines not on this page · raw message

drivers/vfio/pci/pds/cmds.h +7 -1
index 4c592afccf89..3d8a5508c733 100644
@@ -6,5 +6,11 @@
 
 int pds_vfio_register_client_cmd(struct pds_vfio_pci_device *pds_vfio);
 void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio);
-
+int pds_vfio_suspend_device_cmd(struct pds_vfio_pci_device *pds_vfio);
+int pds_vfio_resume_device_cmd(struct pds_vfio_pci_device *pds_vfio);
+int pds_vfio_get_lm_status_cmd(struct pds_vfio_pci_device *pds_vfio, u64 *size);
+int pds_vfio_get_lm_state_cmd(struct pds_vfio_pci_device *pds_vfio);
+int pds_vfio_set_lm_state_cmd(struct pds_vfio_pci_device *pds_vfio);
+void pds_vfio_send_host_vf_lm_status_cmd(struct pds_vfio_pci_device *pds_vfio,
+					 enum pds_lm_host_vf_status vf_status);
 #endif /* _CMDS_H_ */
drivers/vfio/pci/pds/lm.c new file +437 -0

437 lines not on this page · raw message

drivers/vfio/pci/pds/lm.h new file +41 -0

41 lines not on this page · raw message

drivers/vfio/pci/pds/pci_drv.c +15 -0
index dca7d457bd8c..91d9a66d847a 100644
@@ -75,12 +75,27 @@ pds_vfio_pci_table[] = {
 };
 MODULE_DEVICE_TABLE(pci, pds_vfio_pci_table);
 
+static void
+pds_vfio_pci_aer_reset_done(struct pci_dev *pdev)
+{
+	struct pds_vfio_pci_device *pds_vfio = pds_vfio_pci_drvdata(pdev);
+
+	pds_vfio_reset(pds_vfio);
+}
+
+static const struct
+pci_error_handlers pds_vfio_pci_err_handlers = {
+	.reset_done = pds_vfio_pci_aer_reset_done,
+	.error_detected = vfio_pci_core_aer_err_detected,
+};
+
 static struct pci_driver
 pds_vfio_pci_driver = {
 	.name = KBUILD_MODNAME,
 	.id_table = pds_vfio_pci_table,
 	.probe = pds_vfio_pci_probe,
 	.remove = pds_vfio_pci_remove,
+	.err_handler = &pds_vfio_pci_err_handlers,
 	.driver_managed_dma = true,
 };
 
drivers/vfio/pci/pds/vfio_dev.c +126 -1

168 lines not on this page · raw message

drivers/vfio/pci/pds/vfio_dev.h +11 -0
index 92e8ff241ca8..df6208a7140b 100644
@@ -7,12 +7,21 @@
 #include <linux/pci.h>
 #include <linux/vfio_pci_core.h>
 
+#include "lm.h"
+
 struct pdsc;
 
 struct pds_vfio_pci_device {
 	struct vfio_pci_core_device vfio_coredev;
 	struct pdsc *pdsc;
 
+	struct pds_vfio_lm_file *save_file;
+	struct pds_vfio_lm_file *restore_file;
+	struct mutex state_mutex; /* protect migration state */
+	enum vfio_device_mig_state state;
+	spinlock_t reset_lock; /* protect reset_done flow */
+	u8 deferred_reset;
+
 	int vf_id;
 	int pci_id;
 	u16 client_id;
@@ -20,7 +29,9 @@ struct pds_vfio_pci_device {
 
 const struct vfio_device_ops *pds_vfio_ops_info(void);
 struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev);
+void pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio);
 
 struct pci_dev *pds_vfio_to_pci_dev(struct pds_vfio_pci_device *pds_vfio);
+struct device *pds_vfio_to_dev(struct pds_vfio_pci_device *pds_vfio);
 
 #endif /* _VFIO_DEV_H_ */
include/linux/pds/pds_adminq.h +12 -0
index 98a60ce87b92..897b966c52ea 100644
@@ -4,6 +4,8 @@
 #ifndef _PDS_CORE_ADMINQ_H_
 #define _PDS_CORE_ADMINQ_H_
 
+#include "pds_lm.h"
+
 #define PDSC_ADMINQ_MAX_POLL_INTERVAL	256
 
 enum pds_core_adminq_flags {
@@ -600,6 +602,14 @@ union pds_core_adminq_cmd {
 
 	struct pds_core_q_identify_cmd    q_ident;
 	struct pds_core_q_init_cmd        q_init;
+
+	struct pds_lm_suspend_cmd		lm_suspend;
+	struct pds_lm_suspend_status_cmd	lm_suspend_status;
+	struct pds_lm_resume_cmd		lm_resume;
+	struct pds_lm_status_cmd		lm_status;
+	struct pds_lm_save_cmd			lm_save;
+	struct pds_lm_restore_cmd		lm_restore;
+	struct pds_lm_host_vf_status_cmd	lm_host_vf_status;
 };
 
 union pds_core_adminq_comp {
@@ -621,6 +631,8 @@ union pds_core_adminq_comp {
 
 	struct pds_core_q_identify_comp   q_ident;
 	struct pds_core_q_init_comp       q_init;
+
+	struct pds_lm_status_comp		lm_status;
 };
 
 #ifndef __CHECKER__
include/linux/pds/pds_lm.h +206 -0

211 lines not on this page · raw message

Signature · 1 line
2.17.1
#14 lore raw

Re: [PATCH v9 vfio 4/7] vfio/pds: Add VFIO live migration support

Quoted 46 lines · Brett Creeley
drivers/vfio/pci/pds/lm.c new file +46 -0
+static struct pds_vfio_lm_file *
+pds_vfio_get_lm_file(const struct file_operations *fops, int flags, u64 size)
+{
+	struct pds_vfio_lm_file *lm_file = NULL;
+	unsigned long long npages;
+	struct page **pages;
+	int err = 0;
+
+	if (!size)
+		return NULL;
+
+	/* Alloc file structure */
+	lm_file = kzalloc(sizeof(*lm_file), GFP_KERNEL);
+	if (!lm_file)
+		return NULL;
+
+	/* Create file */
+	lm_file->filep = anon_inode_getfile("pds_vfio_lm", fops, lm_file, flags);
+	if (!lm_file->filep)
+		goto err_get_file;
+
+	stream_open(lm_file->filep->f_inode, lm_file->filep);
+	mutex_init(&lm_file->lock);
+
+	lm_file->size = size;
+
+	/* Allocate memory for file pages */
+	npages = DIV_ROUND_UP_ULL(lm_file->size, PAGE_SIZE);
+
+	pages = kcalloc(npages, sizeof(*pages), GFP_KERNEL);
+	if (!pages)
+		goto err_alloc_pages;
+
+	for (unsigned long long i = 0; i < npages; i++) {
+		pages[i] = alloc_page(GFP_KERNEL);
+		if (!pages[i])
+			goto err_alloc_page;
+	}
+
+	lm_file->pages = pages;
+	lm_file->npages = npages;
+	lm_file->alloc_size = npages * PAGE_SIZE;
+
+	/* Create scatterlist of file pages to use for DMA mapping later */
+	err = sg_alloc_table_from_pages(&lm_file->sg_table, pages, npages,
+					0, size, GFP_KERNEL);

drivers/vfio/pci/pds/lm.c line 69

Quoted 10 lines
drivers/vfio/pci/pds/lm.c new file +10 -0
+	if (err)
+		goto err_alloc_sg_table;
+
+	/* prevent file from being released before we are done with it */
+	get_file(lm_file->filep);
+
+	return lm_file;
+
+err_alloc_sg_table:
+err_alloc_page:

What is with these double error out labels? I see it in a few places, that is not the kernel style.

In VFIO we have been trying to label the err outs based on what they free, it is not a 'call from' scheme.

Jason

#15 lore raw

Re: [PATCH v9 vfio 4/7] vfio/pds: Add VFIO live migration support

Quoted 65 lines · Jason Gunthorpe

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 46 lines · Brett Creeley
drivers/vfio/pci/pds/lm.c new file +46 -0
+static struct pds_vfio_lm_file *
+pds_vfio_get_lm_file(const struct file_operations *fops, int flags, u64 size)
+{
+     struct pds_vfio_lm_file *lm_file = NULL;
+     unsigned long long npages;
+     struct page **pages;
+     int err = 0;
+
+     if (!size)
+             return NULL;
+
+     /* Alloc file structure */
+     lm_file = kzalloc(sizeof(*lm_file), GFP_KERNEL);
+     if (!lm_file)
+             return NULL;
+
+     /* Create file */
+     lm_file->filep = anon_inode_getfile("pds_vfio_lm", fops, lm_file, flags);
+     if (!lm_file->filep)
+             goto err_get_file;
+
+     stream_open(lm_file->filep->f_inode, lm_file->filep);
+     mutex_init(&lm_file->lock);
+
+     lm_file->size = size;
+
+     /* Allocate memory for file pages */
+     npages = DIV_ROUND_UP_ULL(lm_file->size, PAGE_SIZE);
+
+     pages = kcalloc(npages, sizeof(*pages), GFP_KERNEL);
+     if (!pages)
+             goto err_alloc_pages;
+
+     for (unsigned long long i = 0; i < npages; i++) {
+             pages[i] = alloc_page(GFP_KERNEL);
+             if (!pages[i])
+                     goto err_alloc_page;
+     }
+
+     lm_file->pages = pages;
+     lm_file->npages = npages;
+     lm_file->alloc_size = npages * PAGE_SIZE;
+
+     /* Create scatterlist of file pages to use for DMA mapping later */
+     err = sg_alloc_table_from_pages(&lm_file->sg_table, pages, npages,
+                                     0, size, GFP_KERNEL);

drivers/vfio/pci/pds/lm.c line 69

Quoted 10 lines
drivers/vfio/pci/pds/lm.c new file +10 -0
+     if (err)
+             goto err_alloc_sg_table;
+
+     /* prevent file from being released before we are done with it */
+     get_file(lm_file->filep);
+
+     return lm_file;
+
+err_alloc_sg_table:
+err_alloc_page:

What is with these double error out labels? I see it in a few places, that is not the kernel style.

Quoted 4 lines

In VFIO we have been trying to label the err outs based on what they free, it is not a 'call from' scheme.

Jason

Yeah this is kind of ugly. Thanks for pointing it out. I will fix it and any other occurrences in the next revision.

Brett

#16 lore raw

Re: [PATCH v9 vfio 4/7] vfio/pds: Add VFIO live migration support

In reply to Brett Creeley

drivers/vfio/pci/pds/lm.c line 16

Quoted 3 lines · Brett Creeley
drivers/vfio/pci/pds/lm.c new file +3 -0
+static struct pds_vfio_lm_file *
+pds_vfio_get_lm_file(const struct file_operations *fops, int flags, u64 size)
+{

I see this function is called with a hardwired 64k size on restore - so this means save is also limited to 64k?

This is really overcomplicated if that is the state size you are working with - the mlx5 code this was copied from is dealing with much larger values.

Just kvalloc() your 64k and be done with it. You need a bit of fussing to DMA map the vmap, but it is much simpler than all of this stuff. See fpga_mgr_buf_load()

Since the kvalloc is linear the read/write is just memcpy.

Jason

#17 lore raw

Re: [PATCH v9 vfio 4/7] vfio/pds: Add VFIO live migration support

Quoted 16 lines · Jason Gunthorpe

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

drivers/vfio/pci/pds/lm.c line 16

Quoted 3 lines · Brett Creeley
drivers/vfio/pci/pds/lm.c new file +3 -0
+static struct pds_vfio_lm_file *
+pds_vfio_get_lm_file(const struct file_operations *fops, int flags, u64 size)
+{

I see this function is called with a hardwired 64k size on restore - so this means save is also limited to 64k?

This is really overcomplicated if that is the state size you are working with - the mlx5 code this was copied from is dealing with much larger values.

Quoted 7 lines

Just kvalloc() your 64k and be done with it. You need a bit of fussing to DMA map the vmap, but it is much simpler than all of this stuff. See fpga_mgr_buf_load()

Since the kvalloc is linear the read/write is just memcpy.

Jason

My only thought here is that if we ever need to support a larger device state size in the future we may have to revisit this implementation once again.

I will look into this before pushing the next revision. Thanks for the example/pointer to fpga_mgr_buf_load() and for the review.

Brett

#18 lore raw

Re: [PATCH v9 vfio 4/7] vfio/pds: Add VFIO live migration support

In reply to Brett Creeley
Quoted 36 lines · Brett Creeley
drivers/vfio/pci/pds/cmds.c +36 -0
+static int
+pds_vfio_dma_map_lm_file(struct device *dev, enum dma_data_direction dir,
+			 struct pds_vfio_lm_file *lm_file)
+{
+	struct pds_lm_sg_elem *sgl, *sge;
+	struct scatterlist *sg;
+	dma_addr_t sgl_addr;
+	size_t sgl_size;
+	int err;
+	int i;
+
+	if (!lm_file)
+		return -EINVAL;
+
+	/* dma map file pages */
+	err = dma_map_sgtable(dev, &lm_file->sg_table, dir, 0);
+	if (err)
+		return err;
+
+	lm_file->num_sge = lm_file->sg_table.nents;
+
+	/* alloc sgl */
+	sgl_size = lm_file->num_sge * sizeof(struct pds_lm_sg_elem);
+	sgl = kzalloc(sgl_size, GFP_KERNEL);
+	if (!sgl) {
+		err = -ENOMEM;
+		goto err_alloc_sgl;
+	}
+
+	sgl_addr = dma_map_single(dev, sgl, sgl_size, DMA_TO_DEVICE);
+	if (dma_mapping_error(dev, sgl_addr)) {
+		err = -EIO;
+		goto err_map_sgl;
+	}
+
+	lm_file->sgl = sgl;

drivers/vfio/pci/pds/cmds.c pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 227

Quoted 10 lines
drivers/vfio/pci/pds/cmds.c +10 -0
+	lm_file->sgl_addr = sgl_addr;
+
+	/* fill sgl */
+	sge = sgl;
+	for_each_sgtable_dma_sg(&lm_file->sg_table, sg, i) {
+		sge->addr = cpu_to_le64(sg_dma_address(sg));
+		sge->len  = cpu_to_le32(sg_dma_len(sg));
+		dev_dbg(dev, "addr = %llx, len = %u\n", sge->addr, sge->len);
+		sge++;
+	}

This sequence is in the wrong order, the dma_map_single() has to be after the data is written to the memory as it synchronizes the caches on some arches

drivers/vfio/pci/pds/cmds.c pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 235

Quoted 8 lines
drivers/vfio/pci/pds/cmds.c +8 -0
+
+	return 0;
+
+err_map_sgl:
+	kfree(sgl);
+err_alloc_sgl:
+	dma_unmap_sgtable(dev, &lm_file->sg_table, dir, 0);
+	return err;

And why is the goto error unwind in a messed up order? Error unwinds should always be strictly in the opposite order of the success path. Check them all when you are fixing the "come from" notation

Jason

#19 lore raw

Re: [PATCH v9 vfio 4/7] vfio/pds: Add VFIO live migration support

Quoted 47 lines · Jason Gunthorpe

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 42 lines · Brett Creeley
drivers/vfio/pci/pds/cmds.c +42 -0
+static int
+pds_vfio_dma_map_lm_file(struct device *dev, enum dma_data_direction dir,
+                      struct pds_vfio_lm_file *lm_file)
+{
+     struct pds_lm_sg_elem *sgl, *sge;
+     struct scatterlist *sg;
+     dma_addr_t sgl_addr;
+     size_t sgl_size;
+     int err;
+     int i;
+
+     if (!lm_file)
+             return -EINVAL;
+
+     /* dma map file pages */
+     err = dma_map_sgtable(dev, &lm_file->sg_table, dir, 0);
+     if (err)
+             return err;
+
+     lm_file->num_sge = lm_file->sg_table.nents;
+
+     /* alloc sgl */
+     sgl_size = lm_file->num_sge * sizeof(struct pds_lm_sg_elem);
+     sgl = kzalloc(sgl_size, GFP_KERNEL);
+     if (!sgl) {
+             err = -ENOMEM;
+             goto err_alloc_sgl;
+     }
+
+     sgl_addr = dma_map_single(dev, sgl, sgl_size, DMA_TO_DEVICE);
+     if (dma_mapping_error(dev, sgl_addr)) {
+             err = -EIO;
+             goto err_map_sgl;
+     }
+
+     lm_file->sgl = sgl;
+     lm_file->sgl_addr = sgl_addr;
+
+     /* fill sgl */
+     sge = sgl;
+     for_each_sgtable_dma_sg(&lm_file->sg_table, sg, i) {
+             sge->addr = cpu_to_le64(sg_dma_address(sg));
Quoted 8 lines

drivers/vfio/pci/pds/cmds.c pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 227

Quoted 4 lines
drivers/vfio/pci/pds/cmds.c +4 -0
+             sge->len  = cpu_to_le32(sg_dma_len(sg));
+             dev_dbg(dev, "addr = %llx, len = %u\n", sge->addr, sge->len);
+             sge++;
+     }

This sequence is in the wrong order, the dma_map_single() has to be after the data is written to the memory as it synchronizes the caches on some arches

This is an easy enough fix and I'll keep this in mind moving forward. Thanks for pointing it out.

Quoted 14 lines

drivers/vfio/pci/pds/cmds.c pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 235

Quoted 9 lines
drivers/vfio/pci/pds/cmds.c +8 -0
+
+     return 0;
+
+err_map_sgl:
+     kfree(sgl);
+err_alloc_sgl:
+     dma_unmap_sgtable(dev, &lm_file->sg_table, dir, 0);
+     return err;

And why is the goto error unwind in a messed up order? Error unwinds should always be strictly in the opposite order of the success path. Check them all when you are fixing the "come from" notation

Quoted 1 line

Jason

Yeah, this wasn't on purpose. Thanks again for pointing things like this out. As you noted I will go through the entire series to make sure the goto labels are named by what they are freeing/clearing and also that they are in the correct opposite order.

Thanks again for the review.

Brett

author maintainer
#20 lore raw

PATCH v9 5/7 vfio/pds: Add support for dirty page tracking

In reply to Brett Creeley

In order to support dirty page tracking, the driver has to implement the VFIO subsystem's vfio_log_ops. This includes log_start, log_stop, and log_read_and_clear.

All of the tracker resources are allocated and dirty tracking on the device is started during log_start. The resources are cleaned up and dirty tracking on the device is stopped during log_stop. The dirty pages are determined and reported during log_read_and_clear.

In order to support these callbacks admin queue commands are used. All of the adminq queue command structures and implementations are included as part of this patch.

PDS_LM_CMD_DIRTY_STATUS is added to query the current status of dirty tracking on the device. This includes if it's enabled (i.e. number of regions being tracked from the device's perspective) and the maximum number of regions supported from the device's perspective.

PDS_LM_CMD_DIRTY_ENABLE is added to enable dirty tracking on the specified number of regions and their iova ranges.

PDS_LM_CMD_DIRTY_DISABLE is added to disable dirty tracking for all regions on the device.

PDS_LM_CMD_READ_SEQ and PDS_LM_CMD_DIRTY_WRITE_ACK are added to support reading and acknowledging the currently dirtied pages.

---
 drivers/vfio/pci/pds/Makefile   |   1 +
 drivers/vfio/pci/pds/cmds.c     | 124 +++++++
 drivers/vfio/pci/pds/cmds.h     |   9 +
 drivers/vfio/pci/pds/dirty.c    | 562 ++++++++++++++++++++++++++++++++
 drivers/vfio/pci/pds/dirty.h    |  45 +++
 drivers/vfio/pci/pds/lm.c       |   3 +-
 drivers/vfio/pci/pds/vfio_dev.c |  12 +-
 drivers/vfio/pci/pds/vfio_dev.h |   5 +
 include/linux/pds/pds_adminq.h  |   5 +
 include/linux/pds/pds_lm.h      | 173 ++++++++++
 10 files changed, 936 insertions(+), 3 deletions(-)
 create mode 100644 drivers/vfio/pci/pds/dirty.c
 create mode 100644 drivers/vfio/pci/pds/dirty.h
drivers/vfio/pci/pds/Makefile +1 -0
index dbaf613d3794..805176f7be9f 100644
@@ -5,6 +5,7 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o
 
 pds_vfio-y := \
 	cmds.o		\
+	dirty.o		\
 	lm.o		\
 	pci_drv.o	\
 	vfio_dev.o
drivers/vfio/pci/pds/cmds.c +124 -0

127 lines not on this page · raw message

drivers/vfio/pci/pds/cmds.h +9 -0
index 3d8a5508c733..fc1f4ae611eb 100644
@@ -13,4 +13,13 @@ int pds_vfio_get_lm_state_cmd(struct pds_vfio_pci_device *pds_vfio);
 int pds_vfio_set_lm_state_cmd(struct pds_vfio_pci_device *pds_vfio);
 void pds_vfio_send_host_vf_lm_status_cmd(struct pds_vfio_pci_device *pds_vfio,
 					 enum pds_lm_host_vf_status vf_status);
+int pds_vfio_dirty_status_cmd(struct pds_vfio_pci_device *pds_vfio,
+			      u64 regions_dma, u8 *max_regions,
+			      u8 *num_regions);
+int pds_vfio_dirty_enable_cmd(struct pds_vfio_pci_device *pds_vfio,
+			      u64 regions_dma, u8 num_regions);
+int pds_vfio_dirty_disable_cmd(struct pds_vfio_pci_device *pds_vfio);
+int pds_vfio_dirty_seq_ack_cmd(struct pds_vfio_pci_device *pds_vfio,
+			       u64 sgl_dma, u16 num_sge, u32 offset,
+			       u32 total_len, bool read_seq);
 #endif /* _CMDS_H_ */
drivers/vfio/pci/pds/dirty.c new file +562 -0

562 lines not on this page · raw message

drivers/vfio/pci/pds/dirty.h new file +45 -0

45 lines not on this page · raw message

drivers/vfio/pci/pds/lm.c +1 -2
index 4e736f415df2..2b8876eb1bfd 100644
@@ -386,8 +386,7 @@ pds_vfio_step_device_state_locked(struct pds_vfio_pci_device *pds_vfio,
 
 	if (cur == VFIO_DEVICE_STATE_STOP_COPY && next == VFIO_DEVICE_STATE_STOP) {
 		pds_vfio_put_save_file(pds_vfio);
-		pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
-						    PDS_LM_STA_NONE);
+		pds_vfio_dirty_disable(pds_vfio);
 		return NULL;
 	}
 
drivers/vfio/pci/pds/vfio_dev.c +11 -1
index 0c6e6c645f34..1adb00ea0114 100644
@@ -5,6 +5,7 @@
 #include <linux/vfio_pci_core.h>
 
 #include "lm.h"
+#include "dirty.h"
 #include "vfio_dev.h"
 
 struct pci_dev *
@@ -28,7 +29,7 @@ pds_vfio_pci_drvdata(struct pci_dev *pdev)
 			    vfio_coredev);
 }
 
-static void
+void
 pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio)
 {
 again:
@@ -125,6 +126,13 @@ pds_vfio_lm_ops = {
 	.migration_get_data_size = pds_vfio_get_device_state_size
 };
 
+static const struct vfio_log_ops
+pds_vfio_log_ops = {
+	.log_start = pds_vfio_dma_logging_start,
+	.log_stop = pds_vfio_dma_logging_stop,
+	.log_read_and_clear = pds_vfio_dma_logging_report,
+};
+
 static int
 pds_vfio_init_device(struct vfio_device *vdev)
 {
@@ -143,6 +151,7 @@ pds_vfio_init_device(struct vfio_device *vdev)
 
 	vdev->migration_flags = VFIO_MIGRATION_STOP_COPY | VFIO_MIGRATION_P2P;
 	vdev->mig_ops = &pds_vfio_lm_ops;
+	vdev->log_ops = &pds_vfio_log_ops;
 
 	dev_dbg(&pdev->dev, "%s: PF %#04x VF %#04x (%d) vf_id %d domain %d pds_vfio %p\n",
 		__func__, pci_dev_id(pdev->physfn),
@@ -182,6 +191,7 @@ pds_vfio_close_device(struct vfio_device *vdev)
 	mutex_lock(&pds_vfio->state_mutex);
 	pds_vfio_put_restore_file(pds_vfio);
 	pds_vfio_put_save_file(pds_vfio);
+	pds_vfio_dirty_disable(pds_vfio);
 	mutex_unlock(&pds_vfio->state_mutex);
 	mutex_destroy(&pds_vfio->state_mutex);
 	vfio_pci_core_close_device(vdev);
drivers/vfio/pci/pds/vfio_dev.h +5 -0
index df6208a7140b..2271f6af2a85 100644
@@ -7,6 +7,7 @@
 #include <linux/pci.h>
 #include <linux/vfio_pci_core.h>
 
+#include "dirty.h"
 #include "lm.h"
 
 struct pdsc;
@@ -17,6 +18,7 @@ struct pds_vfio_pci_device {
 
 	struct pds_vfio_lm_file *save_file;
 	struct pds_vfio_lm_file *restore_file;
+	struct pds_vfio_dirty dirty;
 	struct mutex state_mutex; /* protect migration state */
 	enum vfio_device_mig_state state;
 	spinlock_t reset_lock; /* protect reset_done flow */
@@ -27,6 +29,9 @@ struct pds_vfio_pci_device {
 	u16 client_id;
 };
 
+void
+pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio);
+
 const struct vfio_device_ops *pds_vfio_ops_info(void);
 struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev);
 void pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio);
include/linux/pds/pds_adminq.h +5 -0
index 897b966c52ea..8338722c74d2 100644
@@ -610,6 +610,10 @@ union pds_core_adminq_cmd {
 	struct pds_lm_save_cmd			lm_save;
 	struct pds_lm_restore_cmd		lm_restore;
 	struct pds_lm_host_vf_status_cmd	lm_host_vf_status;
+	struct pds_lm_dirty_status_cmd		lm_dirty_status;
+	struct pds_lm_dirty_enable_cmd		lm_dirty_enable;
+	struct pds_lm_dirty_disable_cmd		lm_dirty_disable;
+	struct pds_lm_dirty_seq_ack_cmd		lm_dirty_seq_ack;
 };
 
 union pds_core_adminq_comp {
@@ -633,6 +637,7 @@ union pds_core_adminq_comp {
 	struct pds_core_q_init_comp       q_init;
 
 	struct pds_lm_status_comp		lm_status;
+	struct pds_lm_dirty_status_comp		lm_dirty_status;
 };
 
 #ifndef __CHECKER__
include/linux/pds/pds_lm.h +173 -0

183 lines not on this page · raw message

Signature · 1 line
2.17.1
#21 lore raw

Re: [PATCH v9 vfio 5/7] vfio/pds: Add support for dirty page tracking

drivers/vfio/pci/pds/cmds.c pds_vfio_send_host_vf_lm_status_cmd(struct pds_vfio_pci_device *pds_vfio, · line 373

Quoted 9 lines · Brett Creeley
drivers/vfio/pci/pds/cmds.c +9 -0
+int
+pds_vfio_dirty_status_cmd(struct pds_vfio_pci_device *pds_vfio,
+			  u64 regions_dma, u8 *max_regions,
+			  u8 *num_regions)
+{
+	union pds_core_adminq_cmd cmd = {
+		.lm_dirty_status.opcode = PDS_LM_CMD_DIRTY_STATUS,
+		.lm_dirty_status.vf_id = cpu_to_le16(pds_vfio->vf_id),
+	};

You can write these as

.lm_dirty_status = {
.opcode = ..,
.vf_if = ..,
},

And save some stuttering

Quoted 5 lines
drivers/vfio/pci/pds/dirty.c new file +5 -0
+static int
+pds_vfio_dirty_alloc_bitmaps(struct pds_vfio_dirty *dirty,
+			     u32 nbits)
+{
+	unsigned long *host_seq_bmp, *host_ack_bmp;

drivers/vfio/pci/pds/dirty.c line 89

Quoted 10 lines
drivers/vfio/pci/pds/dirty.c new file +10 -0
+
+	host_seq_bmp = bitmap_zalloc(nbits, GFP_KERNEL);
+	if (!host_seq_bmp)
+		return -ENOMEM;
+
+	host_ack_bmp = bitmap_zalloc(nbits, GFP_KERNEL);
+	if (!host_ack_bmp) {
+		bitmap_free(host_seq_bmp);
+		return -ENOMEM;
+	}

nbits looks way too big to call bitmap_zalloc?

Quoted 10 lines
drivers/vfio/pci/pds/dirty.c new file +10 -0
+static int
+__pds_vfio_dirty_alloc_sgl(struct pds_vfio_pci_device *pds_vfio,
+			   struct pds_vfio_bmp_info *bmp_info,
+			   u32 page_count)
+{
+	struct pci_dev *pdev = pds_vfio->vfio_coredev.pdev;
+	struct device *pdsc_dev = &pci_physfn(pdev)->dev;
+	struct pds_lm_sg_elem *sgl;
+	dma_addr_t sgl_addr;
+	size_t sgl_size;

drivers/vfio/pci/pds/dirty.c line 156

Quoted 10 lines
drivers/vfio/pci/pds/dirty.c new file +10 -0
+	u32 max_sge;
+
+	max_sge = DIV_ROUND_UP(page_count, PAGE_SIZE * 8);
+	sgl_size = max_sge * sizeof(struct pds_lm_sg_elem);
+
+	sgl = kzalloc(sgl_size, GFP_KERNEL);
+	if (!sgl)
+		return -ENOMEM;
+
+	sgl_addr = dma_map_single(pdsc_dev, sgl, sgl_size, DMA_TO_DEVICE);

This needs to use the streaming API in pds_vfio_dirty_seq_ack()

Jason

author maintainer
#22 lore raw

PATCH v9 6/7 vfio/pds: Add support for firmware recovery

In reply to Brett Creeley

It's possible that the device firmware crashes and is able to recover due to some configuration and/or other issue. If a live migration is in progress while the firmware crashes, the live migration will fail. However, the VF PCI device should still be functional post crash recovery and subsequent migrations should go through as expected.

When the pds_core device notices that firmware crashes it sends an event to all its client drivers. When the pds_vfio driver receives this event while migration is in progress it will request a deferred reset on the next migration state transition. This state transition will report failure as well as any subsequent state transition requests from the VMM/VFIO. Based on uapi/vfio.h the only way out of VFIO_DEVICE_STATE_ERROR is by issuing VFIO_DEVICE_RESET. Once this reset is done, the migration state will be reset to VFIO_DEVICE_STATE_RUNNING and migration can be performed.

If the event is received while no migration is in progress (i.e. the VM is in normal operating mode), then no actions are taken and the migration state remains VFIO_DEVICE_STATE_RUNNING.

---
 drivers/vfio/pci/pds/pci_drv.c  | 108 ++++++++++++++++++++++++++++++++
 drivers/vfio/pci/pds/vfio_dev.c |  29 ++++++++-
 drivers/vfio/pci/pds/vfio_dev.h |   4 ++
 3 files changed, 139 insertions(+), 2 deletions(-)
drivers/vfio/pci/pds/pci_drv.c +108 -0
index 91d9a66d847a..b64c2e492688 100644
@@ -19,6 +19,107 @@
 #define PDS_VFIO_DRV_DESCRIPTION	"AMD/Pensando VFIO Device Driver"
 #define PCI_VENDOR_ID_PENSANDO		0x1dd8
 
+static void
+pds_vfio_recovery(struct pds_vfio_pci_device *pds_vfio)
+{
+	bool deferred_reset_needed = false;
+
+	/*
+	 * Documentation states that the kernel migration driver must not
+	 * generate asynchronous device state transitions outside of
+	 * manipulation by the user or the VFIO_DEVICE_RESET ioctl.
+	 *
+	 * Since recovery is an asynchronous event received from the device,
+	 * initiate a deferred reset. Only issue the deferred reset if a
+	 * migration is in progress, which will cause the next step of the
+	 * migration to fail. Also, if the device is in a state that will
+	 * be set to VFIO_DEVICE_STATE_RUNNING on the next action (i.e. VM is
+	 * shutdown and device is in VFIO_DEVICE_STATE_STOP) as that will clear
+	 * the VFIO_DEVICE_STATE_ERROR when the VM starts back up.
+	 */
+	mutex_lock(&pds_vfio->state_mutex);
+	if ((pds_vfio->state != VFIO_DEVICE_STATE_RUNNING &&
+	     pds_vfio->state != VFIO_DEVICE_STATE_ERROR) ||
+	    (pds_vfio->state == VFIO_DEVICE_STATE_RUNNING &&
+	     pds_vfio_dirty_is_enabled(pds_vfio)))
+		deferred_reset_needed = true;
+	mutex_unlock(&pds_vfio->state_mutex);
+
+	/*
+	 * On the next user initiated state transition, the device will
+	 * transition to the VFIO_DEVICE_STATE_ERROR. At this point it's the user's
+	 * responsibility to reset the device.
+	 *
+	 * If a VFIO_DEVICE_RESET is requested post recovery and before the next
+	 * state transition, then the deferred reset state will be set to
+	 * VFIO_DEVICE_STATE_RUNNING.
+	 */
+	if (deferred_reset_needed)
+		pds_vfio_deferred_reset(pds_vfio, VFIO_DEVICE_STATE_ERROR);
+}
+
+static int
+pds_vfio_pci_notify_handler(struct notifier_block *nb,
+			    unsigned long ecode,
+			    void *data)
+{
+	struct pds_vfio_pci_device *pds_vfio = container_of(nb,
+							    struct pds_vfio_pci_device,
+							    nb);
+	struct device *dev = pds_vfio_to_dev(pds_vfio);
+	union pds_core_notifyq_comp *event = data;
+
+	dev_dbg(dev, "%s: event code %lu\n", __func__, ecode);
+
+	/*
+	 * We don't need to do anything for RESET state==0 as there is no notify
+	 * or feedback mechanism available, and it is possible that we won't
+	 * even see a state==0 event.
+	 *
+	 * Any requests from VFIO while state==0 will fail, which will return
+	 * error and may cause migration to fail.
+	 */
+	if (ecode == PDS_EVENT_RESET) {
+		dev_info(dev, "%s: PDS_EVENT_RESET event received, state==%d\n",
+			 __func__, event->reset.state);
+		if (event->reset.state == 1)
+			pds_vfio_recovery(pds_vfio);
+	}
+
+	return 0;
+}
+
+static int
+pds_vfio_pci_register_event_handler(struct pds_vfio_pci_device *pds_vfio)
+{
+	struct device *dev = pds_vfio_to_dev(pds_vfio);
+	struct notifier_block *nb = &pds_vfio->nb;
+	int err;
+
+	if (!nb->notifier_call) {
+		nb->notifier_call = pds_vfio_pci_notify_handler;
+		err = pdsc_register_notify(nb);
+		if (err) {
+			nb->notifier_call = NULL;
+			dev_err(dev, "failed to register pds event handler: %pe\n",
+				ERR_PTR(err));
+			return -EINVAL;
+		}
+		dev_dbg(dev, "pds event handler registered\n");
+	}
+
+	return 0;
+}
+
+static void
+pds_vfio_pci_unregister_event_handler(struct pds_vfio_pci_device *pds_vfio)
+{
+	if (pds_vfio->nb.notifier_call) {
+		pdsc_unregister_notify(&pds_vfio->nb);
+		pds_vfio->nb.notifier_call = NULL;
+	}
+}
+
 static int
 pds_vfio_pci_probe(struct pci_dev *pdev,
 		   const struct pci_device_id *id)
@@ -49,8 +150,14 @@ pds_vfio_pci_probe(struct pci_dev *pdev,
 		goto out_unregister_coredev;
 	}
 
+	err = pds_vfio_pci_register_event_handler(pds_vfio);
+	if (err)
+		goto out_unregister_client;
+
 	return 0;
 
+out_unregister_client:
+	pds_vfio_unregister_client_cmd(pds_vfio);
 out_unregister_coredev:
 	vfio_pci_core_unregister_device(&pds_vfio->vfio_coredev);
 out_put_vdev:
@@ -63,6 +170,7 @@ pds_vfio_pci_remove(struct pci_dev *pdev)
 {
 	struct pds_vfio_pci_device *pds_vfio = pds_vfio_pci_drvdata(pdev);
 
+	pds_vfio_pci_unregister_event_handler(pds_vfio);
 	pds_vfio_unregister_client_cmd(pds_vfio);
 	vfio_pci_core_unregister_device(&pds_vfio->vfio_coredev);
 	vfio_put_device(&pds_vfio->vfio_coredev.vdev);
drivers/vfio/pci/pds/vfio_dev.c +27 -2
index 1adb00ea0114..0ae9f9250083 100644
@@ -37,10 +37,13 @@ pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio)
 	if (pds_vfio->deferred_reset) {
 		pds_vfio->deferred_reset = false;
 		if (pds_vfio->state == VFIO_DEVICE_STATE_ERROR) {
-			pds_vfio->state = VFIO_DEVICE_STATE_RUNNING;
+			pds_vfio->state = pds_vfio->deferred_reset_state;
 			pds_vfio_put_restore_file(pds_vfio);
 			pds_vfio_put_save_file(pds_vfio);
+		} else if (pds_vfio->deferred_reset_state == VFIO_DEVICE_STATE_ERROR) {
+			pds_vfio->state = VFIO_DEVICE_STATE_ERROR;
 		}
+		pds_vfio->deferred_reset_state = VFIO_DEVICE_STATE_RUNNING;
 		spin_unlock(&pds_vfio->reset_lock);
 		goto again;
 	}
@@ -53,6 +56,7 @@ pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio)
 {
 	spin_lock(&pds_vfio->reset_lock);
 	pds_vfio->deferred_reset = true;
+	pds_vfio->deferred_reset_state = VFIO_DEVICE_STATE_RUNNING;
 	if (!mutex_trylock(&pds_vfio->state_mutex)) {
 		spin_unlock(&pds_vfio->reset_lock);
 		return;
@@ -61,6 +65,16 @@ pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio)
 	pds_vfio_state_mutex_unlock(pds_vfio);
 }
 
+void
+pds_vfio_deferred_reset(struct pds_vfio_pci_device *pds_vfio,
+			enum vfio_device_mig_state reset_state)
+{
+	spin_lock(&pds_vfio->reset_lock);
+	pds_vfio->deferred_reset = true;
+	pds_vfio->deferred_reset_state = reset_state;
+	spin_unlock(&pds_vfio->reset_lock);
+}
+
 static struct file *
 pds_vfio_set_device_state(struct vfio_device *vdev,
 			  enum vfio_device_mig_state new_state)
@@ -71,7 +85,14 @@ pds_vfio_set_device_state(struct vfio_device *vdev,
 	struct file *res = NULL;
 
 	mutex_lock(&pds_vfio->state_mutex);
-	while (new_state != pds_vfio->state) {
+	/*
+	 * only way to transition out of VFIO_DEVICE_STATE_ERROR is via
+	 * VFIO_DEVICE_RESET, so prevent the state machine from running since
+	 * vfio_mig_get_next_state() will throw a WARN_ON() when transitioning
+	 * from VFIO_DEVICE_STATE_ERROR to any other state
+	 */
+	while (pds_vfio->state != VFIO_DEVICE_STATE_ERROR &&
+	       new_state != pds_vfio->state) {
 		enum vfio_device_mig_state next_state;
 
 		int err = vfio_mig_get_next_state(vdev, pds_vfio->state,
@@ -93,6 +114,9 @@ pds_vfio_set_device_state(struct vfio_device *vdev,
 		}
 	}
 	pds_vfio_state_mutex_unlock(pds_vfio);
+	/* still waiting on a deferred_reset */
+	if (pds_vfio->state == VFIO_DEVICE_STATE_ERROR)
+		res = ERR_PTR(-EIO);
 
 	return res;
 }
@@ -175,6 +199,7 @@ pds_vfio_open_device(struct vfio_device *vdev)
 
 	mutex_init(&pds_vfio->state_mutex);
 	pds_vfio->state = VFIO_DEVICE_STATE_RUNNING;
+	pds_vfio->deferred_reset_state = VFIO_DEVICE_STATE_RUNNING;
 
 	vfio_pci_core_finish_enable(&pds_vfio->vfio_coredev);
 
drivers/vfio/pci/pds/vfio_dev.h +4 -0
index 2271f6af2a85..d494aabfce1f 100644
@@ -23,6 +23,8 @@ struct pds_vfio_pci_device {
 	enum vfio_device_mig_state state;
 	spinlock_t reset_lock; /* protect reset_done flow */
 	u8 deferred_reset;
+	enum vfio_device_mig_state deferred_reset_state;
+	struct notifier_block nb;
 
 	int vf_id;
 	int pci_id;
@@ -35,6 +37,8 @@ pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio);
 const struct vfio_device_ops *pds_vfio_ops_info(void);
 struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev);
 void pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio);
+void pds_vfio_deferred_reset(struct pds_vfio_pci_device *pds_vfio,
+			     enum vfio_device_mig_state reset_state);
 
 struct pci_dev *pds_vfio_to_pci_dev(struct pds_vfio_pci_device *pds_vfio);
 struct device *pds_vfio_to_dev(struct pds_vfio_pci_device *pds_vfio);
Signature · 1 line
2.17.1
author maintainer
#23 lore raw

PATCH v9 7/7 vfio/pds: Add Kconfig and documentation

In reply to Brett Creeley

Add Kconfig entries and pds_vfio.rst. Also, add an entry in the MAINTAINERS file for this new driver.

It's not clear where documentation for vendor specific VFIO drivers should live, so just re-use the current amd ethernet location.

---
 .../device_drivers/ethernet/amd/pds_vfio.rst  | 79 +++++++++++++++++++
 .../device_drivers/ethernet/index.rst         |  1 +
 MAINTAINERS                                   |  7 ++
 drivers/vfio/pci/Kconfig                      |  2 +
 drivers/vfio/pci/pds/Kconfig                  | 20 +++++
 5 files changed, 109 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst
 create mode 100644 drivers/vfio/pci/pds/Kconfig
Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst new file +79 -0
new file mode 100644
index 000000000000..7bddde0c7c9d
@@ -0,0 +1,79 @@
+.. SPDX-License-Identifier: GPL-2.0+
+.. note: can be edited and viewed with /usr/bin/formiko-vim
+
+==========================================================
+PCI VFIO driver for the AMD/Pensando(R) DSC adapter family
+==========================================================
+
+AMD/Pensando Linux VFIO PCI Device Driver
+Copyright(c) 2023 Advanced Micro Devices, Inc.
+
+Overview
+========
+
+The ``pds_vfio`` module is a PCI driver that supports Live Migration
+capable Virtual Function (VF) devices in the DSC hardware.
+
+Using the device
+================
+
+The pds_vfio device is enabled via multiple configuration steps and
+depends on the ``pds_core`` driver to create and enable SR-IOV Virtual
+Function devices.
+
+Shown below are the steps to bind the driver to a VF and also to the
+associated auxiliary device created by the ``pds_core`` driver. This
+example assumes the pds_core and pds_vfio modules are already
+loaded.
+
+.. code-block:: bash
+  :name: example-setup-script
+
+  #!/bin/bash
+
+  PF_BUS="0000:60"
+  PF_BDF="0000:60:00.0"
+  VF_BDF="0000:60:00.1"
+
+  # Prevent non-vfio VF driver from probing the VF device
+  echo 0 > /sys/class/pci_bus/$PF_BUS/device/$PF_BDF/sriov_drivers_autoprobe
+
+  # Create single VF for Live Migration via VFIO
+  echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs
+
+  # Allow the VF to be bound to the pds_vfio driver
+  echo "pds_vfio" > /sys/class/pci_bus/$PF_BUS/device/$VF_BDF/driver_override
+
+  # Bind the VF to the pds_vfio driver
+  echo "$VF_BDF" > /sys/bus/pci/drivers/pds_vfio/bind
+
+After performing the steps above, a file in /dev/vfio/<iommu_group>
+should have been created.
+
+
+Enabling the driver
+===================
+
+The driver is enabled via the standard kernel configuration system,
+using the make command::
+
+  make oldconfig/menuconfig/etc.
+
+The driver is located in the menu structure at:
+
+  -> Device Drivers
+    -> VFIO Non-Privileged userspace driver framework
+      -> VFIO support for PDS PCI devices
+
+Support
+=======
+
+For general Linux networking support, please use the netdev mailing
+list, which is monitored by Pensando personnel::
+
+  [email protected]
+
+For more specific support needs, please use the Pensando driver support
+email::
+
+  [email protected]
Documentation/networking/device_drivers/ethernet/index.rst +1 -0
index 417ca514a4d0..0dd88e6f4e7c 100644
@@ -15,6 +15,7 @@ Contents:
    amazon/ena
    altera/altera_tse
    amd/pds_core
+   amd/pds_vfio
    aquantia/atlantic
    chelsio/cxgb
    cirrus/cs89x0
MAINTAINERS +7 -0
index 6ac562e0381e..df9cc59bc858 100644
@@ -21991,6 +21991,13 @@ S:	Maintained
 P:	Documentation/driver-api/vfio-pci-device-specific-driver-acceptance.rst
 F:	drivers/vfio/pci/*/
 
+VFIO PDS PCI DRIVER
+M:	Brett Creeley <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst
+F:	drivers/vfio/pci/pds/
+
 VFIO PLATFORM DRIVER
 M:	Eric Auger <[email protected]>
 L:	[email protected]
drivers/vfio/pci/Kconfig +2 -0
index f9d0c908e738..2c3831dd60ef 100644
@@ -59,4 +59,6 @@ source "drivers/vfio/pci/mlx5/Kconfig"
 
 source "drivers/vfio/pci/hisilicon/Kconfig"
 
+source "drivers/vfio/pci/pds/Kconfig"
+
 endif
drivers/vfio/pci/pds/Kconfig new file +20 -0
new file mode 100644
index 000000000000..149d4986bf43
@@ -0,0 +1,20 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Advanced Micro Devices, Inc.
+
+config PDS_VFIO_PCI
+	tristate "VFIO support for PDS PCI devices"
+	depends on PDS_CORE
+	depends on VFIO_PCI_CORE
+	help
+	  This provides generic PCI support for PDS devices using the VFIO
+	  framework.
+
+	  More specific information on this driver can be
+	  found in
+	  <file:Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst>.
+
+	  To compile this driver as a module, choose M here. The module
+	  will be called pds_vfio.
+
+	  If you don't know what to do here, say N.
+
Signature · 1 line
2.17.1