pds_vfio driver
Subsystems (9)
Brett Creeley
maintainerv10 cover pds_vfio driver
This is a patchset for a new vendor specific VFIO driver (pds_vfio) for use with the AMD/Pensando Distributed Services Card (DSC). This driver makes use of the pds_core driver.
This driver will use the pds_core device's adminq as the VFIO control path to the DSC. In order to make adminq calls, the VFIO instance makes use of functions exported by the pds_core driver.
In order to receive events from pds_core, the pds_vfio driver registers to a private notifier. This is needed for various events that come from the device.
An ASCII diagram of a VFIO instance looks something like this and can be used with the VFIO subsystem to provide the VF device VFIO and live migration support.
.------. .-----------------------.
| QEMU |--| VM .-------------. |
'......' | | Eth VF | |
| | .-------------. |
| | | SR-IOV VF | |
| | '-------------' |
| '------------||---------'
.--------------. ||
|/dev/<vfio_fd>| ||
'--------------' ||
Host Userspace | ||
=================================================== ||
Host Kernel | ||
.--------. ||
|vfio-pci| ||
'--------' ||
.------------------. || ||
| | exported API |<----+ || ||
| '--------------| | || ||
| | .-------------. ||
| pds_core |--->| pds_vfio | ||
'------------------' | '-------------' ||
|| | || ||
09:00.0 notifier 09:00.1 ||
== PCI ===============================================||=====
|| || ||
.----------. .----------. ||
,-----| PF |----------| VF |-------------------,
| '----------' '----------' | VF |
| DSC | data/control |
| | path |
-----------------------------------------------------------The pds_vfio driver is targeted to reside in drivers/vfio/pci/pds. It makes use of and introduces new files in the common include/linux/pds include directory.
Changes:
v10:
- Various fixes/suggestions by Jason Gunthorpe
- Simplify pds_vfio_get_lm_file() based on fpga_mgr_buf_load()
- Clean-ups/fixes based on clang-format
- Remove any double goto labels
- Name goto labels baesed on what needs to be cleaned/freed instead of a "call from" scheme
- Fix any goto unwind ordering issues
- Make sure call dma_map_single() after data is written to memory in pds_vfio_dma_map_lm_file()
- Don't use bitmap_zalloc() for the dirty bitmaps
- Use vzalloc() for dirty bitmaps and refactor how the bitmaps are DMA'd to and from the device in pds_vfio_dirty_seq_ack()
- Remove unnecessary goto in pds_vfio_dirty_disable()
v9:
https://lore.kernel.org/netdev/[email protected]/
- Various fixes/suggestions by Alex Williamson
- Fix how ID is generated in client registration
- Add helper functions to get the VF's struct device and struct pci_dev pointers instead of caching the struct pci dev
- Remove redundant pds_vfio_lm_state() function and remove any places this was being called
- Fix multi-line comments to follow standard convention
- Remove confusing comments in pds_vfio_step_device_state_locked() since the driver's migration states align with the VFIO documentation
- Validate pdsc returned from pdsc_get_pf_struct()
- Various fixes/suggestions by Jason Gunthorpe
- Use struct pdsc instead of void *
- Use {} instead of {0} for structure initialization
- Use unions on the stack instead of casting to the union when sending AQ commands, which required including pds_lm.h in pds_adminq.h
- Replace use of dma_alloc_coherent() when creating the sgl DMA entries for the LM file
- Remove cached struct device *coredev and instead use pci_physfn() to get the pds_core's struct device pointer
- Drop the recovery work item and call pds_vfio_recovery() directly from the notifier callback
- Remove unnecessary #define for "pds_vfio_lm" and just use the string inline to the anon_inode_getfile() argument
- Fix LM file reference counting
- Move initialization of some struct members to when the struct is being initialized for AQ commands
- Make use of GFP_KERNEL_ACCOUNT where it makes sense
- Replace PDS_VFIO_DRV_NAME with KBUILD_MODNAME
- Update to latest pds_core exported functions
- Remove duplicated prototypes for pds_vfio_dma_logging_[start|stop|report] from lm.h
- Hold pds_vfio->state_mutex while starting, stopping, and reporting dirty page tracking in pds_vfio_dma_logging_[start|stop|report]
- Remove duplicate PDS_DEV_TYPE_LM_STR define from pds_lm.h that's already included in pds_common.h
- Replace use of dma_alloc_coherent() when creating the sgl DMA entries for the dirty bitmaps
v8:
https://lore.kernel.org/netdev/[email protected]/
- provide default iommufd callbacks for bind_iommufd, unbind_iommufd, and attach_ioas for the VFIO device as suggested by Shameerali Kolothum Thodi
v7:
https://lore.kernel.org/netdev/[email protected]/
- Disable and clean up dirty page tracking when the VFIO device is closed
- Various improvements suggested by Simon Horman:
- Fix RCT in vfio_combine_iova_ranges()
- Simplify function exit paths by removing unnecessary goto labels
- Cleanup pds_vifo_print_guest_region_info() by adding a goto label for freeing memory, which allowed for reduced indentation on a for loop
- Where possible use C99 style for loops
v6:
https://lore.kernel.org/netdev/[email protected]/
- As suggested by Alex Williamson, use pci_domain_nr() macro to make sure the pds_vfio client's devname is unique
- Remove unnecessary forward declaration and include
- Fix copyright comment to use correct company name
- Remove "." from struct documentation for consistency
v5:
https://lore.kernel.org/netdev/[email protected]/
- Fix SPDX comments in .h files
- Remove adminqcq argument from pdsc_post_adminq() uses
- Unregister client on vfio_pci_core_register_device() failure
- Other minor checkpatch issues
v4:
https://lore.kernel.org/netdev/[email protected]/
- Update cover letter ASCII diagram to reflect new driver architecture
- Remove auxiliary driver implementation
- Use pds_core's exported functions to communicate with the device
- Implement and register notifier for events from the device/pds_core
- Use module_pci_driver() macro since auxiliary driver configuration is no longer needed in __init/__exit
v3:
https://lore.kernel.org/netdev/[email protected]/
- Update copyright year to 2023 and use "Advanced Micro Devices, Inc." for the company name
- Clarify the fact that AMD/Pensando's VFIO solution is device type agnostic, which aligns with other current VFIO solutions
- Add line in drivers/vfio/pci/Makefile to build pds_vfio
- Move documentation to amd sub-directory
- Remove some dead code due to the pds_core implementation of listening to BIND/UNBIND events
- Move a dev_dbg() to a previous patch in the series
- Add implementation for vfio_migration_ops.migration_get_data_size to return the maximum possible device state size
RFC to v2:
https://lore.kernel.org/all/[email protected]/
- Implement state transitions for VFIO_MIGRATION_P2P flag
- Improve auxiliary driver probe by returning EPROBE_DEFER when the PCI driver is not set up correctly
- Add pointer to docs in Documentation/networking/device_drivers/ethernet/index.rst
RFC:
https://lore.kernel.org/all/[email protected]/
Brett Creeley (7):
vfio: Commonize combine_ranges for use in other VFIO drivers vfio/pds: Initial support for pds_vfio VFIO driver vfio/pds: register with the pds_core PF
vfio/pds: Add VFIO live migration support vfio/pds: Add support for dirty page tracking vfio/pds: Add support for firmware recovery vfio/pds: Add Kconfig and documentation
.../device_drivers/ethernet/amd/pds_vfio.rst | 79 +++ .../device_drivers/ethernet/index.rst | 1 + MAINTAINERS | 7 + drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/mlx5/cmd.c | 48 +- drivers/vfio/pci/pds/Kconfig | 20 + drivers/vfio/pci/pds/Makefile | 11 + drivers/vfio/pci/pds/cmds.c | 487 +++++++++++++++ drivers/vfio/pci/pds/cmds.h | 25 + drivers/vfio/pci/pds/dirty.c | 577 ++++++++++++++++++ drivers/vfio/pci/pds/dirty.h | 38 ++ drivers/vfio/pci/pds/lm.c | 421 +++++++++++++ drivers/vfio/pci/pds/lm.h | 41 ++ drivers/vfio/pci/pds/pci_drv.c | 206 +++++++ drivers/vfio/pci/pds/pci_drv.h | 9 + drivers/vfio/pci/pds/vfio_dev.c | 234 +++++++ drivers/vfio/pci/pds/vfio_dev.h | 45 ++ drivers/vfio/vfio_main.c | 47 ++ include/linux/pds/pds_adminq.h | 395 ++++++++++++ include/linux/pds/pds_common.h | 2 + include/linux/vfio.h | 3 + 22 files changed, 2653 insertions(+), 47 deletions(-) create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst create mode 100644 drivers/vfio/pci/pds/Kconfig create mode 100644 drivers/vfio/pci/pds/Makefile create mode 100644 drivers/vfio/pci/pds/cmds.c create mode 100644 drivers/vfio/pci/pds/cmds.h create mode 100644 drivers/vfio/pci/pds/dirty.c create mode 100644 drivers/vfio/pci/pds/dirty.h create mode 100644 drivers/vfio/pci/pds/lm.c create mode 100644 drivers/vfio/pci/pds/lm.h create mode 100644 drivers/vfio/pci/pds/pci_drv.c create mode 100644 drivers/vfio/pci/pds/pci_drv.h create mode 100644 drivers/vfio/pci/pds/vfio_dev.c create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
Signature · 1 line
2.17.1
Brett Creeley
maintainerPATCH v10 1/7 vfio: Commonize combine_ranges for use in other VFIO drivers
Currently only Mellanox uses the combine_ranges function. The new pds_vfio driver also needs this function. So, move it to a common location for other vendor drivers to use.
Also, Simon Harmon noticed that RCT ordering was not followed for vfio_combin_iova_ranges(), so fix that.
--- drivers/vfio/pci/mlx5/cmd.c | 48 +------------------------------------ drivers/vfio/vfio_main.c | 47 ++++++++++++++++++++++++++++++++++++ include/linux/vfio.h | 3 +++ 3 files changed, 51 insertions(+), 47 deletions(-)
drivers/vfio/pci/mlx5/cmd.c +1 -47
@@ -732,52 +732,6 @@ void mlx5fv_cmd_clean_migf_resources(struct mlx5_vf_migration_file *migf)mlx5vf_cmd_dealloc_pd (migf); } -static void combine_ranges (struct rb_root_cached *root,u32 cur_nodes, -u32 req_nodes) -{ -struct interval_tree_node *prev, *curr, *comb_start, *comb_end; -unsigned long min_gap; -unsigned long curr_gap; - -/* Special shortcut when a single range is required */ -if (req_nodes ==1 ) { -unsigned long last; - - curr = comb_start =interval_tree_iter_first (root,0 ,ULONG_MAX ); -while (curr) { - last = curr->last; - prev = curr; - curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); -if (prev != comb_start) -interval_tree_remove (prev, root); - } - comb_start->last = last; -return ; - } - -/* Combine ranges which have the smallest gap */ -while (cur_nodes > req_nodes) { - prev =NULL ; - min_gap =ULONG_MAX ; - curr =interval_tree_iter_first (root,0 ,ULONG_MAX ); -while (curr) { -if (prev) { - curr_gap = curr->start - prev->last; -if (curr_gap < min_gap) { - min_gap = curr_gap; - comb_start = prev; - comb_end = curr; - } - } - prev = curr; - curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); - } - comb_start->last = comb_end->last; -interval_tree_remove (comb_end, root); - cur_nodes--; - } -} -static int mlx5vf_create_tracker (struct mlx5_core_dev *mdev,struct mlx5vf_pci_core_device *mvdev,struct rb_root_cached *ranges,u32 nnodes)
@@ -800,7 +754,7 @@ static int mlx5vf_create_tracker(struct mlx5_core_dev *mdev,int i;if (num_ranges > max_num_range) { -combine_ranges (ranges, nnodes, max_num_range); +vfio_combine_iova_ranges (ranges, nnodes, max_num_range); num_ranges = max_num_range; }
drivers/vfio/vfio_main.c +47 -0
53 lines not on this page · raw message
include/linux/vfio.h +3 -0
@@ -245,6 +245,9 @@ int vfio_mig_get_next_state(struct vfio_device *device,enum vfio_device_mig_state new_fsm, enum vfio_device_mig_state *next_fsm); +void vfio_combine_iova_ranges (struct rb_root_cached *root,u32 cur_nodes, +u32 req_nodes); +/* * External user API */
Signature · 1 line
2.17.1
Tian, Kevin
reviewerRE: [PATCH v10 vfio 1/7] vfio: Commonize combine_ranges for use in other VFIO drivers
Quoted 41 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AM+void vfio_combine_iova_ranges(struct
rb_root_cached*root, u32cur_nodes,drivers/vfio/vfio_main.c +36
-0+ u32 req_nodes) +{ +struct interval_tree_node *prev, *curr, *comb_start, *comb_end; +unsigned long min_gap, curr_gap; + +/* Special shortcut when a single range is required */ +if (req_nodes ==1 ) { +unsigned long last; + + comb_start =interval_tree_iter_first (root,0 ,ULONG_MAX ); + curr = comb_start; +while (curr) { + last = curr->last; + prev = curr; + curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); +if (prev != comb_start) +interval_tree_remove (prev, root); + } + comb_start->last = last; +return ; + } + +/* Combine ranges which have the smallest gap */ +while (cur_nodes > req_nodes) { + prev =NULL ; + min_gap =ULONG_MAX ; + curr =interval_tree_iter_first (root,0 ,ULONG_MAX ); +while (curr) { +if (prev) { + curr_gap = curr->start - prev->last; +if (curr_gap < min_gap) { + min_gap = curr_gap; + comb_start = prev; + comb_end = curr; + } + }
drivers/vfio/vfio_main.c static int vfio_ioctl_device_feature_migration(struct vfio_device *device, · line 914
Quoted 10 lines
drivers/vfio/vfio_main.c +10
-0+ prev = curr; + curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); + } + comb_start->last = comb_end->last; +interval_tree_remove (comb_end, root); + cur_nodes--; + } +} +EXPORT_SYMBOL_GPL (vfio_combine_iova_ranges); +
Being a public function please follow the kernel convention with comment explaining what this function actually does.
btw while you rename it with 'vfio' and 'iova' keywords, the actual logic has nothing to do with either of them. Does it make more sense to move it to a more generic library?
Re: [PATCH v10 vfio 1/7] vfio: Commonize combine_ranges for use in other VFIO drivers
Quoted 49 lines · Kevin
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 46 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AM+void vfio_combine_iova_ranges(struct
rb_root_cached*root, u32cur_nodes,drivers/vfio/vfio_main.c +41
-0+ u32 req_nodes) +{ +struct interval_tree_node *prev, *curr, *comb_start, *comb_end; +unsigned long min_gap, curr_gap; + +/* Special shortcut when a single range is required */ +if (req_nodes ==1 ) { +unsigned long last; + + comb_start =interval_tree_iter_first (root,0 ,ULONG_MAX ); + curr = comb_start; +while (curr) { + last = curr->last; + prev = curr; + curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); +if (prev != comb_start) +interval_tree_remove (prev, root); + } + comb_start->last = last; +return ; + } + +/* Combine ranges which have the smallest gap */ +while (cur_nodes > req_nodes) { + prev =NULL ; + min_gap =ULONG_MAX ; + curr =interval_tree_iter_first (root,0 ,ULONG_MAX ); +while (curr) { +if (prev) { + curr_gap = curr->start - prev->last; +if (curr_gap < min_gap) { + min_gap = curr_gap; + comb_start = prev; + comb_end = curr; + } + } + prev = curr; + curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); + } + comb_start->last = comb_end->last; +interval_tree_remove (comb_end, root);
Quoted 8 lines
drivers/vfio/vfio_main.c static int vfio_ioctl_device_feature_migration(struct vfio_device *device, · line 914
Quoted 5 lines
drivers/vfio/vfio_main.c +5
-0+ cur_nodes--; + } +} +EXPORT_SYMBOL_GPL (vfio_combine_iova_ranges); +Being a public function please follow the kernel convention with comment explaining what this function actually does.
I've seen many cases that there's no documentation for public functions and I don't think any documentation is needed for this function as the name is self explanatory. VFIO drivers can use this to combine iova ranges, hence why I named it vfio_combine_iova_ranges().
Quoted 4 lines
btw while you rename it with 'vfio' and 'iova' keywords, the actual logic has nothing to do with either of them. Does it make more sense to move it to a more generic library?
I think it *could* go into a more generic library, but at this point in time I think it belongs here. As mentioned in the previous comment the function name describes its exact purpose. If/when it ever gets more users it can be moved and renamed.
Brett Creeley
maintainerPATCH v10 2/7 vfio/pds: Initial support for pds_vfio VFIO driver
In reply to Brett CreeleyThis is the initial framework for the new pds_vfio device driver. This does the very basics of registering the PDS PCI device and configuring it as a VFIO PCI device.
With this change, the VF device can be bound to the pds_vfio driver on the host and presented to the VM as the VF's device type.
--- drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/pds/Makefile | 8 ++++ drivers/vfio/pci/pds/pci_drv.c | 69 +++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.c | 72 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.h | 20 +++++++++ 5 files changed, 171 insertions(+) create mode 100644 drivers/vfio/pci/pds/Makefile create mode 100644 drivers/vfio/pci/pds/pci_drv.c create mode 100644 drivers/vfio/pci/pds/vfio_dev.c create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
drivers/vfio/pci/Makefile +2 -0
@@ -11,3 +11,5 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5/ obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/ + +obj-$(CONFIG_PDS_VFIO_PCI) += pds/
drivers/vfio/pci/pds/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Advanced Micro Devices, Inc. + +obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o + +pds_vfio-y := \ + pci_drv.o \ + vfio_dev.o
drivers/vfio/pci/pds/pci_drv.c new file +69 -0
69 lines not on this page · raw message
drivers/vfio/pci/pds/vfio_dev.c new file +72 -0
72 lines not on this page · raw message
drivers/vfio/pci/pds/vfio_dev.h new file +20 -0
@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _VFIO_DEV_H_ +#define _VFIO_DEV_H_ + +#include <linux/pci.h> +#include <linux/vfio_pci_core.h> + +struct pds_vfio_pci_device { +struct vfio_pci_core_device vfio_coredev; + +int vf_id; +int pci_id; +}; + +const struct vfio_device_ops *pds_vfio_ops_info (void ); +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); + +#endif /* _VFIO_DEV_H_ */
Signature · 1 line
2.17.1
Re: [PATCH v10 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver
Quoted 37 lines · On Fri, 2 Jun 2023 15:03:13 -0700 Brett Creeley <[email protected]> wrote:
This is the initial framework for the new
pds_vfiodevice driver. This does the very basics of registering the PDS PCI device and configuring it as a VFIO PCI device.With this change, the VF device can be bound to the
pds_vfiodriver on the host and presented to the VM as the VF's device type.Signed-off-by Brett Creeley <[email protected]>Signed-off-by Shannon Nelson <[email protected]>--- drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/pds/Makefile | 8 ++++ drivers/vfio/pci/pds/pci_drv.c | 69 +++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.c | 72 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.h | 20 +++++++++ 5 files changed, 171 insertions(+) create mode 100644 drivers/vfio/pci/pds/Makefile create mode 100644 drivers/vfio/pci/pds/pci_drv.c create mode 100644 drivers/vfio/pci/pds/vfio_dev.c create mode 100644 drivers/vfio/pci/pds/vfio_dev.hdrivers/vfio/pci/Makefile +2
-0@@ -11,3 +11,5 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5/ obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/ + +obj-$(CONFIG_PDS_VFIO_PCI) += pds/drivers/vfio/pci/pds/Makefile new file +0
-0
drivers/vfio/pci/pds/Makefile line 4
Quoted 4 lines
drivers/vfio/pci/pds/Makefile new file +4
-0@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Advanced Micro Devices, Inc. + +obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o
Given the existing drivers:
obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5-vfio-pci.o obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisi-acc-vfio-pci.o
Does it make sense to name this one pds-vfio-pci?
Quoted 16 lines
drivers/vfio/pci/pds/Makefile new file +4
-0+ +pds_vfio-y := \ + pci_drv.o \ + vfio_dev.odrivers/vfio/pci/pds/pci_drv.c new file +6
-0@@ -0,0 +1,69 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt + +#include <linux/module.h>
drivers/vfio/pci/pds/pci_drv.c line 16
Quoted 10 lines
drivers/vfio/pci/pds/pci_drv.c new file +10
-0+#include <linux/pci.h> +#include <linux/types.h> +#include <linux/vfio.h> + +#include <linux/pds/pds_core_if.h> + +#include "vfio_dev.h" + +#define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO Device Driver" +#define PCI_VENDOR_ID_PENSANDO 0x1dd8
Isn't this a duplicate from the above include:
I also find it defined in ionic.h, which means that it now satisfies pci_ids.h requirement that the identifier is shared between multiple drivers. A trivial follow-up after this series might combine them there.
Quoted 79 lines
drivers/vfio/pci/pds/pci_drv.c new file +53
-0+ +static int pds_vfio_pci_probe (struct pci_dev *pdev, +const struct pci_device_id *id) +{ +struct pds_vfio_pci_device *pds_vfio; +int err; + + pds_vfio =vfio_alloc_device (pds_vfio_pci_device, vfio_coredev.vdev, + &pdev->dev,pds_vfio_ops_info ()); +if (IS_ERR (pds_vfio)) +return PTR_ERR (pds_vfio); + +dev_set_drvdata (&pdev->dev, &pds_vfio->vfio_coredev); + + err =vfio_pci_core_register_device (&pds_vfio->vfio_coredev); +if (err) + goto out_put_vdev; + +return 0 ; + +out_put_vdev: +vfio_put_device (&pds_vfio->vfio_coredev.vdev); +return err; +} + +static void pds_vfio_pci_remove (struct pci_dev *pdev) +{ +struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); + +vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); +vfio_put_device (&pds_vfio->vfio_coredev.vdev); +} + +static const struct pci_device_id +pds_vfio_pci_table[] = { + {PCI_DRIVER_OVERRIDE_DEVICE_VFIO (PCI_VENDOR_ID_PENSANDO ,0x1003 ) },/* Ethernet VF */ + {0 , } +}; +MODULE_DEVICE_TABLE (pci, pds_vfio_pci_table); + +static struct pci_driver pds_vfio_pci_driver = { + .name =KBUILD_MODNAME , + .id_table = pds_vfio_pci_table, + .probe = pds_vfio_pci_probe, + .remove = pds_vfio_pci_remove, + .driver_managed_dma = true, +}; + +module_pci_driver (pds_vfio_pci_driver); + +MODULE_DESCRIPTION (PDS_VFIO_DRV_DESCRIPTION ); +MODULE_AUTHOR ("Advanced Micro Devices, Inc." ); +MODULE_LICENSE ("GPL" );drivers/vfio/pci/pds/vfio_dev.c new file +20
-0@@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/vfio.h> +#include <linux/vfio_pci_core.h> + +#include "vfio_dev.h" + +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *core_device =dev_get_drvdata (&pdev->dev); + + return container_of(core_device, struct pds_vfio_pci_device, + vfio_coredev); +} + +static int pds_vfio_init_device (struct vfio_device *vdev) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device,
drivers/vfio/pci/pds/vfio_dev.c line 30
Quoted 10 lines
drivers/vfio/pci/pds/vfio_dev.c new file +10
-0+ vfio_coredev.vdev); +struct pci_dev *pdev =to_pci_dev (vdev->dev); +int err; + + err =vfio_pci_core_init_dev (vdev); +if (err) +return err; + + pds_vfio->vf_id =pci_iov_vf_id (pdev); + pds_vfio->pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn);
We only ever end up using pci_id for a debug print here that could use a local variable and a slow path client registration that has access to pdev to do a lookup on demand. Why do we bother caching it on the pds_vfio_pci_device? Thanks,
Alex
Quoted 68 lines
drivers/vfio/pci/pds/vfio_dev.c new file +42
-0+ +return 0 ; +} + +static int pds_vfio_open_device (struct vfio_device *vdev) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); +int err; + + err =vfio_pci_core_enable (&pds_vfio->vfio_coredev); +if (err) +return err; + +vfio_pci_core_finish_enable (&pds_vfio->vfio_coredev); + +return 0 ; +} + +static const struct vfio_device_ops pds_vfio_ops = { + .name ="pds-vfio" , + .init = pds_vfio_init_device, + .release = vfio_pci_core_release_dev, + .open_device = pds_vfio_open_device, + .close_device = vfio_pci_core_close_device, + .ioctl = vfio_pci_core_ioctl, + .device_feature = vfio_pci_core_ioctl_feature, + .read = vfio_pci_core_read, + .write = vfio_pci_core_write, + .mmap = vfio_pci_core_mmap, + .request = vfio_pci_core_request, + .match = vfio_pci_core_match, + .bind_iommufd = vfio_iommufd_physical_bind, + .unbind_iommufd = vfio_iommufd_physical_unbind, + .attach_ioas = vfio_iommufd_physical_attach_ioas, +}; + +const struct vfio_device_ops *pds_vfio_ops_info (void ) +{ +return &pds_vfio_ops; +}drivers/vfio/pci/pds/vfio_dev.h new file +20
-0@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _VFIO_DEV_H_ +#define _VFIO_DEV_H_ + +#include <linux/pci.h> +#include <linux/vfio_pci_core.h> + +struct pds_vfio_pci_device { +struct vfio_pci_core_device vfio_coredev; + +int vf_id; +int pci_id; +}; + +const struct vfio_device_ops *pds_vfio_ops_info (void ); +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); + +#endif /* _VFIO_DEV_H_ */
Re: [PATCH v10 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver
Quoted 50 lines · Alex Williamson
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 37 lines · On Fri, 2 Jun 2023 15:03:13 -0700 Brett Creeley <[email protected]> wrote:
This is the initial framework for the new
pds_vfiodevice driver. This does the very basics of registering the PDS PCI device and configuring it as a VFIO PCI device.With this change, the VF device can be bound to the
pds_vfiodriver on the host and presented to the VM as the VF's device type.Signed-off-by Brett Creeley <[email protected]>Signed-off-by Shannon Nelson <[email protected]>--- drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/pds/Makefile | 8 ++++ drivers/vfio/pci/pds/pci_drv.c | 69 +++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.c | 72 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.h | 20 +++++++++ 5 files changed, 171 insertions(+)create mode 100644
drivers/vfio/pci/pds/Makefilecreate mode 100644drivers/vfio/pci/pds/pci_drv.ccreate mode 100644drivers/vfio/pci/pds/vfio_dev.ccreate mode 100644drivers/vfio/pci/pds/vfio_dev.hdrivers/vfio/pci/Makefile +2
-0@@ -11,3 +11,5 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5/ obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/ + +obj-$(CONFIG_PDS_VFIO_PCI) += pds/drivers/vfio/pci/pds/Makefile new file +0
-0drivers/vfio/pci/pds/Makefile line 4
Quoted 4 lines
drivers/vfio/pci/pds/Makefile new file +4
-0@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Advanced Micro Devices, Inc. + +obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.oGiven the existing drivers:
Quoted 4 lines
obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5-vfio-pci.o obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisi-acc-vfio-pci.o
Does it make sense to name this one pds-vfio-pci?
Yeah I think it does make more sense to align. Thanks.
Quoted 30 lines
Quoted 17 lines
drivers/vfio/pci/pds/Makefile new file +4
-0+ +pds_vfio-y := \ + pci_drv.o \ + vfio_dev.odrivers/vfio/pci/pds/pci_drv.c new file +6
-0@@ -0,0 +1,69 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt + +#include <linux/module.h> drivers/vfio/pci/pds/pci_drv.c line 16
Quoted 10 lines
drivers/vfio/pci/pds/pci_drv.c new file +10
-0+#include <linux/pci.h> +#include <linux/types.h> +#include <linux/vfio.h> + +#include <linux/pds/pds_core_if.h> + +#include "vfio_dev.h" + +#define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO Device Driver" +#define PCI_VENDOR_ID_PENSANDO 0x1dd8Isn't this a duplicate from the above include:
Quoted 6 lines
I also find it defined in
ionic.h, which means that it now satisfiespci_ids.hrequirement that the identifier is shared between multiple drivers. A trivial follow-up after this series might combine them there.
Good suggestion. Once this series is merged we will submit the follow up patch. Thanks.
Quoted 96 lines
Quoted 80 lines
drivers/vfio/pci/pds/pci_drv.c new file +53
-0+ +static int pds_vfio_pci_probe (struct pci_dev *pdev, +const struct pci_device_id *id) +{ +struct pds_vfio_pci_device *pds_vfio; +int err; + + pds_vfio =vfio_alloc_device (pds_vfio_pci_device, vfio_coredev.vdev, + &pdev->dev,pds_vfio_ops_info ()); +if (IS_ERR (pds_vfio)) +return PTR_ERR (pds_vfio); + +dev_set_drvdata (&pdev->dev, &pds_vfio->vfio_coredev); + + err =vfio_pci_core_register_device (&pds_vfio->vfio_coredev); +if (err) + goto out_put_vdev; + +return 0 ; + +out_put_vdev: +vfio_put_device (&pds_vfio->vfio_coredev.vdev); +return err; +} + +static void pds_vfio_pci_remove (struct pci_dev *pdev) +{ +struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); + +vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); +vfio_put_device (&pds_vfio->vfio_coredev.vdev); +} + +static const struct pci_device_id +pds_vfio_pci_table[] = { + {PCI_DRIVER_OVERRIDE_DEVICE_VFIO (PCI_VENDOR_ID_PENSANDO ,0x1003 ) },/* Ethernet VF */ + {0 , } +}; +MODULE_DEVICE_TABLE (pci, pds_vfio_pci_table); + +static struct pci_driver pds_vfio_pci_driver = { + .name =KBUILD_MODNAME , + .id_table = pds_vfio_pci_table, + .probe = pds_vfio_pci_probe, + .remove = pds_vfio_pci_remove, + .driver_managed_dma = true, +}; + +module_pci_driver (pds_vfio_pci_driver); + +MODULE_DESCRIPTION (PDS_VFIO_DRV_DESCRIPTION ); +MODULE_AUTHOR ("Advanced Micro Devices, Inc." ); +MODULE_LICENSE ("GPL" );drivers/vfio/pci/pds/vfio_dev.c new file +20
-0@@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/vfio.h> +#include <linux/vfio_pci_core.h> + +#include "vfio_dev.h" + +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) +{ +struct vfio_pci_core_device *core_device =dev_get_drvdata (&pdev->dev); + + return container_of(core_device, struct pds_vfio_pci_device, + vfio_coredev); +} + +static int pds_vfio_init_device (struct vfio_device *vdev) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device,drivers/vfio/pci/pds/vfio_dev.c line 30
Quoted 10 lines
drivers/vfio/pci/pds/vfio_dev.c new file +10
-0+ vfio_coredev.vdev); +struct pci_dev *pdev =to_pci_dev (vdev->dev); +int err; + + err =vfio_pci_core_init_dev (vdev); +if (err) +return err; + + pds_vfio->vf_id =pci_iov_vf_id (pdev); + pds_vfio->pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn);We only ever end up using
pci_idfor a debug print here that could use a local variable and a slow path client registration that has access to pdev to do a lookup on demand. Why do we bother caching it on thepds_vfio_pci_device? Thanks,
Quoted 1 line
Alex
No good reason, so another good suggestion. I will fix this as well. Thanks again for the feedback.
Quoted 70 lines
Quoted 69 lines
drivers/vfio/pci/pds/vfio_dev.c new file +42
-0+ +return 0 ; +} + +static int pds_vfio_open_device (struct vfio_device *vdev) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); +int err; + + err =vfio_pci_core_enable (&pds_vfio->vfio_coredev); +if (err) +return err; + +vfio_pci_core_finish_enable (&pds_vfio->vfio_coredev); + +return 0 ; +} + +static const struct vfio_device_ops pds_vfio_ops = { + .name ="pds-vfio" , + .init = pds_vfio_init_device, + .release = vfio_pci_core_release_dev, + .open_device = pds_vfio_open_device, + .close_device = vfio_pci_core_close_device, + .ioctl = vfio_pci_core_ioctl, + .device_feature = vfio_pci_core_ioctl_feature, + .read = vfio_pci_core_read, + .write = vfio_pci_core_write, + .mmap = vfio_pci_core_mmap, + .request = vfio_pci_core_request, + .match = vfio_pci_core_match, + .bind_iommufd = vfio_iommufd_physical_bind, + .unbind_iommufd = vfio_iommufd_physical_unbind, + .attach_ioas = vfio_iommufd_physical_attach_ioas, +}; + +const struct vfio_device_ops *pds_vfio_ops_info (void ) +{ +return &pds_vfio_ops; +}drivers/vfio/pci/pds/vfio_dev.h new file +20
-0@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _VFIO_DEV_H_ +#define _VFIO_DEV_H_ + +#include <linux/pci.h> +#include <linux/vfio_pci_core.h> + +struct pds_vfio_pci_device { +struct vfio_pci_core_device vfio_coredev; + +int vf_id; +int pci_id; +}; + +const struct vfio_device_ops *pds_vfio_ops_info (void ); +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); + +#endif /* _VFIO_DEV_H_ */
Tian, Kevin
reviewerRE: [PATCH v10 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver
In reply to Brett CreeleyQuoted 9 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMThis is the initial framework for the new
pds_vfiodevice driver. This does the very basics of registering the PDS PCI device and configuring it as a VFIO PCI device.With this change, the VF device can be bound to the
pds_vfiodriver on the host and presented to the VM as the VF's device type.
while this should be generic to multiple PDS device types this patch only supports the ethernet VF. worth a clarification here.
drivers/vfio/pci/pds/pci_drv.c line 51
Quoted 2 lines
drivers/vfio/pci/pds/pci_drv.c new file +2
-0+static const struct pci_device_id +pds_vfio_pci_table[] = {
no need to break line.
drivers/vfio/pci/pds/pci_drv.c line 68
Quoted 3 lines
drivers/vfio/pci/pds/pci_drv.c new file +3
-0+ +MODULE_DESCRIPTION (PDS_VFIO_DRV_DESCRIPTION ); +MODULE_AUTHOR ("Advanced Micro Devices, Inc." );
author usually describes the personal name plus mail address.
drivers/vfio/pci/pds/vfio_dev.c line 29
Quoted 6 lines
drivers/vfio/pci/pds/vfio_dev.c new file +6
-0+ + err =vfio_pci_core_init_dev (vdev); +if (err) +return err; + + pds_vfio->vf_id =pci_iov_vf_id (pdev);
pci_iov_vf_id() could fail.
Re: [PATCH v10 vfio 2/7] vfio/pds: Initial support for pds_vfio VFIO driver
Quoted 13 lines · Kevin
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 9 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMThis is the initial framework for the new
pds_vfiodevice driver. This does the very basics of registering the PDS PCI device and configuring it as a VFIO PCI device.With this change, the VF device can be bound to the
pds_vfiodriver on the host and presented to the VM as the VF's device type.
Quoted 7 lines
while this should be generic to multiple PDS device types this patch only supports the ethernet VF. worth a clarification here.
drivers/vfio/pci/pds/pci_drv.c line 51
Quoted 2 lines
drivers/vfio/pci/pds/pci_drv.c new file +2
-0+static const struct pci_device_id +pds_vfio_pci_table[] = {no need to break line.
Must have missed this one. Thanks.
Quoted 6 lines
drivers/vfio/pci/pds/pci_drv.c line 68
Quoted 4 lines
drivers/vfio/pci/pds/pci_drv.c new file +3
-0+ +MODULE_DESCRIPTION (PDS_VFIO_DRV_DESCRIPTION ); +MODULE_AUTHOR ("Advanced Micro Devices, Inc." );author usually describes the personal name plus mail address.
Will fix. Thanks.
Quoted 9 lines
drivers/vfio/pci/pds/vfio_dev.c line 29
Quoted 7 lines
drivers/vfio/pci/pds/vfio_dev.c new file +6
-0+ + err =vfio_pci_core_init_dev (vdev); +if (err) +return err; + + pds_vfio->vf_id =pci_iov_vf_id (pdev);
pci_iov_vf_id()could fail.
Good catch. I will for failure on the next revision. Thanks.
Quoted 1 line
Brett Creeley
maintainerPATCH v10 3/7 vfio/pds: register with the pds_core PF
In reply to Brett CreeleyThe pds_core driver will supply adminq services, so find the PF and register with the DSC services.
Use the following commands to enable a VF:
echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs
--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 43 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 10 ++++++++ drivers/vfio/pci/pds/pci_drv.c | 19 +++++++++++++++ drivers/vfio/pci/pds/pci_drv.h | 9 +++++++ drivers/vfio/pci/pds/vfio_dev.c | 11 +++++++++ drivers/vfio/pci/pds/vfio_dev.h | 6 +++++ include/linux/pds/pds_common.h | 2 ++ 8 files changed, 101 insertions(+) create mode 100644 drivers/vfio/pci/pds/cmds.c create mode 100644 drivers/vfio/pci/pds/cmds.h create mode 100644 drivers/vfio/pci/pds/pci_drv.h
drivers/vfio/pci/pds/Makefile +1 -0
@@ -4,5 +4,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o pds_vfio-y := \ + cmds.o \ pci_drv.o \ vfio_dev.o
drivers/vfio/pci/pds/cmds.c new file +43 -0
43 lines not on this page · raw message
drivers/vfio/pci/pds/cmds.h new file +10 -0
@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _CMDS_H_ +#define _CMDS_H_ + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); + +#endif /* _CMDS_H_ */
drivers/vfio/pci/pds/pci_drv.c +19 -0
@@ -8,9 +8,13 @@#include <linux/types.h> #include <linux/vfio.h> +#include <linux/pds/pds_common.h> #include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> #include "vfio_dev.h" +#include "pci_drv.h" +#include "cmds.h" #define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO Device Driver"#define PCI_VENDOR_ID_PENSANDO 0x1dd8
@@ -27,13 +31,27 @@ static int pds_vfio_pci_probe(struct pci_dev *pdev,return PTR_ERR (pds_vfio);dev_set_drvdata (&pdev->dev, &pds_vfio->vfio_coredev); + pds_vfio->pdsc =pdsc_get_pf_struct (pdev); +if (IS_ERR_OR_NULL (pds_vfio->pdsc)) { + err =PTR_ERR (pds_vfio->pdsc) ?: -ENODEV ; + goto out_put_vdev; + } err =vfio_pci_core_register_device (&pds_vfio->vfio_coredev);if (err) goto out_put_vdev; + err =pds_vfio_register_client_cmd (pds_vfio); +if (err) { +dev_err (&pdev->dev,"failed to register as client: %pe\n" , +ERR_PTR (err)); + goto out_unregister_coredev; + } +return 0 ; +out_unregister_coredev: +vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); out_put_vdev:vfio_put_device (&pds_vfio->vfio_coredev.vdev);return err;
@@ -43,6 +61,7 @@ static void pds_vfio_pci_remove(struct pci_dev *pdev) {struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); +pds_vfio_unregister_client_cmd (pds_vfio);vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev);vfio_put_device (&pds_vfio->vfio_coredev.vdev); }
drivers/vfio/pci/pds/pci_drv.h new file +9 -0
@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _PCI_DRV_H +#define _PCI_DRV_H + +#include <linux/pci.h> + +#endif /* _PCI_DRV_H */
drivers/vfio/pci/pds/vfio_dev.c +11 -0
@@ -6,6 +6,11 @@#include "vfio_dev.h" +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return pds_vfio->vfio_coredev.pdev; +} +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) {struct vfio_pci_core_device *core_device =dev_get_drvdata (&pdev->dev);
@@ -29,6 +34,12 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id =pci_iov_vf_id (pdev); pds_vfio->pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); +dev_dbg (&pdev->dev, +"%s: PF %#04x VF %#04x (%d) vf_id %d domain %d pds_vfio %p\n" , + __func__,pci_dev_id (pdev->physfn), pds_vfio->pci_id, + pds_vfio->pci_id, pds_vfio->vf_id,pci_domain_nr (pdev->bus), + pds_vfio); +return 0 ; }
drivers/vfio/pci/pds/vfio_dev.h +6 -0
@@ -7,14 +7,20 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +struct pdsc ; +struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev; +struct pdsc *pdsc;int vf_id;int pci_id; +u16 client_id; };const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +#endif /* _VFIO_DEV_H_ */
include/linux/pds/pds_common.h +2 -0
@@ -39,6 +39,8 @@ enum pds_core_vif_types {#define PDS_DEV_TYPE_RDMA_STR "RDMA"#define PDS_DEV_TYPE_LM_STR "LM" +#define PDS_LM_DEV_NAME PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_LM_STR +#define PDS_CORE_IFNAMSIZ 16/**
Signature · 1 line
2.17.1
RE: [PATCH v10 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 67 lines
-----Original Message-----
From: Brett Creeley [mailto:[email protected]]
Sent: 02 June 2023 23:03
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Shameerali Kolothum Thodi <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: [PATCH v10 vfio 3/7] vfio/pds: register with thepds_corePFThe
pds_coredriver will supply adminq services, so find the PF and register with the DSC services.Use the following commands to enable a VF:
echo 1 >/sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfsSigned-off-by Brett Creeley <[email protected]>Signed-off-by Shannon Nelson <[email protected]>--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 43 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 10 ++++++++ drivers/vfio/pci/pds/pci_drv.c | 19 +++++++++++++++ drivers/vfio/pci/pds/pci_drv.h | 9 +++++++ drivers/vfio/pci/pds/vfio_dev.c | 11 +++++++++ drivers/vfio/pci/pds/vfio_dev.h | 6 +++++ include/linux/pds/pds_common.h | 2 ++ 8 files changed, 101 insertions(+) create mode 100644 drivers/vfio/pci/pds/cmds.c create mode 100644 drivers/vfio/pci/pds/cmds.h create mode 100644 drivers/vfio/pci/pds/pci_drv.hdrivers/vfio/pci/pds/Makefile +1
-0@@ -4,5 +4,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o pds_vfio-y := \ + cmds.o \ pci_drv.o \ vfio_dev.odrivers/vfio/pci/pds/cmds.c new file +15
-0@@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/io.h> +#include <linux/types.h> + +#include <linux/pds/pds_common.h> +#include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> + +#include "vfio_dev.h" +#include "cmds.h" + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{
drivers/vfio/pci/pds/cmds.c line 25
Quoted 10 lines
drivers/vfio/pci/pds/cmds.c new file +10
-0+struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +char devname[PDS_DEVNAME_LEN ]; +int ci; + +snprintf (devname,sizeof (devname),"%s.%d-%u" ,PDS_LM_DEV_NAME , +pci_domain_nr (pdev->bus), pds_vfio->pci_id); + + ci =pds_client_register (pci_physfn (pdev), devname); +if (ci <=0 ) +return ci;
So 0 is not a valid id I guess but we return 0 here. But below where pds_vfio_register_client_cmd() is called, 0 return is treated as success.
Note: Also in drivers..../auxbus.c the comment says the function returns 0 on success!.
Please check.
Thanks,
Shameer
Quoted 178 lines
drivers/vfio/pci/pds/cmds.c new file +18
-0+ + pds_vfio->client_id = ci; + +return 0 ; +} + +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +int err; + + err =pds_client_unregister (pci_physfn (pdev), pds_vfio->client_id); +if (err) +dev_err (&pdev->dev,"unregister from DSC failed: %pe\n" , +ERR_PTR (err)); + + pds_vfio->client_id =0 ; +}drivers/vfio/pci/pds/cmds.h new file +10
-0@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _CMDS_H_ +#define _CMDS_H_ + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); + +#endif /* _CMDS_H_ */ drivers/vfio/pci/pds/pci_drv.c +4
-0@@ -8,9 +8,13 @@#include <linux/types.h> #include <linux/vfio.h> +#include <linux/pds/pds_common.h> #include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> #include "vfio_dev.h" +#include "pci_drv.h" +#include "cmds.h" #define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO DeviceDriver"
#definePCI_VENDOR_ID_PENSANDO0x1dd8drivers/vfio/pci/pds/pci_drv.c +15
-0@@ -27,13 +31,27 @@ static int pds_vfio_pci_probe(struct pci_dev *pdev,return PTR_ERR (pds_vfio);dev_set_drvdata (&pdev->dev, &pds_vfio->vfio_coredev); + pds_vfio->pdsc =pdsc_get_pf_struct (pdev); +if (IS_ERR_OR_NULL (pds_vfio->pdsc)) { + err =PTR_ERR (pds_vfio->pdsc) ?: -ENODEV ; + goto out_put_vdev; + } err =vfio_pci_core_register_device (&pds_vfio->vfio_coredev);if (err) goto out_put_vdev; + err =pds_vfio_register_client_cmd (pds_vfio); +if (err) { +dev_err (&pdev->dev,"failed to register as client: %pe\n" , +ERR_PTR (err)); + goto out_unregister_coredev; + } +return 0 ; +out_unregister_coredev: +vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); out_put_vdev:vfio_put_device (&pds_vfio->vfio_coredev.vdev);return err;@@ -43,6 +61,7 @@ static void pds_vfio_pci_remove(struct pci_dev *pdev) {struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); +pds_vfio_unregister_client_cmd (pds_vfio);vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev);vfio_put_device (&pds_vfio->vfio_coredev.vdev); }drivers/vfio/pci/pds/pci_drv.h new file +9
-0@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _PCI_DRV_H +#define _PCI_DRV_H + +#include <linux/pci.h> + +#endif /* _PCI_DRV_H */ drivers/vfio/pci/pds/vfio_dev.c +5
-0@@ -6,6 +6,11 @@#include "vfio_dev.h" +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return pds_vfio->vfio_coredev.pdev; +} +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) { struct vfio_pci_core_device *core_device =
dev_get_drvdata(&pdev->dev);drivers/vfio/pci/pds/vfio_dev.c +6
-0@@ -29,6 +34,12 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id =pci_iov_vf_id (pdev); pds_vfio->pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); +dev_dbg (&pdev->dev, + "%s: PF %#04x VF %#04x (%d) vf_id %d domain %d pds_vfio %p\n", + __func__, pci_dev_id(pdev->physfn), pds_vfio->pci_id, + pds_vfio->pci_id, pds_vfio->vf_id,pci_domain_nr (pdev->bus), + pds_vfio); +return 0 ;}
drivers/vfio/pci/pds/vfio_dev.h +6
-0@@ -7,14 +7,20 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +struct pdsc ; +struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev; +struct pdsc *pdsc;int vf_id;int pci_id; +u16 client_id; };const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +#endif /* _VFIO_DEV_H_ */ include/linux/pds/pds_common.h +1
-0@@ -39,6 +39,8 @@ enum pds_core_vif_types {#define PDS_DEV_TYPE_RDMA_STR "RDMA"#define PDS_DEV_TYPE_LM_STR "LM" +#define PDS_LM_DEV_NAME PDS_CORE_DRV_NAME "."
PDS_DEV_TYPE_LM_STR
+
#definePDS_CORE_IFNAMSIZ16/**
--
2.17.1
Re: [PATCH v10 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 82 lines · Shameerali Kolothum Thodi
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 65 lines
-----Original Message-----
From: Brett Creeley [mailto:[email protected]]
Sent: 02 June 2023 23:03
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Shameerali Kolothum Thodi <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: [PATCH v10 vfio 3/7] vfio/pds: register with thepds_corePFThe
pds_coredriver will supply adminq services, so find the PF and register with the DSC services.Use the following commands to enable a VF:
echo 1 >/sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfsSigned-off-by Brett Creeley <[email protected]>Signed-off-by Shannon Nelson <[email protected]>--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 43 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 10 ++++++++ drivers/vfio/pci/pds/pci_drv.c | 19 +++++++++++++++ drivers/vfio/pci/pds/pci_drv.h | 9 +++++++ drivers/vfio/pci/pds/vfio_dev.c | 11 +++++++++ drivers/vfio/pci/pds/vfio_dev.h | 6 +++++ include/linux/pds/pds_common.h | 2 ++ 8 files changed, 101 insertions(+)create mode 100644
drivers/vfio/pci/pds/cmds.ccreate mode 100644drivers/vfio/pci/pds/cmds.hcreate mode 100644drivers/vfio/pci/pds/pci_drv.hdrivers/vfio/pci/pds/Makefile +1
-0@@ -4,5 +4,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o pds_vfio-y := \ + cmds.o \ pci_drv.o \ vfio_dev.odrivers/vfio/pci/pds/cmds.c new file +15
-0@@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/io.h> +#include <linux/types.h> + +#include <linux/pds/pds_common.h> +#include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> + +#include "vfio_dev.h" +#include "cmds.h" + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{drivers/vfio/pci/pds/cmds.c line 25
Quoted 10 lines
drivers/vfio/pci/pds/cmds.c new file +10
-0+struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +char devname[PDS_DEVNAME_LEN ]; +int ci; + +snprintf (devname,sizeof (devname),"%s.%d-%u" ,PDS_LM_DEV_NAME , +pci_domain_nr (pdev->bus), pds_vfio->pci_id); + + ci =pds_client_register (pci_physfn (pdev), devname); +if (ci <=0 ) +return ci;So 0 is not a valid id I guess but we return 0 here. But below where
pds_vfio_register_client_cmd()is called, 0 return is treated as success.
Quoted 7 lines
Note: Also in
drivers..../auxbus.cthe comment says the function returns 0 on success!.Please check.
Thanks,
Shameer
Hey Shameer,
Thanks for catching these issues. It looks like there are a couple things that need to be fixed.
[1] pds_vfio_register_client_cmd() needs to always return negative on error, which includes ci == 0. I don't think we would ever hit this case because drivers..../auxbus.c returns -EIO when ci == 0, but best to fix it in case that ever changes.
[2] Documentation for pds_client_register in drivers..../auxbus.c needs to be updated to say something like the following:
Return: Client ID on succes, or negative for error
I will fix [1] in the next rev of this series. For [2] we will submit a separate follow on patch to clean up the wording.
Thanks for the review,
Brett
Quoted 179 lines
Quoted 178 lines
drivers/vfio/pci/pds/cmds.c new file +18
-0+ + pds_vfio->client_id = ci; + +return 0 ; +} + +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +int err; + + err =pds_client_unregister (pci_physfn (pdev), pds_vfio->client_id); +if (err) +dev_err (&pdev->dev,"unregister from DSC failed: %pe\n" , +ERR_PTR (err)); + + pds_vfio->client_id =0 ; +}drivers/vfio/pci/pds/cmds.h new file +10
-0@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _CMDS_H_ +#define _CMDS_H_ + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); + +#endif /* _CMDS_H_ */ drivers/vfio/pci/pds/pci_drv.c +4
-0@@ -8,9 +8,13 @@#include <linux/types.h> #include <linux/vfio.h> +#include <linux/pds/pds_common.h> #include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> #include "vfio_dev.h" +#include "pci_drv.h" +#include "cmds.h" #define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO DeviceDriver"
#definePCI_VENDOR_ID_PENSANDO0x1dd8drivers/vfio/pci/pds/pci_drv.c +15
-0@@ -27,13 +31,27 @@ static int pds_vfio_pci_probe(struct pci_dev *pdev,return PTR_ERR (pds_vfio);dev_set_drvdata (&pdev->dev, &pds_vfio->vfio_coredev); + pds_vfio->pdsc =pdsc_get_pf_struct (pdev); +if (IS_ERR_OR_NULL (pds_vfio->pdsc)) { + err =PTR_ERR (pds_vfio->pdsc) ?: -ENODEV ; + goto out_put_vdev; + } err =vfio_pci_core_register_device (&pds_vfio->vfio_coredev);if (err) goto out_put_vdev; + err =pds_vfio_register_client_cmd (pds_vfio); +if (err) { +dev_err (&pdev->dev,"failed to register as client: %pe\n" , +ERR_PTR (err)); + goto out_unregister_coredev; + } +return 0 ; +out_unregister_coredev: +vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); out_put_vdev:vfio_put_device (&pds_vfio->vfio_coredev.vdev);return err;@@ -43,6 +61,7 @@ static void pds_vfio_pci_remove(struct pci_dev *pdev) {struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); +pds_vfio_unregister_client_cmd (pds_vfio);vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev);vfio_put_device (&pds_vfio->vfio_coredev.vdev); }drivers/vfio/pci/pds/pci_drv.h new file +9
-0@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _PCI_DRV_H +#define _PCI_DRV_H + +#include <linux/pci.h> + +#endif /* _PCI_DRV_H */ drivers/vfio/pci/pds/vfio_dev.c +5
-0@@ -6,6 +6,11 @@#include "vfio_dev.h" +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return pds_vfio->vfio_coredev.pdev; +} +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) { struct vfio_pci_core_device *core_device =
dev_get_drvdata(&pdev->dev);drivers/vfio/pci/pds/vfio_dev.c +6
-0@@ -29,6 +34,12 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id =pci_iov_vf_id (pdev); pds_vfio->pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); +dev_dbg (&pdev->dev, + "%s: PF %#04x VF %#04x (%d) vf_id %d domain %d pds_vfio %p\n", + __func__, pci_dev_id(pdev->physfn), pds_vfio->pci_id, + pds_vfio->pci_id, pds_vfio->vf_id,pci_domain_nr (pdev->bus), + pds_vfio); +return 0 ;}drivers/vfio/pci/pds/vfio_dev.h +6
-0@@ -7,14 +7,20 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +struct pdsc ; +struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev; +struct pdsc *pdsc;int vf_id;int pci_id; +u16 client_id; };const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +#endif /* _VFIO_DEV_H_ */ include/linux/pds/pds_common.h +1
-0@@ -39,6 +39,8 @@ enum pds_core_vif_types {#define PDS_DEV_TYPE_RDMA_STR "RDMA"#define PDS_DEV_TYPE_LM_STR "LM" +#define PDS_LM_DEV_NAME PDS_CORE_DRV_NAME "."
PDS_DEV_TYPE_LM_STR
+
#definePDS_CORE_IFNAMSIZ16/**
--
2.17.1
Tian, Kevin
reviewerRE: [PATCH v10 vfio 3/7] vfio/pds: register with the pds_core PF
In reply to Brett CreeleyQuoted 12 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMdrivers/vfio/pci/pds/cmds.c new file +8
-0+ +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +char devname[PDS_DEVNAME_LEN ]; +int ci; + + snprintf(devname, sizeof(devname), "%s.%d-%u",
PDS_LM_DEV_NAME,
drivers/vfio/pci/pds/cmds.c line 25
Quoted 5 lines
drivers/vfio/pci/pds/cmds.c new file +5
-0+ pci_domain_nr(pdev->bus), pds_vfio->pci_id); + + ci =pds_client_register (pci_physfn (pdev), devname); +if (ci <=0 ) +return ci;
'ci' cannot be 0 since pds_client_register() already converts 0 into -EIO.
btw the description of pds_client_register() is wrong. It said return 0 on success. should be positive client_id on success.
drivers/vfio/pci/pds/vfio_dev.c line 12
Quoted 5 lines
drivers/vfio/pci/pds/vfio_dev.c +4
-0+struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return pds_vfio->vfio_coredev.pdev; +}
Does this wrapper actually save the length?
drivers/vfio/pci/pds/vfio_dev.c static int pds_vfio_init_device(struct vfio_device *vdev) · line 41
Quoted 7 lines
(quoted diff) +2
-0+ dev_dbg(&pdev->dev, + "%s: PF %#04x VF %#04x (%d) vf_id %d domain %d
pds_vfio%p\n",drivers/vfio/pci/pds/vfio_dev.c +3
-0+ __func__,pci_dev_id (pdev->physfn), pds_vfio->pci_id, + pds_vfio->pci_id, pds_vfio->vf_id,pci_domain_nr (pdev->bus), + pds_vfio);
why printing pds_vfio->pci_id twice?
Quoted 4 lines
+#define
PDS_LM_DEV_NAMEPDS_CORE_DRV_NAME"."PDS_DEV_TYPE_LM_STR
+
should this name include a 'vfio' string?
Re: [PATCH v10 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 15 lines · Kevin
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 12 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMdrivers/vfio/pci/pds/cmds.c new file +8
-0+ +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +char devname[PDS_DEVNAME_LEN ]; +int ci; + + snprintf(devname, sizeof(devname), "%s.%d-%u",
PDS_LM_DEV_NAME,
Quoted 8 lines
drivers/vfio/pci/pds/cmds.c line 25
Quoted 5 lines
drivers/vfio/pci/pds/cmds.c new file +5
-0+ pci_domain_nr(pdev->bus), pds_vfio->pci_id); + + ci =pds_client_register (pci_physfn (pdev), devname); +if (ci <=0 ) +return ci;'ci' cannot be 0 since
pds_client_register()already converts 0 into-EIO.
Yeah, Shameer already mentioned this and I have already fixed this issue for the next revision. Thanks.
Quoted 3 lines
btw the description of
pds_client_register()is wrong. It said return 0 on success. should be positiveclient_idon success.
Yeah, this was also mentioned by Shameer. I will submit a follow on patch that updates the documentation in pds_client_register(). Thanks.
Quoted 8 lines
drivers/vfio/pci/pds/vfio_dev.c line 12
Quoted 6 lines
drivers/vfio/pci/pds/vfio_dev.c +4
-0+struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return pds_vfio->vfio_coredev.pdev; +}Does this wrapper actually save the length?o
It wasn't so much about length but encapsulating the multiple de-references and multiple uses into a function call.
Quoted 10 lines
drivers/vfio/pci/pds/vfio_dev.c static int pds_vfio_init_device(struct vfio_device *vdev) · line 41
Quoted 8 lines
(quoted diff) +2
-0+ dev_dbg(&pdev->dev, + "%s: PF %#04x VF %#04x (%d) vf_id %d domain %d
pds_vfio%p\n",drivers/vfio/pci/pds/vfio_dev.c +3
-0+ __func__,pci_dev_id (pdev->physfn), pds_vfio->pci_id, + pds_vfio->pci_id, pds_vfio->vf_id,pci_domain_nr (pdev->bus), + pds_vfio);why printing
pds_vfio->pci_idtwice?
Will fix. Thanks.
Quoted 7 lines
Quoted 5 lines
+#define
PDS_LM_DEV_NAMEPDS_CORE_DRV_NAME"."PDS_DEV_TYPE_LM_STR
+should this name include a 'vfio' string?
This aligns with what our DSC/firmware expects, so no it's not needed.
Tian, Kevin
reviewerRE: [PATCH v10 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 11 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 17, 2023 3:02 AMQuoted 7 lines
Quoted 5 lines
+#define
PDS_LM_DEV_NAMEPDS_CORE_DRV_NAME"."PDS_DEV_TYPE_LM_STR
+should this name include a 'vfio' string?
This aligns with what our DSC/firmware expects, so no it's not needed.
/**
* pds_client_register - Link the client to the firmware
* @pf_pdev: ptr to the PF driver struct
* @devname: name that includes service into, e.g. pds_core.vDPA
The comment mentions vDPA which confused me on whether the client should include a keyword of client to differentiate.
e.g. here the name is "pds_core.LM". If both VFIO/vDPA want to support live migration with pds_core will there be a conflict or fine for multiple drivers registering to a same service?
Brett Creeley
maintainerPATCH v10 4/7 vfio/pds: Add VFIO live migration support
In reply to Brett CreeleyAdd live migration support via the VFIO subsystem. The migration implementation aligns with the definition from uapi/vfio.h and uses the pds_core PF's adminq for device configuration.
The ability to suspend, resume, and transfer VF device state data is included along with the required admin queue command structures and implementations.
PDS_LM_CMD_SUSPEND and PDS_LM_CMD_SUSPEND_STATUS are added to support the VF device suspend operation.
PDS_LM_CMD_RESUME is added to support the VF device resume operation.
PDS_LM_CMD_STATUS is added to determine the exact size of the VF device state data.
PDS_LM_CMD_SAVE is added to get the VF device state data.
PDS_LM_CMD_RESTORE is added to restore the VF device with the previously saved data from PDS_LM_CMD_SAVE.
PDS_LM_CMD_HOST_VF_STATUS is added to notify the device when a migration is in/not-in progress from the host's perspective.
--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 319 ++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 8 +- drivers/vfio/pci/pds/lm.c | 421 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/lm.h | 41 ++++ drivers/vfio/pci/pds/pci_drv.c | 13 + drivers/vfio/pci/pds/vfio_dev.c | 120 ++++++++- drivers/vfio/pci/pds/vfio_dev.h | 11 + include/linux/pds/pds_adminq.h | 217 ++++++++++++++++ 9 files changed, 1149 insertions(+), 2 deletions(-) create mode 100644 drivers/vfio/pci/pds/lm.c create mode 100644 drivers/vfio/pci/pds/lm.h
drivers/vfio/pci/pds/Makefile +1 -0
@@ -5,5 +5,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o pds_vfio-y := \ cmds.o \ + lm.o \ pci_drv.o \ vfio_dev.o
drivers/vfio/pci/pds/cmds.c +319 -0
334 lines not on this page · raw message
drivers/vfio/pci/pds/cmds.h +7 -1
@@ -6,5 +6,11 @@int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio);void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); - +int pds_vfio_suspend_device_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_resume_device_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_get_lm_status_cmd (struct pds_vfio_pci_device *pds_vfio,u64 *size); +int pds_vfio_get_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_set_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_send_host_vf_lm_status_cmd (struct pds_vfio_pci_device *pds_vfio, +enum pds_lm_host_vf_status vf_status);#endif /* _CMDS_H_ */
drivers/vfio/pci/pds/lm.c new file +421 -0
421 lines not on this page · raw message
drivers/vfio/pci/pds/lm.h new file +41 -0
41 lines not on this page · raw message
drivers/vfio/pci/pds/pci_drv.c +13 -0
@@ -73,11 +73,24 @@ pds_vfio_pci_table[] = { };MODULE_DEVICE_TABLE (pci, pds_vfio_pci_table); +static void pds_vfio_pci_aer_reset_done (struct pci_dev *pdev) +{ +struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); + +pds_vfio_reset (pds_vfio); +} + +static const struct pci_error_handlers pds_vfio_pci_err_handlers = { + .reset_done = pds_vfio_pci_aer_reset_done, + .error_detected = vfio_pci_core_aer_err_detected, +}; +static struct pci_driver pds_vfio_pci_driver = { .name =KBUILD_MODNAME , .id_table = pds_vfio_pci_table, .probe = pds_vfio_pci_probe, .remove = pds_vfio_pci_remove, + .err_handler = &pds_vfio_pci_err_handlers, .driver_managed_dma = true, };
drivers/vfio/pci/pds/vfio_dev.c +119 -1
160 lines not on this page · raw message
drivers/vfio/pci/pds/vfio_dev.h +11 -0
@@ -7,12 +7,21 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +#include "lm.h" +struct pdsc ;struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev;struct pdsc *pdsc; +struct pds_vfio_lm_file *save_file; +struct pds_vfio_lm_file *restore_file; +struct mutex state_mutex;/* protect migration state */ +enum vfio_device_mig_state state; +spinlock_t reset_lock;/* protect reset_done flow */ +u8 deferred_reset; +int vf_id;int pci_id;u16 client_id;
@@ -20,7 +29,9 @@ struct pds_vfio_pci_device {const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio);struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +struct device *pds_vfio_to_dev (struct pds_vfio_pci_device *pds_vfio);#endif /* _VFIO_DEV_H_ */
include/linux/pds/pds_adminq.h +217 -0
235 lines not on this page · raw message
Signature · 1 line
2.17.1
RE: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 80 lines
-----Original Message-----
From: Brett Creeley [mailto:[email protected]]
Sent: 02 June 2023 23:03
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Shameerali Kolothum Thodi <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration supportAdd live migration support via the VFIO subsystem. The migration implementation aligns with the definition from
uapi/vfio.hand uses thepds_corePF's adminq for device configuration.The ability to suspend, resume, and transfer VF device state data is included along with the required admin queue command structures and implementations.
PDS_LM_CMD_SUSPENDandPDS_LM_CMD_SUSPEND_STATUSare added to support
the VF device suspend operation.
PDS_LM_CMD_RESUMEis added to support the VF device resume operation.
PDS_LM_CMD_STATUSis added to determine the exact size of the VF device state data.
PDS_LM_CMD_SAVEis added to get the VF device state data.
PDS_LM_CMD_RESTOREis added to restore the VF device with the previously saved data fromPDS_LM_CMD_SAVE.
PDS_LM_CMD_HOST_VF_STATUSis added to notify the device when a migration is in/not-in progress from the host's perspective.Signed-off-by Brett Creeley <[email protected]>Signed-off-by Shannon Nelson <[email protected]>--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 319 ++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 8 +- drivers/vfio/pci/pds/lm.c | 421 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/lm.h | 41 ++++ drivers/vfio/pci/pds/pci_drv.c | 13 + drivers/vfio/pci/pds/vfio_dev.c | 120 ++++++++- drivers/vfio/pci/pds/vfio_dev.h | 11 + include/linux/pds/pds_adminq.h | 217 ++++++++++++++++ 9 files changed, 1149 insertions(+), 2 deletions(-) create mode 100644 drivers/vfio/pci/pds/lm.c create mode 100644 drivers/vfio/pci/pds/lm.hdrivers/vfio/pci/pds/Makefile +1
-0@@ -5,5 +5,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o pds_vfio-y := \ cmds.o \ + lm.o \ pci_drv.o \ vfio_dev.odrivers/vfio/pci/pds/cmds.c +1
-0@@ -3,6 +3,7 @@#include <linux/io.h> #include <linux/types.h> +#include <linux/delay.h> #include <linux/pds/pds_common.h> #include <linux/pds/pds_core_if.h> @@ -11,6 +12,34 @@#include "vfio_dev.h" #include "cmds.h"
drivers/vfio/pci/pds/cmds.c line 23
Quoted 10 lines
drivers/vfio/pci/pds/cmds.c +8
-0+#define SUSPEND_TIMEOUT_S 5 +#define SUSPEND_CHECK_INTERVAL_MS 1 + +static int pds_vfio_client_adminq_cmd (struct pds_vfio_pci_device *pds_vfio, +union pds_core_adminq_cmd *req, +size_t req_len, +union pds_core_adminq_comp *resp, +u64 flags)
Why u64? Do we expect more flags to follow? The core interface below only takes a bool(fast_poll) though.
Thanks,
Shameer
Quoted 1,324 lines
drivers/vfio/pci/pds/cmds.c +20
-0+{ +union pds_core_adminq_cmd cmd = {}; +size_t cp_len; +int err; + +/* Wrap the client request */ + cmd.client_request.opcode =PDS_AQ_CMD_CLIENT_CMD ; + cmd.client_request.client_id =cpu_to_le16 (pds_vfio->client_id); + cp_len =min_t (size_t, req_len,sizeof (cmd.client_request.client_cmd)); +memcpy (cmd.client_request.client_cmd, req, cp_len); + + err =pdsc_adminq_post (pds_vfio->pdsc, &cmd, resp, + !!(flags &PDS_AQ_FLAG_FASTPOLL )); +if (err && err != -EAGAIN ) +dev_info (pds_vfio_to_dev (pds_vfio), +"client admin cmd failed: %pe\n" ,ERR_PTR (err)); + +return err; +} +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) {drivers/vfio/pci/pds/cmds.c +30
-0struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio);@@ -41,3 +70,293 @@ void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) pds_vfio->client_id = 0; } + +static int +pds_vfio_suspend_wait_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_suspend_status = { + .opcode =PDS_LM_CMD_SUSPEND_STATUS , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +unsigned long time_limit; +unsigned long time_start; +unsigned long time_done; +int err; + + time_start = jiffies; + time_limit = time_start +HZ *SUSPEND_TIMEOUT_S ; +do { + err =pds_vfio_client_adminq_cmd (pds_vfio, &cmd,sizeof (cmd), + &comp,PDS_AQ_FLAG_FASTPOLL ); +if (err != -EAGAIN ) +break ; + +msleep (SUSPEND_CHECK_INTERVAL_MS ); + }while (time_before (jiffies, time_limit)); + + time_done = jiffies; + dev_dbg(dev, "%s: vf%u: Suspend comp received in %d msecs\n",
__func__,drivers/vfio/pci/pds/cmds.c +27
-0+ pds_vfio->vf_id, jiffies_to_msecs(time_done - time_start)); + +/* Check the results */ +if (time_after_eq (time_done, time_limit)) { +dev_err (dev,"%s: vf%u: Suspend comp timeout\n" , __func__, + pds_vfio->vf_id); + err = -ETIMEDOUT ; + } + +return err; +} + +int pds_vfio_suspend_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_suspend = { + .opcode =PDS_LM_CMD_SUSPEND , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +int err; + +dev_dbg (dev,"vf%u: Suspend device\n" , pds_vfio->vf_id); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd),&comp,
drivers/vfio/pci/pds/cmds.c +24
-0+ PDS_AQ_FLAG_FASTPOLL); +if (err) { +dev_err (dev,"vf%u: Suspend failed: %pe\n" , pds_vfio->vf_id, +ERR_PTR (err)); +return err; + } + +return pds_vfio_suspend_wait_device_cmd (pds_vfio); +} + +int pds_vfio_resume_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_resume = { + .opcode =PDS_LM_CMD_RESUME , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; + +dev_dbg (dev,"vf%u: Resume device\n" , pds_vfio->vf_id); + + return pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd),&comp,
drivers/vfio/pci/pds/cmds.c +4
-0+ 0); +} + +int pds_vfio_get_lm_status_cmd(struct pds_vfio_pci_device *pds_vfio, u64*size)
drivers/vfio/pci/pds/cmds.c +14
-0+{ +union pds_core_adminq_cmd cmd = { + .lm_status = { + .opcode =PDS_LM_CMD_STATUS , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +int err; + +dev_dbg (dev,"vf%u: Get migration status\n" , pds_vfio->vf_id); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd), &comp,0);
drivers/vfio/pci/pds/cmds.c +105
-0+if (err) +return err; + + *size =le64_to_cpu (comp.lm_status.size); +return 0 ; +} + +static int pds_vfio_dma_map_lm_file (struct device *dev, +enum dma_data_direction dir, +struct pds_vfio_lm_file *lm_file) +{ +struct pds_lm_sg_elem *sgl, *sge; +struct scatterlist *sg; +dma_addr_t sgl_addr; +size_t sgl_size; +int err; +int i; + +if (!lm_file) +return -EINVAL ; + +/* dma map file pages */ + err =dma_map_sgtable (dev, &lm_file->sg_table, dir,0 ); +if (err) +return err; + + lm_file->num_sge = lm_file->sg_table.nents; + +/* alloc sgl */ + sgl_size = lm_file->num_sge *sizeof (struct pds_lm_sg_elem ); + sgl =kzalloc (sgl_size,GFP_KERNEL ); +if (!sgl) { + err = -ENOMEM ; + goto out_unmap_sgtable; + } + +/* fill sgl */ + sge = sgl; +for_each_sgtable_dma_sg (&lm_file->sg_table, sg, i) { + sge->addr =cpu_to_le64 (sg_dma_address (sg)); + sge->len =cpu_to_le32 (sg_dma_len (sg)); +dev_dbg (dev,"addr = %llx, len = %u\n" , sge->addr, sge->len); + sge++; + } + + sgl_addr =dma_map_single (dev, sgl, sgl_size,DMA_TO_DEVICE ); +if (dma_mapping_error (dev, sgl_addr)) { + err = -EIO ; + goto out_free_sgl; + } + + lm_file->sgl = sgl; + lm_file->sgl_addr = sgl_addr; + +return 0 ; + +out_free_sgl: +kfree (sgl); +out_unmap_sgtable: + lm_file->num_sge =0 ; +dma_unmap_sgtable (dev, &lm_file->sg_table, dir,0 ); +return err; +} + +static void pds_vfio_dma_unmap_lm_file (struct device *dev, +enum dma_data_direction dir, +struct pds_vfio_lm_file *lm_file) +{ +if (!lm_file) +return ; + +/* free sgl */ +if (lm_file->sgl) { +dma_unmap_single (dev, lm_file->sgl_addr, + lm_file->num_sge *sizeof (*lm_file->sgl), +DMA_TO_DEVICE ); +kfree (lm_file->sgl); + lm_file->sgl =NULL ; + lm_file->sgl_addr =DMA_MAPPING_ERROR ; + lm_file->num_sge =0 ; + } + +/* dma unmap file pages */ +dma_unmap_sgtable (dev, &lm_file->sg_table, dir,0 ); +} + +int pds_vfio_get_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_save = { + .opcode =PDS_LM_CMD_SAVE , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +struct device *pdsc_dev = &pci_physfn (pdev)->dev; +union pds_core_adminq_comp comp = {}; +struct pds_vfio_lm_file *lm_file; +int err; + +dev_dbg (&pdev->dev,"vf%u: Get migration state\n" , pds_vfio->vf_id); + + lm_file = pds_vfio->save_file; + + err = pds_vfio_dma_map_lm_file(pdsc_dev, DMA_FROM_DEVICE,
lm_file);drivers/vfio/pci/pds/cmds.c +10
-0+if (err) { +dev_err (&pdev->dev,"failed to map save migration file: %pe\n" , +ERR_PTR (err)); +return err; + } + + cmd.lm_save.sgl_addr =cpu_to_le64 (lm_file->sgl_addr); + cmd.lm_save.num_sge =cpu_to_le32 (lm_file->num_sge); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd), &comp,0);
drivers/vfio/pci/pds/cmds.c +39
-0+if (err) +dev_err (&pdev->dev,"failed to get migration state: %pe\n" , +ERR_PTR (err)); + +pds_vfio_dma_unmap_lm_file (pdsc_dev,DMA_FROM_DEVICE , lm_file); + +return err; +} + +int pds_vfio_set_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_restore = { + .opcode =PDS_LM_CMD_RESTORE , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +struct device *pdsc_dev = &pci_physfn (pdev)->dev; +union pds_core_adminq_comp comp = {}; +struct pds_vfio_lm_file *lm_file; +int err; + +dev_dbg (&pdev->dev,"vf%u: Set migration state\n" , pds_vfio->vf_id); + + lm_file = pds_vfio->restore_file; + + err =pds_vfio_dma_map_lm_file (pdsc_dev,DMA_TO_DEVICE , lm_file); +if (err) { +dev_err (&pdev->dev, +"failed to map restore migration file: %pe\n" , +ERR_PTR (err)); +return err; + } + + cmd.lm_restore.sgl_addr =cpu_to_le64 (lm_file->sgl_addr); + cmd.lm_restore.num_sge =cpu_to_le32 (lm_file->num_sge); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd), &comp,0);
drivers/vfio/pci/pds/cmds.c +10
-0+if (err) +dev_err (&pdev->dev,"failed to set migration state: %pe\n" , +ERR_PTR (err)); + +pds_vfio_dma_unmap_lm_file (pdsc_dev,DMA_TO_DEVICE , lm_file); + +return err; +} + +void pds_vfio_send_host_vf_lm_status_cmd(struct pds_vfio_pci_device
*pds_vfio,drivers/vfio/pci/pds/cmds.c +23
-0+ enum pds_lm_host_vf_status vf_status) +{ +union pds_core_adminq_cmd cmd = { + .lm_host_vf_status = { + .opcode =PDS_LM_CMD_HOST_VF_STATUS , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + .status = vf_status, + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +int err; + +dev_dbg (dev,"vf%u: Set host VF LM status: %u" , pds_vfio->vf_id, + vf_status); +if (vf_status !=PDS_LM_STA_IN_PROGRESS && + vf_status !=PDS_LM_STA_NONE ) { +dev_warn (dev,"Invalid host VF migration status, %d\n" , + vf_status); +return ; + } + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd), &comp,0);
drivers/vfio/pci/pds/cmds.c +4
-0+if (err) +dev_warn (dev,"failed to send host VF migration status: %pe\n" , +ERR_PTR (err)); +}drivers/vfio/pci/pds/cmds.h +6
-1@@ -6,5 +6,11 @@int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio);void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); - +int pds_vfio_suspend_device_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_resume_device_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_get_lm_status_cmd (struct pds_vfio_pci_device *pds_vfio,u64 *size); +int pds_vfio_get_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_set_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_send_host_vf_lm_status_cmd(struct pds_vfio_pci_device
*pds_vfio,drivers/vfio/pci/pds/cmds.h +1
-0+ enum pds_lm_host_vf_status vf_status);#endif /* _CMDS_H_ */ drivers/vfio/pci/pds/lm.c new file +268
-0@@ -0,0 +1,421 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/anon_inodes.h> +#include <linux/file.h> +#include <linux/fs.h> +#include <linux/highmem.h> +#include <linux/vfio.h> +#include <linux/vfio_pci_core.h> + +#include "vfio_dev.h" +#include "cmds.h" + +static struct pds_vfio_lm_file * +pds_vfio_get_lm_file (const struct file_operations *fops,int flags,u64 size) +{ +struct pds_vfio_lm_file *lm_file =NULL ; +unsigned long long npages; +struct page **pages; +void *page_mem; +const void *p; + +if (!size) +return NULL ; + +/* Alloc file structure */ + lm_file =kzalloc (sizeof (*lm_file),GFP_KERNEL ); +if (!lm_file) +return NULL ; + +/* Create file */ + lm_file->filep = +anon_inode_getfile ("pds_vfio_lm" , fops, lm_file, flags); +if (!lm_file->filep) + goto out_free_file; + +stream_open (lm_file->filep->f_inode, lm_file->filep); +mutex_init (&lm_file->lock); + +/* prevent file from being released before we are done with it */ +get_file (lm_file->filep); + +/* Allocate memory for file pages */ + npages =DIV_ROUND_UP_ULL (size,PAGE_SIZE ); + pages =kmalloc_array (npages,sizeof (*pages),GFP_KERNEL ); +if (!pages) + goto out_put_file; + + page_mem =kvzalloc (ALIGN (size,PAGE_SIZE ),GFP_KERNEL ); +if (!page_mem) + goto out_free_pages_array; + + p = page_mem -offset_in_page (page_mem); +for (unsigned long long i =0 ; i < npages; i++) { +if (is_vmalloc_addr (p)) + pages[i] =vmalloc_to_page (p); +else + pages[i] =kmap_to_page ((void *)p); +if (!pages[i]) + goto out_free_page_mem; + + p +=PAGE_SIZE ; + } + +/* Create scatterlist of file pages to use for DMA mapping later */ +if (sg_alloc_table_from_pages (&lm_file->sg_table, pages, npages,0 , + size,GFP_KERNEL )) + goto out_free_page_mem; + + lm_file->size = size; + lm_file->pages = pages; + lm_file->npages = npages; + lm_file->page_mem = page_mem; + lm_file->alloc_size = npages *PAGE_SIZE ; + +return lm_file; + +out_free_page_mem: +kvfree (page_mem); +out_free_pages_array: +kfree (pages); +out_put_file: +fput (lm_file->filep); +mutex_destroy (&lm_file->lock); +out_free_file: +kfree (lm_file); + +return NULL ; +} + +static void pds_vfio_put_lm_file (struct pds_vfio_lm_file *lm_file) +{ +mutex_lock (&lm_file->lock); + + lm_file->size =0 ; + lm_file->alloc_size =0 ; + +/* Free scatter list of file pages */ +sg_free_table (&lm_file->sg_table); + +kvfree (lm_file->page_mem); + lm_file->page_mem =NULL ; +kfree (lm_file->pages); + lm_file->pages =NULL ; + +mutex_unlock (&lm_file->lock); + +/* allow file to be released since we are done with it */ +fput (lm_file->filep); +} + +void pds_vfio_put_save_file (struct pds_vfio_pci_device *pds_vfio) +{ +if (!pds_vfio->save_file) +return ; + +pds_vfio_put_lm_file (pds_vfio->save_file); + pds_vfio->save_file =NULL ; +} + +void pds_vfio_put_restore_file (struct pds_vfio_pci_device *pds_vfio) +{ +if (!pds_vfio->restore_file) +return ; + +pds_vfio_put_lm_file (pds_vfio->restore_file); + pds_vfio->restore_file =NULL ; +} + +static struct page *pds_vfio_get_file_page (struct pds_vfio_lm_file *lm_file, +unsigned long offset) +{ +unsigned long cur_offset =0 ; +struct scatterlist *sg; +unsigned int i; + +/* All accesses are sequential */ +if (offset < lm_file->last_offset || !lm_file->last_offset_sg) { + lm_file->last_offset =0 ; + lm_file->last_offset_sg = lm_file->sg_table.sgl; + lm_file->sg_last_entry =0 ; + } + + cur_offset = lm_file->last_offset; + +for_each_sg (lm_file->last_offset_sg, sg, + lm_file->sg_table.orig_nents - lm_file->sg_last_entry, i) { +if (offset < sg->length + cur_offset) { + lm_file->last_offset_sg = sg; + lm_file->sg_last_entry += i; + lm_file->last_offset = cur_offset; +return nth_page (sg_page (sg), + (offset - cur_offset) /PAGE_SIZE ); + } + cur_offset += sg->length; + } + +return NULL ; +} + +static int pds_vfio_release_file (struct inode *inode,struct file *filp) +{ +struct pds_vfio_lm_file *lm_file = filp->private_data; + +mutex_lock (&lm_file->lock); + lm_file->filep->f_pos =0 ; + lm_file->size =0 ; +mutex_unlock (&lm_file->lock); +mutex_destroy (&lm_file->lock); +kfree (lm_file); + +return 0 ; +} + +static ssize_t pds_vfio_save_read (struct file *filp,char __user *buf, +size_t len,loff_t *pos) +{ +struct pds_vfio_lm_file *lm_file = filp->private_data; +ssize_t done =0 ; + +if (pos) +return -ESPIPE ; + pos = &filp->f_pos; + +mutex_lock (&lm_file->lock); +if (*pos > lm_file->size) { + done = -EINVAL ; + goto out_unlock; + } + + len =min_t (size_t, lm_file->size - *pos, len); +while (len) { +size_t page_offset; +struct page *page; +size_t page_len; +u8 *from_buff; +int err; + + page_offset = (*pos) %PAGE_SIZE ; + page =pds_vfio_get_file_page (lm_file, *pos - page_offset); +if (!page) { +if (done ==0 ) + done = -EINVAL ; + goto out_unlock; + } + + page_len =min_t (size_t, len,PAGE_SIZE - page_offset); + from_buff =kmap_local_page (page); + err =copy_to_user (buf, from_buff + page_offset, page_len); +kunmap_local (from_buff); +if (err) { + done = -EFAULT ; + goto out_unlock; + } + *pos += page_len; + len -= page_len; + done += page_len; + buf += page_len; + } + +out_unlock: +mutex_unlock (&lm_file->lock); +return done; +} + +static const struct file_operations pds_vfio_save_fops = { + .owner =THIS_MODULE , + .read = pds_vfio_save_read, + .release = pds_vfio_release_file, + .llseek = no_llseek, +}; + +static int pds_vfio_get_save_file (struct pds_vfio_pci_device *pds_vfio) +{ +struct device *dev = &pds_vfio->vfio_coredev.pdev->dev; +struct pds_vfio_lm_file *lm_file; +int err; +u64 size; + +/* Get live migration state size in this state */ + err =pds_vfio_get_lm_status_cmd (pds_vfio, &size); +if (err) { +dev_err (dev,"failed to get save status: %pe\n" ,ERR_PTR (err)); +return err; + } + +dev_dbg (dev,"save status, size = %lld\n" , size); + +if (!size) { +dev_err (dev,"invalid state size\n" ); +return -EIO ; + } + + lm_file =pds_vfio_get_lm_file (&pds_vfio_save_fops,O_RDONLY , size); +if (!lm_file) { +dev_err (dev,"failed to create save file\n" ); +return -ENOENT ; + } + +dev_dbg (dev,"size = %lld, alloc_size = %lld, npages = %lld\n" , + lm_file->size, lm_file->alloc_size, lm_file->npages); + + pds_vfio->save_file = lm_file; + +return 0 ; +} + +static ssize_t pds_vfio_restore_write(struct file *filp, const char __user*buf,
drivers/vfio/pci/pds/lm.c new file +73
-0+ size_t len, loff_t *pos) +{ + struct pds_vfio_lm_file *lm_file = filp->private_data; +loff_t requested_length; +ssize_t done =0 ; + +if (pos) +return -ESPIPE ; + + pos = &filp->f_pos; + +if (*pos <0 || +check_add_overflow ((loff_t )len, *pos, &requested_length)) +return -EINVAL ; + +mutex_lock (&lm_file->lock); + +while (len) { +size_t page_offset; +struct page *page; +size_t page_len; +u8 *to_buff; +int err; + + page_offset = (*pos) %PAGE_SIZE ; + page =pds_vfio_get_file_page (lm_file, *pos - page_offset); +if (!page) { +if (done ==0 ) + done = -EINVAL ; + goto out_unlock; + } + + page_len =min_t (size_t, len,PAGE_SIZE - page_offset); + to_buff =kmap_local_page (page); + err =copy_from_user (to_buff + page_offset, buf, page_len); +kunmap_local (to_buff); +if (err) { + done = -EFAULT ; + goto out_unlock; + } + *pos += page_len; + len -= page_len; + done += page_len; + buf += page_len; + lm_file->size += page_len; + } +out_unlock: +mutex_unlock (&lm_file->lock); +return done; +} + +static const struct file_operations pds_vfio_restore_fops = { + .owner =THIS_MODULE , + .write = pds_vfio_restore_write, + .release = pds_vfio_release_file, + .llseek = no_llseek, +}; + +static int pds_vfio_get_restore_file (struct pds_vfio_pci_device *pds_vfio) +{ +struct device *dev = &pds_vfio->vfio_coredev.pdev->dev; +struct pds_vfio_lm_file *lm_file; +u64 size; + + size =sizeof (union pds_lm_dev_state ); +dev_dbg (dev,"restore status, size = %lld\n" , size); + +if (!size) { +dev_err (dev,"invalid state size" ); +return -EIO ; + } + + lm_file = pds_vfio_get_lm_file(&pds_vfio_restore_fops, O_WRONLY,size);
drivers/vfio/pci/pds/lm.c new file +17
-0+if (!lm_file) { +dev_err (dev,"failed to create restore file" ); +return -ENOENT ; + } + pds_vfio->restore_file = lm_file; + +return 0 ; +} + +struct file * +pds_vfio_step_device_state_locked(struct pds_vfio_pci_device *pds_vfio, + enum vfio_device_mig_state next) +{ + enum vfio_device_mig_state cur = pds_vfio->state; + int err; + + if (cur == VFIO_DEVICE_STATE_STOP && next ==
VFIO_DEVICE_STATE_STOP_COPY) {drivers/vfio/pci/pds/lm.c new file +14
-0+ err =pds_vfio_get_save_file (pds_vfio); +if (err) +return ERR_PTR (err); + + err =pds_vfio_get_lm_state_cmd (pds_vfio); +if (err) { +pds_vfio_put_save_file (pds_vfio); +return ERR_PTR (err); + } + +return pds_vfio->save_file->filep; + } + + if (cur == VFIO_DEVICE_STATE_STOP_COPY && next ==
VFIO_DEVICE_STATE_STOP) {drivers/vfio/pci/pds/lm.c new file +2
-0+pds_vfio_put_save_file (pds_vfio); + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_NONE);drivers/vfio/pci/pds/lm.c new file +4
-0+return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_STOP && next ==
VFIO_DEVICE_STATE_RESUMING) {drivers/vfio/pci/pds/lm.c new file +8
-0+ err =pds_vfio_get_restore_file (pds_vfio); +if (err) +return ERR_PTR (err); + +return pds_vfio->restore_file->filep; + } + + if (cur == VFIO_DEVICE_STATE_RESUMING && next ==
VFIO_DEVICE_STATE_STOP) {drivers/vfio/pci/pds/lm.c new file +9
-0+ err =pds_vfio_set_lm_state_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + +pds_vfio_put_restore_file (pds_vfio); +return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_RUNNING && next ==
VFIO_DEVICE_STATE_RUNNING_P2P) {drivers/vfio/pci/pds/lm.c new file +10
-0+pds_vfio_send_host_vf_lm_status_cmd (pds_vfio, +PDS_LM_STA_IN_PROGRESS ); + err =pds_vfio_suspend_device_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + +return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==
VFIO_DEVICE_STATE_RUNNING) {drivers/vfio/pci/pds/lm.c new file +5
-0+ err =pds_vfio_resume_device_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_NONE);drivers/vfio/pci/pds/lm.c new file +4
-0+return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_STOP && next ==
VFIO_DEVICE_STATE_RUNNING_P2P)drivers/vfio/pci/pds/lm.c new file +3
-0+return NULL ; + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==
VFIO_DEVICE_STATE_STOP)drivers/vfio/pci/pds/lm.c new file +4
-0+return NULL ; + +return ERR_PTR (-EINVAL ); +}drivers/vfio/pci/pds/lm.h new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _LM_H_ +#define _LM_H_ + +#include <linux/fs.h> +#include <linux/mutex.h> +#include <linux/scatterlist.h> +#include <linux/types.h> + +#include <linux/pds/pds_common.h> +#include <linux/pds/pds_adminq.h> + +struct pds_vfio_lm_file { +struct file *filep; +struct mutex lock;/* protect live migration data file */ +u64 size;/* Size with valid data */ +u64 alloc_size;/* Total allocated size. Always >= len */ +void *page_mem;/* memory allocated for pages */ +struct page **pages;/* Backing pages for file */ +unsigned long long npages; +struct sg_table sg_table;/* SG table for backing pages */ +struct pds_lm_sg_elem *sgl;/* DMA mapping */ +dma_addr_t sgl_addr; +u16 num_sge; +struct scatterlist *last_offset_sg;/* Iterator */ +unsigned int sg_last_entry; +unsigned long last_offset; +}; + +struct pds_vfio_pci_device ; + +struct file * +pds_vfio_step_device_state_locked (struct pds_vfio_pci_device *pds_vfio, +enum vfio_device_mig_state next); + +void pds_vfio_put_save_file (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_put_restore_file (struct pds_vfio_pci_device *pds_vfio); + +#endif /* _LM_H_ */ drivers/vfio/pci/pds/pci_drv.c +13
-0@@ -73,11 +73,24 @@ pds_vfio_pci_table[] = { };MODULE_DEVICE_TABLE (pci, pds_vfio_pci_table); +static void pds_vfio_pci_aer_reset_done (struct pci_dev *pdev) +{ +struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); + +pds_vfio_reset (pds_vfio); +} + +static const struct pci_error_handlers pds_vfio_pci_err_handlers = { + .reset_done = pds_vfio_pci_aer_reset_done, + .error_detected = vfio_pci_core_aer_err_detected, +}; +static struct pci_driver pds_vfio_pci_driver = { .name =KBUILD_MODNAME , .id_table = pds_vfio_pci_table, .probe = pds_vfio_pci_probe, .remove = pds_vfio_pci_remove, + .err_handler = &pds_vfio_pci_err_handlers, .driver_managed_dma = true, };drivers/vfio/pci/pds/vfio_dev.c +6
-0@@ -4,6 +4,7 @@#include <linux/vfio.h> #include <linux/vfio_pci_core.h> +#include "lm.h" #include "vfio_dev.h" struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio)@@ -11,6 +12,11 @@ struct pci_dev *pds_vfio_to_pci_dev(struct pds_vfio_pci_device *pds_vfio)return pds_vfio->vfio_coredev.pdev; } +struct device *pds_vfio_to_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return &pds_vfio_to_pci_dev (pds_vfio)->dev; +} +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) {struct
vfio_pci_core_device*core_device=dev_get_drvdata(&pdev->dev);drivers/vfio/pci/pds/vfio_dev.c +92
-0@@ -19,6 +25,98 @@ struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev) vfio_coredev); } +static void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) +{ +again: +spin_lock (&pds_vfio->reset_lock); +if (pds_vfio->deferred_reset) { + pds_vfio->deferred_reset = false; +if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) { + pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; +pds_vfio_put_restore_file (pds_vfio); +pds_vfio_put_save_file (pds_vfio); + } +spin_unlock (&pds_vfio->reset_lock); + goto again; + } +mutex_unlock (&pds_vfio->state_mutex); +spin_unlock (&pds_vfio->reset_lock); +} + +void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio) +{ +spin_lock (&pds_vfio->reset_lock); + pds_vfio->deferred_reset = true; +if (!mutex_trylock (&pds_vfio->state_mutex)) { +spin_unlock (&pds_vfio->reset_lock); +return ; + } +spin_unlock (&pds_vfio->reset_lock); +pds_vfio_state_mutex_unlock (pds_vfio); +} + +static struct file * +pds_vfio_set_device_state (struct vfio_device *vdev, +enum vfio_device_mig_state new_state) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); +struct file *res =NULL ; + +mutex_lock (&pds_vfio->state_mutex); +while (new_state != pds_vfio->state) { +enum vfio_device_mig_state next_state; + +int err =vfio_mig_get_next_state (vdev, pds_vfio->state, + new_state, &next_state); +if (err) { + res =ERR_PTR (err); +break ; + } + + res =pds_vfio_step_device_state_locked (pds_vfio, next_state); +if (IS_ERR (res)) +break ; + + pds_vfio->state = next_state; + +if (WARN_ON (res && new_state != pds_vfio->state)) { + res =ERR_PTR (-EINVAL ); +break ; + } + } +pds_vfio_state_mutex_unlock (pds_vfio); + +return res; +} + +static int pds_vfio_get_device_state (struct vfio_device *vdev, +enum vfio_device_mig_state *current_state) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); + +mutex_lock (&pds_vfio->state_mutex); + *current_state = pds_vfio->state; +pds_vfio_state_mutex_unlock (pds_vfio); +return 0 ; +} + +static int pds_vfio_get_device_state_size (struct vfio_device *vdev, +unsigned long *stop_copy_length) +{ + *stop_copy_length =PDS_LM_DEVICE_STATE_LENGTH ; +return 0 ; +} + +static const struct vfio_migration_ops pds_vfio_lm_ops = { + .migration_set_state = pds_vfio_set_device_state, + .migration_get_state = pds_vfio_get_device_state, + .migration_get_data_size = pds_vfio_get_device_state_size +}; +static int pds_vfio_init_device (struct vfio_device *vdev){
drivers/vfio/pci/pds/vfio_dev.c +3
-0struct pds_vfio_pci_device *pds_vfio =@@ -34,6 +132,9 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id =pci_iov_vf_id (pdev); pds_vfio->pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); + vdev->migration_flags =VFIO_MIGRATION_STOP_COPY |VFIO_MIGRATION_P2P ; + vdev->mig_ops = &pds_vfio_lm_ops; + dev_dbg(&pdev->dev,"%s: PF %#04x VF %#04x (%d)
vf_id%d domain %dpds_vfio%p\n",drivers/vfio/pci/pds/vfio_dev.c +18
-1__func__, pci_dev_id(pdev->physfn), pds_vfio->pci_id,@@ -54,17 +155,34 @@ static int pds_vfio_open_device(struct vfio_device *vdev)if (err)return err; +mutex_init (&pds_vfio->state_mutex); + pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; +vfio_pci_core_finish_enable (&pds_vfio->vfio_coredev);return 0 ; } +static void pds_vfio_close_device (struct vfio_device *vdev) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); + +mutex_lock (&pds_vfio->state_mutex); +pds_vfio_put_restore_file (pds_vfio); +pds_vfio_put_save_file (pds_vfio); +mutex_unlock (&pds_vfio->state_mutex); +mutex_destroy (&pds_vfio->state_mutex); +vfio_pci_core_close_device (vdev); +} +static const struct vfio_device_ops pds_vfio_ops = { .name ="pds-vfio" , .init = pds_vfio_init_device, .release = vfio_pci_core_release_dev, .open_device = pds_vfio_open_device, - .close_device = vfio_pci_core_close_device, + .close_device = pds_vfio_close_device, .ioctl = vfio_pci_core_ioctl, .device_feature = vfio_pci_core_ioctl_feature,drivers/vfio/pci/pds/vfio_dev.c +0
-0.read = vfio_pci_core_read,drivers/vfio/pci/pds/vfio_dev.h +11
-0@@ -7,12 +7,21 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +#include "lm.h" +struct pdsc ;struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev;struct pdsc *pdsc; +struct pds_vfio_lm_file *save_file; +struct pds_vfio_lm_file *restore_file; +struct mutex state_mutex;/* protect migration state */ +enum vfio_device_mig_state state; +spinlock_t reset_lock;/* protect reset_done flow */ +u8 deferred_reset; +int vf_id;int pci_id;u16 client_id;@@ -20,7 +29,9 @@ struct pds_vfio_pci_device {const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio);struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +struct device *pds_vfio_to_dev (struct pds_vfio_pci_device *pds_vfio);#endif /* _VFIO_DEV_H_ */ include/linux/pds/pds_adminq.h +60
-0@@ -584,6 +584,213 @@ struct pds_core_q_init_comp {u8 color; }; +#define PDS_LM_DEVICE_STATE_LENGTH 65536 +#define PDS_LM_CHECK_DEVICE_STATE_LENGTH (X ) \ + PDS_CORE_SIZE_CHECK(union, PDS_LM_DEVICE_STATE_LENGTH, X) + +/* +* enum pds_lm_cmd_opcode - Live Migration Device commands +*/ +enum pds_lm_cmd_opcode { +PDS_LM_CMD_HOST_VF_STATUS =1 , + +/* Device state commands */ +PDS_LM_CMD_STATUS =16 , +PDS_LM_CMD_SUSPEND =18 , +PDS_LM_CMD_SUSPEND_STATUS =19 , +PDS_LM_CMD_RESUME =20 , +PDS_LM_CMD_SAVE =21 , +PDS_LM_CMD_RESTORE =22 , +}; + +/** +* struct pds_lm_cmd - generic command +* @opcode: Opcode +* @rsvd: Word boundary padding +* @vf_id: VF id +* @rsvd2: Structure padding to 60 Bytes +*/ +struct pds_lm_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 rsvd2[56 ]; +}; + +/** +* struct pds_lm_comp - generic command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Structure padding to 16 Bytes +*/ +struct pds_lm_comp { +u8 status; +u8 rsvd[15 ]; +}; + +/** +* struct pds_lm_status_cmd - STATUS command +* @opcode: Opcode +* @rsvd: Word boundary padding +* @vf_id: VF id +*/ +struct pds_lm_status_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_status_comp - STATUS command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Word boundary padding +* @comp_index: Index in the desc ring for which this is the completion
+ * @size: Size of the device state
+ * @rsvd2: Word boundary padding
+ * @color: Color bit
+ */include/linux/pds/pds_adminq.h +28
-0+struct pds_lm_status_comp { +u8 status; +u8 rsvd; +__le16 comp_index; +union { +__le64 size; +u8 rsvd2[11 ]; + } __packed; +u8 color; +}; + +/** +* struct pds_lm_suspend_cmd - SUSPEND command +* @opcode: Opcode PDS_LM_CMD_SUSPEND +* @rsvd: Word boundary padding +* @vf_id: VF id +*/ +struct pds_lm_suspend_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_suspend_comp - SUSPEND command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Word boundary padding +* @comp_index: Index in the desc ring for which this is the completion
+ * @state_size: Size of the device state computed post suspend
+ * @rsvd2: Word boundary padding
+ * @color: Color bit
+ */include/linux/pds/pds_adminq.h +103
-0+struct pds_lm_suspend_comp { +u8 status; +u8 rsvd; +__le16 comp_index; +union { +__le64 state_size; +u8 rsvd2[11 ]; + } __packed; +u8 color; +}; + +/** +* struct pds_lm_suspend_status_cmd - SUSPEND status command +* @opcode: Opcode PDS_AQ_CMD_LM_SUSPEND_STATUS +* @rsvd: Word boundary padding +* @vf_id: VF id +*/ +struct pds_lm_suspend_status_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_resume_cmd - RESUME command +* @opcode: Opcode PDS_LM_CMD_RESUME +* @rsvd: Word boundary padding +* @vf_id: VF id +*/ +struct pds_lm_resume_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_sg_elem - Transmit scatter-gather (SG) descriptor element +* @addr: DMA address of SG element data buffer +* @len: Length of SG element data buffer, in bytes +* @rsvd: Word boundary padding +*/ +struct pds_lm_sg_elem { +__le64 addr; +__le32 len; +__le16 rsvd[2 ]; +}; + +/** +* struct pds_lm_save_cmd - SAVE command +* @opcode: Opcode PDS_LM_CMD_SAVE +* @rsvd: Word boundary padding +* @vf_id: VF id +* @rsvd2: Word boundary padding +* @sgl_addr: IOVA address of the SGL to dma the device state +* @num_sge: Total number of SG elements +*/ +struct pds_lm_save_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 rsvd2[4 ]; +__le64 sgl_addr; +__le32 num_sge; +} __packed; + +/** +* struct pds_lm_restore_cmd - RESTORE command +* @opcode: Opcode PDS_LM_CMD_RESTORE +* @rsvd: Word boundary padding +* @vf_id: VF id +* @rsvd2: Word boundary padding +* @sgl_addr: IOVA address of the SGL to dma the device state +* @num_sge: Total number of SG elements +*/ +struct pds_lm_restore_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 rsvd2[4 ]; +__le64 sgl_addr; +__le32 num_sge; +} __packed; + +/** +* union pds_lm_dev_state - device state information +* @words: Device state words +*/ +union pds_lm_dev_state { +__le32 words[PDS_LM_DEVICE_STATE_LENGTH /sizeof (__le32)]; +}; + +enum pds_lm_host_vf_status { +PDS_LM_STA_NONE =0 , +PDS_LM_STA_IN_PROGRESS , +PDS_LM_STA_MAX , +}; + +/** +* struct pds_lm_host_vf_status_cmd - HOST_VF_STATUS command +* @opcode: Opcode PDS_LM_CMD_HOST_VF_STATUS +* @rsvd: Word boundary padding +* @vf_id: VF id +* @status: Current LM status of host VF driver (enum
pds_lm_host_status)
+ */include/linux/pds/pds_adminq.h +17
-1+struct pds_lm_host_vf_status_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 status; +}; +union pds_core_adminq_cmd {u8 opcode;u8 bytes[64 ];@@ -600,6 +807,14 @@ union pds_core_adminq_cmd {struct pds_core_q_identify_cmd q_ident;struct pds_core_q_init_cmd q_init; + +struct pds_lm_suspend_cmd lm_suspend; +struct pds_lm_suspend_status_cmd lm_suspend_status; +struct pds_lm_resume_cmd lm_resume; +struct pds_lm_status_cmd lm_status; +struct pds_lm_save_cmd lm_save; +struct pds_lm_restore_cmd lm_restore; +struct pds_lm_host_vf_status_cmd lm_host_vf_status; };union pds_core_adminq_comp {@@ -621,6 +836,8 @@ union pds_core_adminq_comp {struct pds_core_q_identify_comp q_ident;struct pds_core_q_init_comp q_init; + +struct pds_lm_status_comp lm_status; };#ifndef __CHECKER__ --2.17.1
Re: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 95 lines · Shameerali Kolothum Thodi
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 78 lines
-----Original Message-----
From: Brett Creeley [mailto:[email protected]]
Sent: 02 June 2023 23:03
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Shameerali Kolothum Thodi <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration supportAdd live migration support via the VFIO subsystem. The migration implementation aligns with the definition from
uapi/vfio.hand uses thepds_corePF's adminq for device configuration.The ability to suspend, resume, and transfer VF device state data is included along with the required admin queue command structures and implementations.
PDS_LM_CMD_SUSPENDandPDS_LM_CMD_SUSPEND_STATUSare added to support
the VF device suspend operation.
PDS_LM_CMD_RESUMEis added to support the VF device resume operation.
PDS_LM_CMD_STATUSis added to determine the exact size of the VF device state data.
PDS_LM_CMD_SAVEis added to get the VF device state data.
PDS_LM_CMD_RESTOREis added to restore the VF device with the previously saved data fromPDS_LM_CMD_SAVE.
PDS_LM_CMD_HOST_VF_STATUSis added to notify the device when a migration is in/not-in progress from the host's perspective.Signed-off-by Brett Creeley <[email protected]>Signed-off-by Shannon Nelson <[email protected]>--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 319 ++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 8 +- drivers/vfio/pci/pds/lm.c | 421 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/lm.h | 41 ++++ drivers/vfio/pci/pds/pci_drv.c | 13 + drivers/vfio/pci/pds/vfio_dev.c | 120 ++++++++- drivers/vfio/pci/pds/vfio_dev.h | 11 + include/linux/pds/pds_adminq.h | 217 ++++++++++++++++ 9 files changed, 1149 insertions(+), 2 deletions(-)create mode 100644
drivers/vfio/pci/pds/lm.c
create mode 100644drivers/vfio/pci/pds/lm.hdrivers/vfio/pci/pds/Makefile +1
-0@@ -5,5 +5,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o pds_vfio-y := \ cmds.o \ + lm.o \ pci_drv.o \ vfio_dev.odrivers/vfio/pci/pds/cmds.c +1
-0@@ -3,6 +3,7 @@#include <linux/io.h> #include <linux/types.h> +#include <linux/delay.h> #include <linux/pds/pds_common.h> #include <linux/pds/pds_core_if.h> @@ -11,6 +12,34 @@#include "vfio_dev.h" #include "cmds.h" drivers/vfio/pci/pds/cmds.c line 23
Quoted 10 lines
drivers/vfio/pci/pds/cmds.c +8
-0+#define SUSPEND_TIMEOUT_S 5 +#define SUSPEND_CHECK_INTERVAL_MS 1 + +static int pds_vfio_client_adminq_cmd (struct pds_vfio_pci_device *pds_vfio, +union pds_core_adminq_cmd *req, +size_t req_len, +union pds_core_adminq_comp *resp, +u64 flags)Why u64? Do we expect more flags to follow? The core interface below only takes a
bool(fast_poll)though.
Quoted 2 lines
Thanks,
Shameer >
Shameer,
Another good catch. This was leftover from the original set of patches, but using flags is definitely unnecessary in pds_vfio_client_adminq_cmd(). If we ever need more flags I can update then. I will change this to a bool in the next revision.
Thanks for the review,
Brett
Quoted 1,325 lines
Quoted 1,324 lines
drivers/vfio/pci/pds/cmds.c +20
-0+{ +union pds_core_adminq_cmd cmd = {}; +size_t cp_len; +int err; + +/* Wrap the client request */ + cmd.client_request.opcode =PDS_AQ_CMD_CLIENT_CMD ; + cmd.client_request.client_id =cpu_to_le16 (pds_vfio->client_id); + cp_len =min_t (size_t, req_len,sizeof (cmd.client_request.client_cmd)); +memcpy (cmd.client_request.client_cmd, req, cp_len); + + err =pdsc_adminq_post (pds_vfio->pdsc, &cmd, resp, + !!(flags &PDS_AQ_FLAG_FASTPOLL )); +if (err && err != -EAGAIN ) +dev_info (pds_vfio_to_dev (pds_vfio), +"client admin cmd failed: %pe\n" ,ERR_PTR (err)); + +return err; +} +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) {struct pci_dev *pdev = pds_vfio_to_pci_dev(pds_vfio);drivers/vfio/pci/pds/cmds.c +30
-0@@ -41,3 +70,293 @@ void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) pds_vfio->client_id = 0; } + +static int +pds_vfio_suspend_wait_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_suspend_status = { + .opcode =PDS_LM_CMD_SUSPEND_STATUS , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +unsigned long time_limit; +unsigned long time_start; +unsigned long time_done; +int err; + + time_start = jiffies; + time_limit = time_start +HZ *SUSPEND_TIMEOUT_S ; +do { + err =pds_vfio_client_adminq_cmd (pds_vfio, &cmd,sizeof (cmd), + &comp,PDS_AQ_FLAG_FASTPOLL ); +if (err != -EAGAIN ) +break ; + +msleep (SUSPEND_CHECK_INTERVAL_MS ); + }while (time_before (jiffies, time_limit)); + + time_done = jiffies; + dev_dbg(dev, "%s: vf%u: Suspend comp received in %d msecs\n",
__func__,drivers/vfio/pci/pds/cmds.c +27
-0+ pds_vfio->vf_id, jiffies_to_msecs(time_done - time_start)); + +/* Check the results */ +if (time_after_eq (time_done, time_limit)) { +dev_err (dev,"%s: vf%u: Suspend comp timeout\n" , __func__, + pds_vfio->vf_id); + err = -ETIMEDOUT ; + } + +return err; +} + +int pds_vfio_suspend_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_suspend = { + .opcode =PDS_LM_CMD_SUSPEND , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +int err; + +dev_dbg (dev,"vf%u: Suspend device\n" , pds_vfio->vf_id); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd),&comp,
drivers/vfio/pci/pds/cmds.c +24
-0+ PDS_AQ_FLAG_FASTPOLL); +if (err) { +dev_err (dev,"vf%u: Suspend failed: %pe\n" , pds_vfio->vf_id, +ERR_PTR (err)); +return err; + } + +return pds_vfio_suspend_wait_device_cmd (pds_vfio); +} + +int pds_vfio_resume_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_resume = { + .opcode =PDS_LM_CMD_RESUME , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; + +dev_dbg (dev,"vf%u: Resume device\n" , pds_vfio->vf_id); + + return pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd),&comp,
drivers/vfio/pci/pds/cmds.c +4
-0+ 0); +} + +int pds_vfio_get_lm_status_cmd(struct pds_vfio_pci_device *pds_vfio, u64*size)
drivers/vfio/pci/pds/cmds.c +14
-0+{ +union pds_core_adminq_cmd cmd = { + .lm_status = { + .opcode =PDS_LM_CMD_STATUS , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +int err; + +dev_dbg (dev,"vf%u: Get migration status\n" , pds_vfio->vf_id); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd), &comp,0);
drivers/vfio/pci/pds/cmds.c +105
-0+if (err) +return err; + + *size =le64_to_cpu (comp.lm_status.size); +return 0 ; +} + +static int pds_vfio_dma_map_lm_file (struct device *dev, +enum dma_data_direction dir, +struct pds_vfio_lm_file *lm_file) +{ +struct pds_lm_sg_elem *sgl, *sge; +struct scatterlist *sg; +dma_addr_t sgl_addr; +size_t sgl_size; +int err; +int i; + +if (!lm_file) +return -EINVAL ; + +/* dma map file pages */ + err =dma_map_sgtable (dev, &lm_file->sg_table, dir,0 ); +if (err) +return err; + + lm_file->num_sge = lm_file->sg_table.nents; + +/* alloc sgl */ + sgl_size = lm_file->num_sge *sizeof (struct pds_lm_sg_elem ); + sgl =kzalloc (sgl_size,GFP_KERNEL ); +if (!sgl) { + err = -ENOMEM ; + goto out_unmap_sgtable; + } + +/* fill sgl */ + sge = sgl; +for_each_sgtable_dma_sg (&lm_file->sg_table, sg, i) { + sge->addr =cpu_to_le64 (sg_dma_address (sg)); + sge->len =cpu_to_le32 (sg_dma_len (sg)); +dev_dbg (dev,"addr = %llx, len = %u\n" , sge->addr, sge->len); + sge++; + } + + sgl_addr =dma_map_single (dev, sgl, sgl_size,DMA_TO_DEVICE ); +if (dma_mapping_error (dev, sgl_addr)) { + err = -EIO ; + goto out_free_sgl; + } + + lm_file->sgl = sgl; + lm_file->sgl_addr = sgl_addr; + +return 0 ; + +out_free_sgl: +kfree (sgl); +out_unmap_sgtable: + lm_file->num_sge =0 ; +dma_unmap_sgtable (dev, &lm_file->sg_table, dir,0 ); +return err; +} + +static void pds_vfio_dma_unmap_lm_file (struct device *dev, +enum dma_data_direction dir, +struct pds_vfio_lm_file *lm_file) +{ +if (!lm_file) +return ; + +/* free sgl */ +if (lm_file->sgl) { +dma_unmap_single (dev, lm_file->sgl_addr, + lm_file->num_sge *sizeof (*lm_file->sgl), +DMA_TO_DEVICE ); +kfree (lm_file->sgl); + lm_file->sgl =NULL ; + lm_file->sgl_addr =DMA_MAPPING_ERROR ; + lm_file->num_sge =0 ; + } + +/* dma unmap file pages */ +dma_unmap_sgtable (dev, &lm_file->sg_table, dir,0 ); +} + +int pds_vfio_get_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_save = { + .opcode =PDS_LM_CMD_SAVE , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +struct device *pdsc_dev = &pci_physfn (pdev)->dev; +union pds_core_adminq_comp comp = {}; +struct pds_vfio_lm_file *lm_file; +int err; + +dev_dbg (&pdev->dev,"vf%u: Get migration state\n" , pds_vfio->vf_id); + + lm_file = pds_vfio->save_file; + + err = pds_vfio_dma_map_lm_file(pdsc_dev, DMA_FROM_DEVICE,
lm_file);drivers/vfio/pci/pds/cmds.c +10
-0+if (err) { +dev_err (&pdev->dev,"failed to map save migration file: %pe\n" , +ERR_PTR (err)); +return err; + } + + cmd.lm_save.sgl_addr =cpu_to_le64 (lm_file->sgl_addr); + cmd.lm_save.num_sge =cpu_to_le32 (lm_file->num_sge); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd), &comp,0);
drivers/vfio/pci/pds/cmds.c +39
-0+if (err) +dev_err (&pdev->dev,"failed to get migration state: %pe\n" , +ERR_PTR (err)); + +pds_vfio_dma_unmap_lm_file (pdsc_dev,DMA_FROM_DEVICE , lm_file); + +return err; +} + +int pds_vfio_set_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_restore = { + .opcode =PDS_LM_CMD_RESTORE , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +struct device *pdsc_dev = &pci_physfn (pdev)->dev; +union pds_core_adminq_comp comp = {}; +struct pds_vfio_lm_file *lm_file; +int err; + +dev_dbg (&pdev->dev,"vf%u: Set migration state\n" , pds_vfio->vf_id); + + lm_file = pds_vfio->restore_file; + + err =pds_vfio_dma_map_lm_file (pdsc_dev,DMA_TO_DEVICE , lm_file); +if (err) { +dev_err (&pdev->dev, +"failed to map restore migration file: %pe\n" , +ERR_PTR (err)); +return err; + } + + cmd.lm_restore.sgl_addr =cpu_to_le64 (lm_file->sgl_addr); + cmd.lm_restore.num_sge =cpu_to_le32 (lm_file->num_sge); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd), &comp,0);
drivers/vfio/pci/pds/cmds.c +10
-0+if (err) +dev_err (&pdev->dev,"failed to set migration state: %pe\n" , +ERR_PTR (err)); + +pds_vfio_dma_unmap_lm_file (pdsc_dev,DMA_TO_DEVICE , lm_file); + +return err; +} + +void pds_vfio_send_host_vf_lm_status_cmd(struct pds_vfio_pci_device
*pds_vfio,drivers/vfio/pci/pds/cmds.c +23
-0+ enum pds_lm_host_vf_status vf_status) +{ +union pds_core_adminq_cmd cmd = { + .lm_host_vf_status = { + .opcode =PDS_LM_CMD_HOST_VF_STATUS , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + .status = vf_status, + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +int err; + +dev_dbg (dev,"vf%u: Set host VF LM status: %u" , pds_vfio->vf_id, + vf_status); +if (vf_status !=PDS_LM_STA_IN_PROGRESS && + vf_status !=PDS_LM_STA_NONE ) { +dev_warn (dev,"Invalid host VF migration status, %d\n" , + vf_status); +return ; + } + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd), &comp,0);
drivers/vfio/pci/pds/cmds.c +4
-0+if (err) +dev_warn (dev,"failed to send host VF migration status: %pe\n" , +ERR_PTR (err)); +}drivers/vfio/pci/pds/cmds.h +6
-1@@ -6,5 +6,11 @@int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio);void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); - +int pds_vfio_suspend_device_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_resume_device_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_get_lm_status_cmd (struct pds_vfio_pci_device *pds_vfio,u64 *size); +int pds_vfio_get_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_set_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_send_host_vf_lm_status_cmd(struct pds_vfio_pci_device
*pds_vfio,drivers/vfio/pci/pds/cmds.h +1
-0+ enum pds_lm_host_vf_status vf_status);#endif /* _CMDS_H_ */ drivers/vfio/pci/pds/lm.c new file +268
-0@@ -0,0 +1,421 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/anon_inodes.h> +#include <linux/file.h> +#include <linux/fs.h> +#include <linux/highmem.h> +#include <linux/vfio.h> +#include <linux/vfio_pci_core.h> + +#include "vfio_dev.h" +#include "cmds.h" + +static struct pds_vfio_lm_file * +pds_vfio_get_lm_file (const struct file_operations *fops,int flags,u64 size) +{ +struct pds_vfio_lm_file *lm_file =NULL ; +unsigned long long npages; +struct page **pages; +void *page_mem; +const void *p; + +if (!size) +return NULL ; + +/* Alloc file structure */ + lm_file =kzalloc (sizeof (*lm_file),GFP_KERNEL ); +if (!lm_file) +return NULL ; + +/* Create file */ + lm_file->filep = +anon_inode_getfile ("pds_vfio_lm" , fops, lm_file, flags); +if (!lm_file->filep) + goto out_free_file; + +stream_open (lm_file->filep->f_inode, lm_file->filep); +mutex_init (&lm_file->lock); + +/* prevent file from being released before we are done with it */ +get_file (lm_file->filep); + +/* Allocate memory for file pages */ + npages =DIV_ROUND_UP_ULL (size,PAGE_SIZE ); + pages =kmalloc_array (npages,sizeof (*pages),GFP_KERNEL ); +if (!pages) + goto out_put_file; + + page_mem =kvzalloc (ALIGN (size,PAGE_SIZE ),GFP_KERNEL ); +if (!page_mem) + goto out_free_pages_array; + + p = page_mem -offset_in_page (page_mem); +for (unsigned long long i =0 ; i < npages; i++) { +if (is_vmalloc_addr (p)) + pages[i] =vmalloc_to_page (p); +else + pages[i] =kmap_to_page ((void *)p); +if (!pages[i]) + goto out_free_page_mem; + + p +=PAGE_SIZE ; + } + +/* Create scatterlist of file pages to use for DMA mapping later */ +if (sg_alloc_table_from_pages (&lm_file->sg_table, pages, npages,0 , + size,GFP_KERNEL )) + goto out_free_page_mem; + + lm_file->size = size; + lm_file->pages = pages; + lm_file->npages = npages; + lm_file->page_mem = page_mem; + lm_file->alloc_size = npages *PAGE_SIZE ; + +return lm_file; + +out_free_page_mem: +kvfree (page_mem); +out_free_pages_array: +kfree (pages); +out_put_file: +fput (lm_file->filep); +mutex_destroy (&lm_file->lock); +out_free_file: +kfree (lm_file); + +return NULL ; +} + +static void pds_vfio_put_lm_file (struct pds_vfio_lm_file *lm_file) +{ +mutex_lock (&lm_file->lock); + + lm_file->size =0 ; + lm_file->alloc_size =0 ; + +/* Free scatter list of file pages */ +sg_free_table (&lm_file->sg_table); + +kvfree (lm_file->page_mem); + lm_file->page_mem =NULL ; +kfree (lm_file->pages); + lm_file->pages =NULL ; + +mutex_unlock (&lm_file->lock); + +/* allow file to be released since we are done with it */ +fput (lm_file->filep); +} + +void pds_vfio_put_save_file (struct pds_vfio_pci_device *pds_vfio) +{ +if (!pds_vfio->save_file) +return ; + +pds_vfio_put_lm_file (pds_vfio->save_file); + pds_vfio->save_file =NULL ; +} + +void pds_vfio_put_restore_file (struct pds_vfio_pci_device *pds_vfio) +{ +if (!pds_vfio->restore_file) +return ; + +pds_vfio_put_lm_file (pds_vfio->restore_file); + pds_vfio->restore_file =NULL ; +} + +static struct page *pds_vfio_get_file_page (struct pds_vfio_lm_file *lm_file, +unsigned long offset) +{ +unsigned long cur_offset =0 ; +struct scatterlist *sg; +unsigned int i; + +/* All accesses are sequential */ +if (offset < lm_file->last_offset || !lm_file->last_offset_sg) { + lm_file->last_offset =0 ; + lm_file->last_offset_sg = lm_file->sg_table.sgl; + lm_file->sg_last_entry =0 ; + } + + cur_offset = lm_file->last_offset; + +for_each_sg (lm_file->last_offset_sg, sg, + lm_file->sg_table.orig_nents - lm_file->sg_last_entry, i) { +if (offset < sg->length + cur_offset) { + lm_file->last_offset_sg = sg; + lm_file->sg_last_entry += i; + lm_file->last_offset = cur_offset; +return nth_page (sg_page (sg), + (offset - cur_offset) /PAGE_SIZE ); + } + cur_offset += sg->length; + } + +return NULL ; +} + +static int pds_vfio_release_file (struct inode *inode,struct file *filp) +{ +struct pds_vfio_lm_file *lm_file = filp->private_data; + +mutex_lock (&lm_file->lock); + lm_file->filep->f_pos =0 ; + lm_file->size =0 ; +mutex_unlock (&lm_file->lock); +mutex_destroy (&lm_file->lock); +kfree (lm_file); + +return 0 ; +} + +static ssize_t pds_vfio_save_read (struct file *filp,char __user *buf, +size_t len,loff_t *pos) +{ +struct pds_vfio_lm_file *lm_file = filp->private_data; +ssize_t done =0 ; + +if (pos) +return -ESPIPE ; + pos = &filp->f_pos; + +mutex_lock (&lm_file->lock); +if (*pos > lm_file->size) { + done = -EINVAL ; + goto out_unlock; + } + + len =min_t (size_t, lm_file->size - *pos, len); +while (len) { +size_t page_offset; +struct page *page; +size_t page_len; +u8 *from_buff; +int err; + + page_offset = (*pos) %PAGE_SIZE ; + page =pds_vfio_get_file_page (lm_file, *pos - page_offset); +if (!page) { +if (done ==0 ) + done = -EINVAL ; + goto out_unlock; + } + + page_len =min_t (size_t, len,PAGE_SIZE - page_offset); + from_buff =kmap_local_page (page); + err =copy_to_user (buf, from_buff + page_offset, page_len); +kunmap_local (from_buff); +if (err) { + done = -EFAULT ; + goto out_unlock; + } + *pos += page_len; + len -= page_len; + done += page_len; + buf += page_len; + } + +out_unlock: +mutex_unlock (&lm_file->lock); +return done; +} + +static const struct file_operations pds_vfio_save_fops = { + .owner =THIS_MODULE , + .read = pds_vfio_save_read, + .release = pds_vfio_release_file, + .llseek = no_llseek, +}; + +static int pds_vfio_get_save_file (struct pds_vfio_pci_device *pds_vfio) +{ +struct device *dev = &pds_vfio->vfio_coredev.pdev->dev; +struct pds_vfio_lm_file *lm_file; +int err; +u64 size; + +/* Get live migration state size in this state */ + err =pds_vfio_get_lm_status_cmd (pds_vfio, &size); +if (err) { +dev_err (dev,"failed to get save status: %pe\n" ,ERR_PTR (err)); +return err; + } + +dev_dbg (dev,"save status, size = %lld\n" , size); + +if (!size) { +dev_err (dev,"invalid state size\n" ); +return -EIO ; + } + + lm_file =pds_vfio_get_lm_file (&pds_vfio_save_fops,O_RDONLY , size); +if (!lm_file) { +dev_err (dev,"failed to create save file\n" ); +return -ENOENT ; + } + +dev_dbg (dev,"size = %lld, alloc_size = %lld, npages = %lld\n" , + lm_file->size, lm_file->alloc_size, lm_file->npages); + + pds_vfio->save_file = lm_file; + +return 0 ; +} + +static ssize_t pds_vfio_restore_write(struct file *filp, const char __user*buf,
drivers/vfio/pci/pds/lm.c new file +73
-0+ size_t len, loff_t *pos) +{ + struct pds_vfio_lm_file *lm_file = filp->private_data; +loff_t requested_length; +ssize_t done =0 ; + +if (pos) +return -ESPIPE ; + + pos = &filp->f_pos; + +if (*pos <0 || +check_add_overflow ((loff_t )len, *pos, &requested_length)) +return -EINVAL ; + +mutex_lock (&lm_file->lock); + +while (len) { +size_t page_offset; +struct page *page; +size_t page_len; +u8 *to_buff; +int err; + + page_offset = (*pos) %PAGE_SIZE ; + page =pds_vfio_get_file_page (lm_file, *pos - page_offset); +if (!page) { +if (done ==0 ) + done = -EINVAL ; + goto out_unlock; + } + + page_len =min_t (size_t, len,PAGE_SIZE - page_offset); + to_buff =kmap_local_page (page); + err =copy_from_user (to_buff + page_offset, buf, page_len); +kunmap_local (to_buff); +if (err) { + done = -EFAULT ; + goto out_unlock; + } + *pos += page_len; + len -= page_len; + done += page_len; + buf += page_len; + lm_file->size += page_len; + } +out_unlock: +mutex_unlock (&lm_file->lock); +return done; +} + +static const struct file_operations pds_vfio_restore_fops = { + .owner =THIS_MODULE , + .write = pds_vfio_restore_write, + .release = pds_vfio_release_file, + .llseek = no_llseek, +}; + +static int pds_vfio_get_restore_file (struct pds_vfio_pci_device *pds_vfio) +{ +struct device *dev = &pds_vfio->vfio_coredev.pdev->dev; +struct pds_vfio_lm_file *lm_file; +u64 size; + + size =sizeof (union pds_lm_dev_state ); +dev_dbg (dev,"restore status, size = %lld\n" , size); + +if (!size) { +dev_err (dev,"invalid state size" ); +return -EIO ; + } + + lm_file = pds_vfio_get_lm_file(&pds_vfio_restore_fops, O_WRONLY,size);
drivers/vfio/pci/pds/lm.c new file +17
-0+if (!lm_file) { +dev_err (dev,"failed to create restore file" ); +return -ENOENT ; + } + pds_vfio->restore_file = lm_file; + +return 0 ; +} + +struct file * +pds_vfio_step_device_state_locked(struct pds_vfio_pci_device *pds_vfio, + enum vfio_device_mig_state next) +{ + enum vfio_device_mig_state cur = pds_vfio->state; + int err; + + if (cur == VFIO_DEVICE_STATE_STOP && next ==
VFIO_DEVICE_STATE_STOP_COPY) {drivers/vfio/pci/pds/lm.c new file +14
-0+ err =pds_vfio_get_save_file (pds_vfio); +if (err) +return ERR_PTR (err); + + err =pds_vfio_get_lm_state_cmd (pds_vfio); +if (err) { +pds_vfio_put_save_file (pds_vfio); +return ERR_PTR (err); + } + +return pds_vfio->save_file->filep; + } + + if (cur == VFIO_DEVICE_STATE_STOP_COPY && next ==
VFIO_DEVICE_STATE_STOP) {drivers/vfio/pci/pds/lm.c new file +2
-0+pds_vfio_put_save_file (pds_vfio); + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_NONE);drivers/vfio/pci/pds/lm.c new file +4
-0+return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_STOP && next ==
VFIO_DEVICE_STATE_RESUMING) {drivers/vfio/pci/pds/lm.c new file +8
-0+ err =pds_vfio_get_restore_file (pds_vfio); +if (err) +return ERR_PTR (err); + +return pds_vfio->restore_file->filep; + } + + if (cur == VFIO_DEVICE_STATE_RESUMING && next ==
VFIO_DEVICE_STATE_STOP) {drivers/vfio/pci/pds/lm.c new file +9
-0+ err =pds_vfio_set_lm_state_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + +pds_vfio_put_restore_file (pds_vfio); +return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_RUNNING && next ==
VFIO_DEVICE_STATE_RUNNING_P2P) {drivers/vfio/pci/pds/lm.c new file +10
-0+pds_vfio_send_host_vf_lm_status_cmd (pds_vfio, +PDS_LM_STA_IN_PROGRESS ); + err =pds_vfio_suspend_device_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + +return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==
VFIO_DEVICE_STATE_RUNNING) {drivers/vfio/pci/pds/lm.c new file +5
-0+ err =pds_vfio_resume_device_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_NONE);drivers/vfio/pci/pds/lm.c new file +4
-0+return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_STOP && next ==
VFIO_DEVICE_STATE_RUNNING_P2P)drivers/vfio/pci/pds/lm.c new file +3
-0+return NULL ; + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==
VFIO_DEVICE_STATE_STOP)drivers/vfio/pci/pds/lm.c new file +4
-0+return NULL ; + +return ERR_PTR (-EINVAL ); +}drivers/vfio/pci/pds/lm.h new file +41
-0@@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _LM_H_ +#define _LM_H_ + +#include <linux/fs.h> +#include <linux/mutex.h> +#include <linux/scatterlist.h> +#include <linux/types.h> + +#include <linux/pds/pds_common.h> +#include <linux/pds/pds_adminq.h> + +struct pds_vfio_lm_file { +struct file *filep; +struct mutex lock;/* protect live migration data file */ +u64 size;/* Size with valid data */ +u64 alloc_size;/* Total allocated size. Always >= len */ +void *page_mem;/* memory allocated for pages */ +struct page **pages;/* Backing pages for file */ +unsigned long long npages; +struct sg_table sg_table;/* SG table for backing pages */ +struct pds_lm_sg_elem *sgl;/* DMA mapping */ +dma_addr_t sgl_addr; +u16 num_sge; +struct scatterlist *last_offset_sg;/* Iterator */ +unsigned int sg_last_entry; +unsigned long last_offset; +}; + +struct pds_vfio_pci_device ; + +struct file * +pds_vfio_step_device_state_locked (struct pds_vfio_pci_device *pds_vfio, +enum vfio_device_mig_state next); + +void pds_vfio_put_save_file (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_put_restore_file (struct pds_vfio_pci_device *pds_vfio); + +#endif /* _LM_H_ */ drivers/vfio/pci/pds/pci_drv.c +13
-0@@ -73,11 +73,24 @@ pds_vfio_pci_table[] = { };MODULE_DEVICE_TABLE (pci, pds_vfio_pci_table); +static void pds_vfio_pci_aer_reset_done (struct pci_dev *pdev) +{ +struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); + +pds_vfio_reset (pds_vfio); +} + +static const struct pci_error_handlers pds_vfio_pci_err_handlers = { + .reset_done = pds_vfio_pci_aer_reset_done, + .error_detected = vfio_pci_core_aer_err_detected, +}; +static struct pci_driver pds_vfio_pci_driver = { .name =KBUILD_MODNAME , .id_table = pds_vfio_pci_table, .probe = pds_vfio_pci_probe, .remove = pds_vfio_pci_remove, + .err_handler = &pds_vfio_pci_err_handlers, .driver_managed_dma = true, };drivers/vfio/pci/pds/vfio_dev.c +6
-0@@ -4,6 +4,7 @@#include <linux/vfio.h> #include <linux/vfio_pci_core.h> +#include "lm.h" #include "vfio_dev.h" struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio)@@ -11,6 +12,11 @@ struct pci_dev *pds_vfio_to_pci_dev(struct pds_vfio_pci_device *pds_vfio)return pds_vfio->vfio_coredev.pdev; } +struct device *pds_vfio_to_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return &pds_vfio_to_pci_dev (pds_vfio)->dev; +} +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) {struct
vfio_pci_core_device*core_device=dev_get_drvdata(&pdev->dev);drivers/vfio/pci/pds/vfio_dev.c +92
-0@@ -19,6 +25,98 @@ struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev) vfio_coredev); } +static void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) +{ +again: +spin_lock (&pds_vfio->reset_lock); +if (pds_vfio->deferred_reset) { + pds_vfio->deferred_reset = false; +if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) { + pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; +pds_vfio_put_restore_file (pds_vfio); +pds_vfio_put_save_file (pds_vfio); + } +spin_unlock (&pds_vfio->reset_lock); + goto again; + } +mutex_unlock (&pds_vfio->state_mutex); +spin_unlock (&pds_vfio->reset_lock); +} + +void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio) +{ +spin_lock (&pds_vfio->reset_lock); + pds_vfio->deferred_reset = true; +if (!mutex_trylock (&pds_vfio->state_mutex)) { +spin_unlock (&pds_vfio->reset_lock); +return ; + } +spin_unlock (&pds_vfio->reset_lock); +pds_vfio_state_mutex_unlock (pds_vfio); +} + +static struct file * +pds_vfio_set_device_state (struct vfio_device *vdev, +enum vfio_device_mig_state new_state) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); +struct file *res =NULL ; + +mutex_lock (&pds_vfio->state_mutex); +while (new_state != pds_vfio->state) { +enum vfio_device_mig_state next_state; + +int err =vfio_mig_get_next_state (vdev, pds_vfio->state, + new_state, &next_state); +if (err) { + res =ERR_PTR (err); +break ; + } + + res =pds_vfio_step_device_state_locked (pds_vfio, next_state); +if (IS_ERR (res)) +break ; + + pds_vfio->state = next_state; + +if (WARN_ON (res && new_state != pds_vfio->state)) { + res =ERR_PTR (-EINVAL ); +break ; + } + } +pds_vfio_state_mutex_unlock (pds_vfio); + +return res; +} + +static int pds_vfio_get_device_state (struct vfio_device *vdev, +enum vfio_device_mig_state *current_state) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); + +mutex_lock (&pds_vfio->state_mutex); + *current_state = pds_vfio->state; +pds_vfio_state_mutex_unlock (pds_vfio); +return 0 ; +} + +static int pds_vfio_get_device_state_size (struct vfio_device *vdev, +unsigned long *stop_copy_length) +{ + *stop_copy_length =PDS_LM_DEVICE_STATE_LENGTH ; +return 0 ; +} + +static const struct vfio_migration_ops pds_vfio_lm_ops = { + .migration_set_state = pds_vfio_set_device_state, + .migration_get_state = pds_vfio_get_device_state, + .migration_get_data_size = pds_vfio_get_device_state_size +}; +static int pds_vfio_init_device (struct vfio_device *vdev){ struct pds_vfio_pci_device *pds_vfio =drivers/vfio/pci/pds/vfio_dev.c +3
-0@@ -34,6 +132,9 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id =pci_iov_vf_id (pdev); pds_vfio->pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); + vdev->migration_flags =VFIO_MIGRATION_STOP_COPY |VFIO_MIGRATION_P2P ; + vdev->mig_ops = &pds_vfio_lm_ops; + dev_dbg(&pdev->dev,"%s: PF %#04x VF %#04x (%d) vf_id %d domain %d pds_vfio %p\n", __func__, pci_dev_id(pdev->physfn), pds_vfio->pci_id,drivers/vfio/pci/pds/vfio_dev.c +18
-1@@ -54,17 +155,34 @@ static int pds_vfio_open_device(struct vfio_device *vdev)if (err)return err; +mutex_init (&pds_vfio->state_mutex); + pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; +vfio_pci_core_finish_enable (&pds_vfio->vfio_coredev);return 0 ; } +static void pds_vfio_close_device (struct vfio_device *vdev) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); + +mutex_lock (&pds_vfio->state_mutex); +pds_vfio_put_restore_file (pds_vfio); +pds_vfio_put_save_file (pds_vfio); +mutex_unlock (&pds_vfio->state_mutex); +mutex_destroy (&pds_vfio->state_mutex); +vfio_pci_core_close_device (vdev); +} +static const struct vfio_device_ops pds_vfio_ops = { .name ="pds-vfio" , .init = pds_vfio_init_device, .release = vfio_pci_core_release_dev, .open_device = pds_vfio_open_device, - .close_device = vfio_pci_core_close_device, + .close_device = pds_vfio_close_device, .ioctl = vfio_pci_core_ioctl, .device_feature = vfio_pci_core_ioctl_feature,.read =
vfio_pci_core_read,drivers/vfio/pci/pds/vfio_dev.h +11
-0@@ -7,12 +7,21 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +#include "lm.h" +struct pdsc ;struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev;struct pdsc *pdsc; +struct pds_vfio_lm_file *save_file; +struct pds_vfio_lm_file *restore_file; +struct mutex state_mutex;/* protect migration state */ +enum vfio_device_mig_state state; +spinlock_t reset_lock;/* protect reset_done flow */ +u8 deferred_reset; +int vf_id;int pci_id;u16 client_id;@@ -20,7 +29,9 @@ struct pds_vfio_pci_device {const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio);struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +struct device *pds_vfio_to_dev (struct pds_vfio_pci_device *pds_vfio);#endif /* _VFIO_DEV_H_ */ include/linux/pds/pds_adminq.h +60
-0@@ -584,6 +584,213 @@ struct pds_core_q_init_comp {u8 color; }; +#define PDS_LM_DEVICE_STATE_LENGTH 65536 +#define PDS_LM_CHECK_DEVICE_STATE_LENGTH (X ) \ + PDS_CORE_SIZE_CHECK(union, PDS_LM_DEVICE_STATE_LENGTH, X) + +/* +* enum pds_lm_cmd_opcode - Live Migration Device commands +*/ +enum pds_lm_cmd_opcode { +PDS_LM_CMD_HOST_VF_STATUS =1 , + +/* Device state commands */ +PDS_LM_CMD_STATUS =16 , +PDS_LM_CMD_SUSPEND =18 , +PDS_LM_CMD_SUSPEND_STATUS =19 , +PDS_LM_CMD_RESUME =20 , +PDS_LM_CMD_SAVE =21 , +PDS_LM_CMD_RESTORE =22 , +}; + +/** +* struct pds_lm_cmd - generic command +* @opcode: Opcode +* @rsvd: Word boundary padding +* @vf_id: VF id +* @rsvd2: Structure padding to 60 Bytes +*/ +struct pds_lm_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 rsvd2[56 ]; +}; + +/** +* struct pds_lm_comp - generic command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Structure padding to 16 Bytes +*/ +struct pds_lm_comp { +u8 status; +u8 rsvd[15 ]; +}; + +/** +* struct pds_lm_status_cmd - STATUS command +* @opcode: Opcode +* @rsvd: Word boundary padding +* @vf_id: VF id +*/ +struct pds_lm_status_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_status_comp - STATUS command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Word boundary padding +* @comp_index: Index in the desc ring for which this is the completion
+ * @size: Size of the device state + * @rsvd2: Word boundary padding + * @color: Color bit+ */
include/linux/pds/pds_adminq.h +28
-0+struct pds_lm_status_comp { +u8 status; +u8 rsvd; +__le16 comp_index; +union { +__le64 size; +u8 rsvd2[11 ]; + } __packed; +u8 color; +}; + +/** +* struct pds_lm_suspend_cmd - SUSPEND command +* @opcode: Opcode PDS_LM_CMD_SUSPEND +* @rsvd: Word boundary padding +* @vf_id: VF id +*/ +struct pds_lm_suspend_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_suspend_comp - SUSPEND command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Word boundary padding +* @comp_index: Index in the desc ring for which this is the completion
+ * @state_size: Size of the device state computed post suspend
+ * @rsvd2: Word boundary padding
+ * @color: Color bit
+ */include/linux/pds/pds_adminq.h +103
-0+struct pds_lm_suspend_comp { +u8 status; +u8 rsvd; +__le16 comp_index; +union { +__le64 state_size; +u8 rsvd2[11 ]; + } __packed; +u8 color; +}; + +/** +* struct pds_lm_suspend_status_cmd - SUSPEND status command +* @opcode: Opcode PDS_AQ_CMD_LM_SUSPEND_STATUS +* @rsvd: Word boundary padding +* @vf_id: VF id +*/ +struct pds_lm_suspend_status_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_resume_cmd - RESUME command +* @opcode: Opcode PDS_LM_CMD_RESUME +* @rsvd: Word boundary padding +* @vf_id: VF id +*/ +struct pds_lm_resume_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_sg_elem - Transmit scatter-gather (SG) descriptor element +* @addr: DMA address of SG element data buffer +* @len: Length of SG element data buffer, in bytes +* @rsvd: Word boundary padding +*/ +struct pds_lm_sg_elem { +__le64 addr; +__le32 len; +__le16 rsvd[2 ]; +}; + +/** +* struct pds_lm_save_cmd - SAVE command +* @opcode: Opcode PDS_LM_CMD_SAVE +* @rsvd: Word boundary padding +* @vf_id: VF id +* @rsvd2: Word boundary padding +* @sgl_addr: IOVA address of the SGL to dma the device state +* @num_sge: Total number of SG elements +*/ +struct pds_lm_save_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 rsvd2[4 ]; +__le64 sgl_addr; +__le32 num_sge; +} __packed; + +/** +* struct pds_lm_restore_cmd - RESTORE command +* @opcode: Opcode PDS_LM_CMD_RESTORE +* @rsvd: Word boundary padding +* @vf_id: VF id +* @rsvd2: Word boundary padding +* @sgl_addr: IOVA address of the SGL to dma the device state +* @num_sge: Total number of SG elements +*/ +struct pds_lm_restore_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 rsvd2[4 ]; +__le64 sgl_addr; +__le32 num_sge; +} __packed; + +/** +* union pds_lm_dev_state - device state information +* @words: Device state words +*/ +union pds_lm_dev_state { +__le32 words[PDS_LM_DEVICE_STATE_LENGTH /sizeof (__le32)]; +}; + +enum pds_lm_host_vf_status { +PDS_LM_STA_NONE =0 , +PDS_LM_STA_IN_PROGRESS , +PDS_LM_STA_MAX , +}; + +/** +* struct pds_lm_host_vf_status_cmd - HOST_VF_STATUS command +* @opcode: Opcode PDS_LM_CMD_HOST_VF_STATUS +* @rsvd: Word boundary padding +* @vf_id: VF id +* @status: Current LM status of host VF driver (enum
pds_lm_host_status)
+ */include/linux/pds/pds_adminq.h +17
-1+struct pds_lm_host_vf_status_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 status; +}; +union pds_core_adminq_cmd {u8 opcode;u8 bytes[64 ];@@ -600,6 +807,14 @@ union pds_core_adminq_cmd {struct pds_core_q_identify_cmd q_ident;struct pds_core_q_init_cmd q_init; + +struct pds_lm_suspend_cmd lm_suspend; +struct pds_lm_suspend_status_cmd lm_suspend_status; +struct pds_lm_resume_cmd lm_resume; +struct pds_lm_status_cmd lm_status; +struct pds_lm_save_cmd lm_save; +struct pds_lm_restore_cmd lm_restore; +struct pds_lm_host_vf_status_cmd lm_host_vf_status; };union pds_core_adminq_comp {@@ -621,6 +836,8 @@ union pds_core_adminq_comp {struct pds_core_q_identify_comp q_ident;struct pds_core_q_init_comp q_init; + +struct pds_lm_status_comp lm_status; };#ifndef __CHECKER__ --2.17.1
Tian, Kevin
reviewerRE: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
In reply to Brett CreeleyQuoted 20 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMAdd live migration support via the VFIO subsystem. The migration implementation aligns with the definition from
uapi/vfio.hand uses thepds_corePF's adminq for device configuration.The ability to suspend, resume, and transfer VF device state data is included along with the required admin queue command structures and implementations.
PDS_LM_CMD_SUSPENDandPDS_LM_CMD_SUSPEND_STATUSare added to support
the VF device suspend operation.
PDS_LM_CMD_RESUMEis added to support the VF device resume operation.
PDS_LM_CMD_STATUSis added to determine the exact size of the VF device state data.
Quoted 7 lines
PDS_LM_CMD_SAVEis added to get the VF device state data.
PDS_LM_CMD_RESTOREis added to restore the VF device with the previously saved data fromPDS_LM_CMD_SAVE.
PDS_LM_CMD_HOST_VF_STATUSis added to notify the device when a migration is in/not-in progress from the host's perspective.
Here is 'the device' referring to the PF or VF?
and how would the device use this information?
Quoted 15 lines
drivers/vfio/pci/pds/cmds.c +15
-0+ +static int pds_vfio_client_adminq_cmd (struct pds_vfio_pci_device *pds_vfio, +union pds_core_adminq_cmd *req, +size_t req_len, +union pds_core_adminq_comp *resp, +u64 flags) +{ +union pds_core_adminq_cmd cmd = {}; +size_t cp_len; +int err; + +/* Wrap the client request */ + cmd.client_request.opcode =PDS_AQ_CMD_CLIENT_CMD ; + cmd.client_request.client_id =cpu_to_le16 (pds_vfio->client_id); + cp_len = min_t(size_t, req_len,
Quoted 1 line
sizeof(cmd.client_request.client_cmd));
'req_len' is kind of redundant. Looks all the callers use sizeof(req).
Quoted 23 lines
drivers/vfio/pci/pds/cmds.c +20
-0+static int +pds_vfio_suspend_wait_device_cmd(struct pds_vfio_pci_device *pds_vfio) +{ + union pds_core_adminq_cmd cmd = { + .lm_suspend_status = { + .opcode = PDS_LM_CMD_SUSPEND_STATUS, + .vf_id = cpu_to_le16(pds_vfio->vf_id), + }, + }; + struct device *dev = pds_vfio_to_dev(pds_vfio); + union pds_core_adminq_comp comp = {}; + unsigned long time_limit; + unsigned long time_start; + unsigned long time_done; + int err; + + time_start = jiffies; + time_limit = time_start + HZ * SUSPEND_TIMEOUT_S; + do { + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd,sizeof(cmd),
(quoted diff) +1
-0+ &comp,
PDS_AQ_FLAG_FASTPOLL);
drivers/vfio/pci/pds/cmds.c void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 99
Quoted 5 lines
drivers/vfio/pci/pds/cmds.c +5
-0+if (err != -EAGAIN ) +break ; + +msleep (SUSPEND_CHECK_INTERVAL_MS ); + }while (time_before (jiffies, time_limit));
pds_vfio_client_adminq_cmd() has the exactly same mechanism with 5s timeout and 1ms poll interval when FASTPOLL is set.
probably you can introduce another flag to indicate retry on -EAGAIN and then handle it fully in pds_vfio_client_adminq_cmd()?
Quoted 16 lines
drivers/vfio/pci/pds/cmds.c +15
-0+int pds_vfio_suspend_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_suspend = { + .opcode =PDS_LM_CMD_SUSPEND , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +int err; + +dev_dbg (dev,"vf%u: Suspend device\n" , pds_vfio->vf_id); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd),&comp,
drivers/vfio/pci/pds/cmds.c void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 138
Quoted 9 lines
drivers/vfio/pci/pds/cmds.c +9
-0+ PDS_AQ_FLAG_FASTPOLL); +if (err) { +dev_err (dev,"vf%u: Suspend failed: %pe\n" , pds_vfio->vf_id, +ERR_PTR (err)); +return err; + } + +return pds_vfio_suspend_wait_device_cmd (pds_vfio); +}
The logic in this function is very confusing.
PDS_LM_CMD_SUSPEND has a completion record:
+struct pds_lm_suspend_comp {
+ u8 status;
+ u8 rsvd;
+ __le16 comp_index;
+ union {
+ __le64 state_size;
+ u8 rsvd2[11];
+ } __packed;
+ u8 color;
Presumably this function can look at the completion record to know whether the suspend request succeeds.
Why do you require another wait_device step to query the suspend status?
and I have another question. Is it correct to hard-code the 5s timeout in the kernel w/o any input from the VMM? Note the guest has been stopped at this point then very likely the 5s timeout will kill any reasonable SLA which CSPs try to reach hard.
Ideally the VMM has an estimation how long a VM can be paused based on SLA, to-be-migrated state size, available network bandwidth, etc. and that hint should be passed to the kernel so any state transition which may violate that expectation can fail quickly to break the migration process and put the VM back to the running state.
Jason/Shameer, is there similar concern in mlx/hisilicon drivers?
Quoted 16 lines
drivers/vfio/pci/pds/cmds.c +15
-0+ +int pds_vfio_resume_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_resume = { + .opcode =PDS_LM_CMD_RESUME , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; + +dev_dbg (dev,"vf%u: Resume device\n" , pds_vfio->vf_id); + + return pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd),&comp,
drivers/vfio/pci/pds/cmds.c void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 154
Quoted 1 line
drivers/vfio/pci/pds/cmds.c +1
-0+ 0);
'resume' is also in the blackout phase when the guest is not running.
So presumably FAST_POLL should be set otherwise the max 256ms poll interval (PDSC_ADMINQ_MAX_POLL_INTERVAL) is really inefficient.
Quoted 30 lines
(quoted diff) +2
-0+ + if (cur == VFIO_DEVICE_STATE_RUNNING && next ==
VFIO_DEVICE_STATE_RUNNING_P2P) {drivers/vfio/pci/pds/lm.c new file +2
-0+ pds_vfio_send_host_vf_lm_status_cmd(pds_vfio, +
PDS_LM_STA_IN_PROGRESS);drivers/vfio/pci/pds/lm.c new file +8
-0+ err =pds_vfio_suspend_device_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + +return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==
VFIO_DEVICE_STATE_RUNNING) {drivers/vfio/pci/pds/lm.c new file +5
-0+ err =pds_vfio_resume_device_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_NONE);drivers/vfio/pci/pds/lm.c new file +4
-0+return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_STOP && next ==
VFIO_DEVICE_STATE_RUNNING_P2P)drivers/vfio/pci/pds/lm.c new file +3
-0+return NULL ; + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==
VFIO_DEVICE_STATE_STOP)
Review
Quoted 1 line
(quoted diff) +1
-0+ return NULL;
I'm not sure whether P2P is actually supported here. By definition P2P means the device is stopped but still responds to p2p request from other devices. If you look at mlx example it uses different cmds between RUNNING->RUNNING_P2P and RUNNING_P2P->STOP.
But in your case seems you simply move what is required in STOP into P2P. Probably you can just remove the support of P2P like hisilicon does.
include/linux/pds/pds_adminq.h struct pds_core_q_init_comp { · line 628
Quoted 10 lines
include/linux/pds/pds_adminq.h +10
-0+ +/** +* struct pds_lm_comp - generic command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Structure padding to 16 Bytes +*/ +struct pds_lm_comp { +u8 status; +u8 rsvd[15 ]; +};
not used. Looks most comp structures are defined w/o an user except struct pds_lm_status_comp.
Re: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 25 lines · Kevin
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 22 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMAdd live migration support via the VFIO subsystem. The migration implementation aligns with the definition from
uapi/vfio.hand uses thepds_corePF's adminq for device configuration.The ability to suspend, resume, and transfer VF device state data is included along with the required admin queue command structures and implementations.
PDS_LM_CMD_SUSPENDandPDS_LM_CMD_SUSPEND_STATUSare added to support
the VF device suspend operation.
PDS_LM_CMD_RESUMEis added to support the VF device resume operation.
PDS_LM_CMD_STATUSis added to determine the exact size of the VF device state data.
PDS_LM_CMD_SAVEis added to get the VF device state data.
Quoted 7 lines
Quoted 5 lines
PDS_LM_CMD_RESTOREis added to restore the VF device with the previously saved data fromPDS_LM_CMD_SAVE.
PDS_LM_CMD_HOST_VF_STATUSis added to notify the device when a migration is in/not-in progress from the host's perspective.Here is 'the device' referring to the PF or VF?
Device is referring to the DSC/firmware not the function. I will clarify the wording here. Thanks.
Quoted 13 lines
and how would the device use this information?
Quoted 10 lines
drivers/vfio/pci/pds/cmds.c +10
-0+ +static int pds_vfio_client_adminq_cmd (struct pds_vfio_pci_device *pds_vfio, +union pds_core_adminq_cmd *req, +size_t req_len, +union pds_core_adminq_comp *resp, +u64 flags) +{ +union pds_core_adminq_cmd cmd = {}; +size_t cp_len; +int err;
Quoted 8 lines
Quoted 6 lines
drivers/vfio/pci/pds/cmds.c +5
-0+ +/* Wrap the client request */ + cmd.client_request.opcode =PDS_AQ_CMD_CLIENT_CMD ; + cmd.client_request.client_id =cpu_to_le16 (pds_vfio->client_id); + cp_len = min_t(size_t, req_len,
sizeof(cmd.client_request.client_cmd));'
req_len' is kind of redundant. Looks all the callers use sizeof(req).
It does a memcpy based on the min size between req_len and the size of the request.
Quoted 33 lines
Quoted 24 lines
drivers/vfio/pci/pds/cmds.c +20
-0+static int +pds_vfio_suspend_wait_device_cmd(struct pds_vfio_pci_device *pds_vfio) +{ + union pds_core_adminq_cmd cmd = { + .lm_suspend_status = { + .opcode = PDS_LM_CMD_SUSPEND_STATUS, + .vf_id = cpu_to_le16(pds_vfio->vf_id), + }, + }; + struct device *dev = pds_vfio_to_dev(pds_vfio); + union pds_core_adminq_comp comp = {}; + unsigned long time_limit; + unsigned long time_start; + unsigned long time_done; + int err; + + time_start = jiffies; + time_limit = time_start + HZ * SUSPEND_TIMEOUT_S; + do { + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd,sizeof(cmd),
+ &comp,PDS_AQ_FLAG_FASTPOLL);drivers/vfio/pci/pds/cmds.c void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 99
Quoted 5 lines
drivers/vfio/pci/pds/cmds.c +5
-0+if (err != -EAGAIN ) +break ; + +msleep (SUSPEND_CHECK_INTERVAL_MS ); + }while (time_before (jiffies, time_limit));
pds_vfio_client_adminq_cmd()has the exactly same mechanism with 5s timeout and 1ms poll interval when FASTPOLL is set.
Quoted 2 lines
probably you can introduce another flag to indicate retry on
-EAGAINand then handle it fully inpds_vfio_client_adminq_cmd()?
That's the entire purpose of this command. It uses pds_vfio_client_adminq_cmd() to poll the SUSPEND_STATUS. IMHO adding the polling mechanism in pds_vfio_client_adminq_cmd() and using it depending on a flag is just adding to the complexity and not offering any benefit. I plan to keep this function as is to separate the functionality. Thanks.
Quoted 41 lines
Quoted 17 lines
drivers/vfio/pci/pds/cmds.c +15
-0+int pds_vfio_suspend_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_suspend = { + .opcode =PDS_LM_CMD_SUSPEND , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; +int err; + +dev_dbg (dev,"vf%u: Suspend device\n" , pds_vfio->vf_id); + + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd),&comp,
drivers/vfio/pci/pds/cmds.c void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 138
Quoted 9 lines
drivers/vfio/pci/pds/cmds.c +9
-0+ PDS_AQ_FLAG_FASTPOLL); +if (err) { +dev_err (dev,"vf%u: Suspend failed: %pe\n" , pds_vfio->vf_id, +ERR_PTR (err)); +return err; + } + +return pds_vfio_suspend_wait_device_cmd (pds_vfio); +}The logic in this function is very confusing.
PDS_LM_CMD_SUSPENDhas a completion record:include/linux/pds/pds_adminq.h +9
-0+struct pds_lm_suspend_comp { +u8 status; +u8 rsvd; +__le16 comp_index; +union { +__le64 state_size; +u8 rsvd2[11 ]; + } __packed; +u8 color;
Quoted 4 lines
Presumably this function can look at the completion record to know whether the suspend request succeeds.
Why do you require another
wait_devicestep to query the suspend status?
The driver sends the initial suspend request to tell the DSC/firmware to suspend the VF's data/control path. The DSC/firmware will ack/nack the suspend request in the completion.
Then the driver polls the DSC/firmware to find when the VF's data/control path has been fully suspended. When the DSC/firmware isn't done suspending yet it will return -EAGAIN. Otherwise it will return success/failure.
I will add some comments clarifying these details.
Quoted 5 lines
and I have another question. Is it correct to hard-code the 5s timeout in the kernel w/o any input from the VMM? Note the guest has been stopped at this point then very likely the 5s timeout will kill any reasonable SLA which CSPs try to reach hard.
This gives the device a max of 5 seconds to suspend the VF's data/control path.
Quoted 6 lines
Ideally the VMM has an estimation how long a VM can be paused based on SLA, to-be-migrated state size, available network bandwidth, etc. and that hint should be passed to the kernel so any state transition which may violate that expectation can fail quickly to break the migration process and put the VM back to the running state.
For QEMU there is a parameter that can specify the downtime-limit that's used as you mentioned, but this does not include how long it takes the device to STOP (i.e. suspend the data/control path).
Quoted 23 lines
Jason/Shameer, is there similar concern in mlx/hisilicon drivers?
Quoted 15 lines
drivers/vfio/pci/pds/cmds.c +15
-0+ +int pds_vfio_resume_device_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +union pds_core_adminq_cmd cmd = { + .lm_resume = { + .opcode =PDS_LM_CMD_RESUME , + .vf_id =cpu_to_le16 (pds_vfio->vf_id), + }, + }; +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_adminq_comp comp = {}; + +dev_dbg (dev,"vf%u: Resume device\n" , pds_vfio->vf_id); + + return pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd),Quoted 2 lines
&comp,
+ 0);'resume' is also in the blackout phase when the guest is not running.
Quoted 2 lines
So presumably
FAST_POLLshould be set otherwise the max 256ms poll interval (PDSC_ADMINQ_MAX_POLL_INTERVAL) is really inefficient.
Yeah this is a good catch. I think setting fast_poll = true would be a good idea here. Thanks.
Quoted 38 lines
Quoted 30 lines
(quoted diff) +2
-0+ + if (cur == VFIO_DEVICE_STATE_RUNNING && next ==
VFIO_DEVICE_STATE_RUNNING_P2P) {drivers/vfio/pci/pds/lm.c new file +2
-0+ pds_vfio_send_host_vf_lm_status_cmd(pds_vfio, +
PDS_LM_STA_IN_PROGRESS);drivers/vfio/pci/pds/lm.c new file +8
-0+ err =pds_vfio_suspend_device_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + +return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==
VFIO_DEVICE_STATE_RUNNING) {drivers/vfio/pci/pds/lm.c new file +5
-0+ err =pds_vfio_resume_device_cmd (pds_vfio); +if (err) +return ERR_PTR (err); + + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_NONE);drivers/vfio/pci/pds/lm.c new file +4
-0+return NULL ; + } + + if (cur == VFIO_DEVICE_STATE_STOP && next ==
VFIO_DEVICE_STATE_RUNNING_P2P)drivers/vfio/pci/pds/lm.c new file +3
-0+return NULL ; + + if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==Quoted 2 lines
VFIO_DEVICE_STATE_STOP)
+ return NULL;I'm not sure whether P2P is actually supported here. By definition P2P means the device is stopped but still responds to p2p request from other devices. If you look at mlx example it uses different cmds between
RUNNING->RUNNING_P2PandRUNNING_P2P->STOP.
Quoted 3 lines
But in your case seems you simply move what is required in STOP into P2P. Probably you can just remove the support of P2P like hisilicon does.
In a previous review it was mentioned that P2P is more or less supported and this is how we are able to support it. Ideally we would not set the P2P feature and just implement the standard STOP/RUNNING states.
Quoted 14 lines
include/linux/pds/pds_adminq.h struct pds_core_q_init_comp { · line 628
Quoted 11 lines
include/linux/pds/pds_adminq.h +10
-0+ +/** +* struct pds_lm_comp - generic command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Structure padding to 16 Bytes +*/ +struct pds_lm_comp { +u8 status; +u8 rsvd[15 ]; +};not used. Looks most comp structures are defined w/o an user except struct
pds_lm_status_comp.
I will look into this. Thanks.
Tian, Kevin
reviewerRE: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 31 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 17, 2023 12:45 PMQuoted 1 line · Kevin
Caution: This message originated from an External Source. Use proper
caution when opening attachments, clicking links, or responding.
Quoted 7 lines
Review
Quoted 7 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AM(quoted diff) +2
-0+ +static int pds_vfio_client_adminq_cmd(struct pds_vfio_pci_device
*pds_vfio,Quoted 8 lines
Quoted 8 lines
drivers/vfio/pci/pds/cmds.c +8
-0+union pds_core_adminq_cmd *req, + size_t req_len, + union pds_core_adminq_comp *resp, + u64 flags) +{ + union pds_core_adminq_cmd cmd = {}; +size_t cp_len; +int err;Quoted 8 lines
Quoted 6 lines
drivers/vfio/pci/pds/cmds.c +5
-0+ +/* Wrap the client request */ + cmd.client_request.opcode =PDS_AQ_CMD_CLIENT_CMD ; + cmd.client_request.client_id =cpu_to_le16 (pds_vfio->client_id); + cp_len = min_t(size_t, req_len,
sizeof(cmd.client_request.client_cmd));'
req_len' is kind of redundant. Looks all the callers use sizeof(req).
Quoted 2 lines
It does a memcpy based on the min size between
req_lenand the size of the request.
If all the callers just pass in sizeof(union) as 'req_len', then it's pointless to do min_t and you can just use sizeof(cmd.client_request.client_cmd) here which is always smaller than or equal to the sizeof(union).
Quoted 37 lines
Quoted 27 lines
drivers/vfio/pci/pds/cmds.c void pds_vfio_unregister_client_cmd(struct pds_vfio_pci_device *pds_vfio) · line 138
Quoted 12 lines
(quoted diff) +2
-0+ + err = pds_vfio_client_adminq_cmd(pds_vfio, &cmd, sizeof(cmd),&comp,
drivers/vfio/pci/pds/cmds.c +9
-0+ PDS_AQ_FLAG_FASTPOLL); +if (err) { +dev_err (dev,"vf%u: Suspend failed: %pe\n" , pds_vfio->vf_id, +ERR_PTR (err)); +return err; + } + +return pds_vfio_suspend_wait_device_cmd (pds_vfio); +}The logic in this function is very confusing.
PDS_LM_CMD_SUSPENDhas a completion record:include/linux/pds/pds_adminq.h +9
-0+struct pds_lm_suspend_comp { +u8 status; +u8 rsvd; +__le16 comp_index; +union { +__le64 state_size; +u8 rsvd2[11 ]; + } __packed; +u8 color;Quoted 1 line
Presumably this function can look at the completion record to know
whether
Quoted 3 lines
the suspend request succeeds.
Why do you require another
wait_devicestep to query the suspend status?The driver sends the initial suspend request to tell the DSC/firmware to suspend the VF's data/control path. The DSC/firmware will ack/nack the suspend request in the completion.
Quoted 6 lines
Then the driver polls the DSC/firmware to find when the VF's data/control path has been fully suspended. When the DSC/firmware isn't done suspending yet it will return
-EAGAIN. Otherwise it will return success/failure.I will add some comments clarifying these details.
Yes more comment is welcomed.
It's also misleading to have a ' state_size ' field in suspend_comp. In concept the firmware cannot calculate it accurately before the VF is fully suspended.
Jason Gunthorpe
reviewerRe: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
In reply to Tian, KevinQuoted 7 lines · Kevin
Ideally the VMM has an estimation how long a VM can be paused based on SLA, to-be-migrated state size, available network bandwidth, etc. and that hint should be passed to the kernel so any state transition which may violate that expectation can fail quickly to break the migration process and put the VM back to the running state.
Jason/Shameer, is there similar concern in mlx/hisilicon drivers?
It is handled through the vfio_device_feature_mig_data_size mechanism..
Quoted 9 lines
Review
Quoted 3 lines
(quoted diff) +1
-0+ if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==
VFIO_DEVICE_STATE_STOP)(quoted diff) +1
-0+ return NULL;I'm not sure whether P2P is actually supported here. By definition P2P means the device is stopped but still responds to p2p request from other devices. If you look at mlx example it uses different cmds between
RUNNING->RUNNING_P2PandRUNNING_P2P->STOP.
Quoted 3 lines
But in your case seems you simply move what is required in STOP into P2P. Probably you can just remove the support of P2P like hisilicon does.
We want new devices to get their architecture right, they need to support P2P. Didn't we talk about this already and Brett was going to fix it?
Jason
Tian, Kevin
reviewerRE: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 8 lines
From: Jason Gunthorpe <[email protected]>
Sent: Monday, June 19, 2023 8:47 PMQuoted 3 lines · Kevin
Ideally the VMM has an estimation how long a VM can be paused based on SLA, to-be-migrated state size, available network bandwidth, etc. and that hint should be passed to the kernel so any state transition which may
Quoted 7 lines
violate
Quoted 4 lines
that expectation can fail quickly to break the migration process and put the VM back to the running state.
Jason/Shameer, is there similar concern in mlx/hisilicon drivers?
It is handled through the
vfio_device_feature_mig_data_sizemechanism..
that is only for estimation of copied data.
IMHO the stop time when the VM is paused includes both the time of stopping the device and the time of migrating the VM state.
For a software-emulated device the time of stopping the device is negligible.
But certainly for assigned device the worst-case hard-coded 5s timeout as done in this patch will kill whatever reasonable 'VM dead time' SLA (usually in milliseconds) which CSPs try to meet purely based on the size of copied data.
Wouldn't a user-specified stop-device timeout be required to at least allow breaking migration early according to the desired SLA?
Quoted 10 lines
Quoted 10 lines
Review
Quoted 4 lines
(quoted diff) +1
-0+ if (cur == VFIO_DEVICE_STATE_RUNNING_P2P && next ==
VFIO_DEVICE_STATE_STOP)(quoted diff) +1
-0+ return NULL;I'm not sure whether P2P is actually supported here. By definition P2P means the device is stopped but still responds to p2p request from other devices. If you look at mlx example it uses different cmds between
RUNNING->RUNNING_P2PandRUNNING_P2P->STOP.
Quoted 8 lines
Quoted 3 lines
But in your case seems you simply move what is required in STOP into P2P. Probably you can just remove the support of P2P like hisilicon does.
We want new devices to get their architecture right, they need to support P2P. Didn't we talk about this already and Brett was going to fix it?
Looks it's not fixed since RUNNING_P2P->STOP is a nop in this patch.
Jason Gunthorpe
reviewerRe: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 21 lines · Kevin
Quoted 8 lines
From: Jason Gunthorpe <[email protected]>
Sent: Monday, June 19, 2023 8:47 PMQuoted 3 lines · Kevin
Ideally the VMM has an estimation how long a VM can be paused based on SLA, to-be-migrated state size, available network bandwidth, etc. and that hint should be passed to the kernel so any state transition which may
Quoted 7 lines
violate
Quoted 4 lines
that expectation can fail quickly to break the migration process and put the VM back to the running state.
Jason/Shameer, is there similar concern in mlx/hisilicon drivers?
It is handled through the
vfio_device_feature_mig_data_sizemechanism..that is only for estimation of copied data.
IMHO the stop time when the VM is paused includes both the time of stopping the device and the time of migrating the VM state.
Quoted 6 lines
For a software-emulated device the time of stopping the device is negligible.
But certainly for assigned device the worst-case hard-coded 5s timeout as done in this patch will kill whatever reasonable 'VM dead time' SLA (usually in milliseconds) which CSPs try to meet purely based on the size of copied data.
There is not alot that can be done here, the stop time cannot be predicted in advance on these devices - the system relies on the device having a reasonable time window.
Quoted 2 lines
Wouldn't a user-specified stop-device timeout be required to at least allow breaking migration early according to the desired SLA?
Not really, the device is going to still execute the stop regardless of the timeout, and when it does the VM will be broken.
With a FW approach like this it is pretty stuck, we need the FW to remain in sync as the highest priority.
Quoted 5 lines
Quoted 3 lines
We want new devices to get their architecture right, they need to support P2P. Didn't we talk about this already and Brett was going to fix it?
Looks it's not fixed since
RUNNING_P2P->STOPis a nop in this patch.
That could be OK, it needs a comment explaining why it is OK
Jason
Tian, Kevin
reviewerRE: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 30 lines
From: Jason Gunthorpe <[email protected]>
Sent: Tuesday, June 20, 2023 8:31 PMQuoted 6 lines · Kevin
Quoted 6 lines
From: Jason Gunthorpe <[email protected]>
Sent: Monday, June 19, 2023 8:47 PMQuoted 1 line · Kevin
Ideally the VMM has an estimation how long a VM can be paused based
on
Quoted 1 line
Quoted 1 line
Quoted 1 line
SLA, to-be-migrated state size, available network bandwidth, etc. and
that
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
hint should be passed to the kernel so any state transition which may
violate
Quoted 1 line
that expectation can fail quickly to break the migration process and put
the
Quoted 13 lines
Quoted 5 lines
Quoted 3 lines
VM back to the running state.
Jason/Shameer, is there similar concern in mlx/hisilicon drivers?
It is handled through the
vfio_device_feature_mig_data_sizemechanism..that is only for estimation of copied data.
IMHO the stop time when the VM is paused includes both the time of stopping the device and the time of migrating the VM state.
For a software-emulated device the time of stopping the device is negligible.
Quoted 8 lines
Quoted 4 lines
But certainly for assigned device the worst-case hard-coded 5s timeout as done in this patch will kill whatever reasonable 'VM dead time' SLA (usually in milliseconds) which CSPs try to meet purely based on the size of copied data.
There is not alot that can be done here, the stop time cannot be predicted in advance on these devices - the system relies on the device having a reasonable time window.
What is the criteria for 'reasonable'? How does CSPs judge that such device can guarantee a *reliable* reasonable window so live migration can be enabled in the production environment?
I'm afraid that we are hiding a non-deterministic factor in current protocol.
Looking at mlx5 case which has a even larger timeout:
[MLX5_TO_CMD_MS] = 60000,
Quoted 9 lines
Quoted 3 lines
Wouldn't a user-specified stop-device timeout be required to at least allow breaking migration early according to the desired SLA?
Not really, the device is going to still execute the stop regardless of the timeout, and when it does the VM will be broken.
With a FW approach like this it is pretty stuck, we need the FW to remain in sync as the highest priority.
This makes some sense.
But still I don't think it's a good situation where the user has ZERO knowledge about the non-negligible time in the stopping path...
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
We want new devices to get their architecture right, they need to support P2P. Didn't we talk about this already and Brett was going to fix it?
Looks it's not fixed since
RUNNING_P2P->STOPis a nop in this patch.That could be OK, it needs a comment explaining why it is OK
Yes, a comment is welcomed. having RUNNING_P2P->STOP as nop kind of suggest that the device has been fully stopped in RUNNING_P2P to meet the definition of the STOP state. But then it violates the definition of RUNNING_P2P.
Jason Gunthorpe
reviewerRe: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 3 lines · Kevin
What is the criteria for 'reasonable'? How does CSPs judge that such device can guarantee a *reliable* reasonable window so live migration can be enabled in the production environment?
The CSP needs to work with the device vendor to understand how it fits into their system, I don't see how we can externalize this kind of detail in a general way.
Quoted 1 line
I'm afraid that we are hiding a non-deterministic factor in current protocol.
Yes
Quoted 2 lines
But still I don't think it's a good situation where the user has ZERO knowledge about the non-negligible time in the stopping path...
In any sane device design this will be a small period of time. These timeouts should be to protect against a device that has gone wild.
Jason
Tian, Kevin
reviewerRE: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 15 lines
From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, June 21, 2023 9:27 PMQuoted 3 lines · Kevin
What is the criteria for 'reasonable'? How does CSPs judge that such device can guarantee a *reliable* reasonable window so live migration can be enabled in the production environment?
The CSP needs to work with the device vendor to understand how it fits into their system, I don't see how we can externalize this kind of detail in a general way.
Quoted 1 line
I'm afraid that we are hiding a non-deterministic factor in current protocol.
Quoted 8 lines
Yes
Quoted 2 lines
But still I don't think it's a good situation where the user has ZERO knowledge about the non-negligible time in the stopping path...
In any sane device design this will be a small period of time. These timeouts should be to protect against a device that has gone wild.
Any example how 'small' it will be (e.g. <1ms)?
Should we define a *reasonable* threshold in VFIO community which any new variant driver should provide information to judge against?
If the worst-case stop time (assuming the device doesn't go wild) may exceed the threshold then it's time to consider whether a new interface is required to communicate such constraint to userspace.
The reason why I keep discussing it is that IMHO achieving negligible stop time is a very challenging task for many accelerators. e.g. IDXD can be stopped only after completing all the pending requests. While it allows software to configure the max pending work size (and a reasonable setting could meet both migration SLA and performance SLA) the worst-case draining latency could be in 10's milliseconds which cannot be ignored by the VMM.
Or do you think it's still better left to CSP working with the device vendor even in this case, given the worst-case latency could be affected by many factors hence not something which a kernel driver can accurately estimate?
Thanks
Kevin
Jason Gunthorpe
reviewerRe: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 17 lines · Kevin
Quoted 17 lines
From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, June 21, 2023 9:27 PMQuoted 3 lines · Kevin
What is the criteria for 'reasonable'? How does CSPs judge that such device can guarantee a *reliable* reasonable window so live migration can be enabled in the production environment?
The CSP needs to work with the device vendor to understand how it fits into their system, I don't see how we can externalize this kind of detail in a general way.
Quoted 1 line
I'm afraid that we are hiding a non-deterministic factor in current protocol.
Yes
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
But still I don't think it's a good situation where the user has ZERO knowledge about the non-negligible time in the stopping path...
In any sane device design this will be a small period of time. These timeouts should be to protect against a device that has gone wild.
Any example how 'small' it will be (e.g. <1ms)?
Not personally..
Quoted 2 lines
Should we define a *reasonable* threshold in VFIO community which any new variant driver should provide information to judge against?
Ah, I think we are just too new to get into such details. I think we need some real world experience to see if this is really an issue.
Quoted 7 lines
The reason why I keep discussing it is that IMHO achieving negligible stop time is a very challenging task for many accelerators. e.g. IDXD can be stopped only after completing all the pending requests. While it allows software to configure the max pending work size (and a reasonable setting could meet both migration SLA and performance SLA) the worst-case draining latency could be in 10's milliseconds which cannot be ignored by the VMM.
Well, what would you report here if you had the opportunity to report something? Some big number? Then what?
Quoted 4 lines
Or do you think it's still better left to CSP working with the device vendor even in this case, given the worst-case latency could be affected by many factors hence not something which a kernel driver can accurately estimate?
This is my fear, that it is so complicated that reducing it to any sort of cross-vendor data is not feasible. At least I'd like to see someone experiment with what information would be useful to qemu before we add kernel ABI..
Jason
Tian, Kevin
reviewerRE: [PATCH v10 vfio 4/7] vfio/pds: Add VFIO live migration support
Quoted 55 lines
From: Jason Gunthorpe <[email protected]>
Sent: Tuesday, June 27, 2023 2:14 AMQuoted 9 lines · Kevin
Quoted 9 lines
From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, June 21, 2023 9:27 PMQuoted 3 lines · Kevin
What is the criteria for 'reasonable'? How does CSPs judge that such device can guarantee a *reliable* reasonable window so live migration can be enabled in the production environment?
Quoted 5 lines
Quoted 5 lines
The CSP needs to work with the device vendor to understand how it fits into their system, I don't see how we can externalize this kind of detail in a general way.
Quoted 1 line
I'm afraid that we are hiding a non-deterministic factor in current
protocol.
Quoted 11 lines
Quoted 9 lines
Yes
Quoted 2 lines
But still I don't think it's a good situation where the user has ZERO knowledge about the non-negligible time in the stopping path...
In any sane device design this will be a small period of time. These timeouts should be to protect against a device that has gone wild.
Any example how 'small' it will be (e.g. <1ms)?
Not personally..
Quoted 2 lines
Should we define a *reasonable* threshold in VFIO community which any new variant driver should provide information to judge against?
Ah, I think we are just too new to get into such details. I think we need some real world experience to see if this is really an issue.
Quoted 7 lines
The reason why I keep discussing it is that IMHO achieving negligible stop time is a very challenging task for many accelerators. e.g. IDXD can be stopped only after completing all the pending requests. While it allows software to configure the max pending work size (and a reasonable setting could meet both migration SLA and performance SLA) the worst-case draining latency could be in 10's milliseconds which cannot be ignored by the VMM.
Well, what would you report here if you had the opportunity to report something? Some big number? Then what?
Quoted 4 lines
Or do you think it's still better left to CSP working with the device vendor even in this case, given the worst-case latency could be affected by many factors hence not something which a kernel driver can accurately estimate?
Quoted 5 lines
This is my fear, that it is so complicated that reducing it to any sort of cross-vendor data is not feasible. At least I'd like to see someone experiment with what information would be useful to qemu before we add kernel ABI..
OK. make sense.
Brett Creeley
maintainerPATCH v10 5/7 vfio/pds: Add support for dirty page tracking
In reply to Brett CreeleyIn order to support dirty page tracking, the driver has to implement the VFIO subsystem's vfio_log_ops. This includes log_start, log_stop, and log_read_and_clear.
All of the tracker resources are allocated and dirty tracking on the device is started during log_start. The resources are cleaned up and dirty tracking on the device is stopped during log_stop. The dirty pages are determined and reported during log_read_and_clear.
In order to support these callbacks admin queue commands are used. All of the adminq queue command structures and implementations are included as part of this patch.
PDS_LM_CMD_DIRTY_STATUS is added to query the current status of dirty tracking on the device. This includes if it's enabled (i.e. number of regions being tracked from the device's perspective) and the maximum number of regions supported from the device's perspective.
PDS_LM_CMD_DIRTY_ENABLE is added to enable dirty tracking on the specified number of regions and their iova ranges.
PDS_LM_CMD_DIRTY_DISABLE is added to disable dirty tracking for all regions on the device.
PDS_LM_CMD_READ_SEQ and PDS_LM_CMD_DIRTY_WRITE_ACK are added to support reading and acknowledging the currently dirtied pages.
--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 125 +++++++ drivers/vfio/pci/pds/cmds.h | 9 + drivers/vfio/pci/pds/dirty.c | 577 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/dirty.h | 38 +++ drivers/vfio/pci/pds/lm.c | 2 +- drivers/vfio/pci/pds/vfio_dev.c | 11 +- drivers/vfio/pci/pds/vfio_dev.h | 4 + include/linux/pds/pds_adminq.h | 178 ++++++++++ 9 files changed, 943 insertions(+), 2 deletions(-) create mode 100644 drivers/vfio/pci/pds/dirty.c create mode 100644 drivers/vfio/pci/pds/dirty.h
drivers/vfio/pci/pds/Makefile +1 -0
@@ -5,6 +5,7 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds_vfio.o pds_vfio-y := \ cmds.o \ + dirty.o \ lm.o \ pci_drv.o \ vfio_dev.o
drivers/vfio/pci/pds/cmds.c +125 -0
128 lines not on this page · raw message
drivers/vfio/pci/pds/cmds.h +9 -0
@@ -13,4 +13,13 @@ int pds_vfio_get_lm_state_cmd(struct pds_vfio_pci_device *pds_vfio);int pds_vfio_set_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio);void pds_vfio_send_host_vf_lm_status_cmd (struct pds_vfio_pci_device *pds_vfio,enum pds_lm_host_vf_status vf_status); +int pds_vfio_dirty_status_cmd (struct pds_vfio_pci_device *pds_vfio, +u64 regions_dma,u8 *max_regions, +u8 *num_regions); +int pds_vfio_dirty_enable_cmd (struct pds_vfio_pci_device *pds_vfio, +u64 regions_dma,u8 num_regions); +int pds_vfio_dirty_disable_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_dirty_seq_ack_cmd (struct pds_vfio_pci_device *pds_vfio, +u64 sgl_dma,u16 num_sge,u32 offset, +u32 total_len,bool read_seq);#endif /* _CMDS_H_ */
drivers/vfio/pci/pds/dirty.c new file +577 -0
577 lines not on this page · raw message
drivers/vfio/pci/pds/dirty.h new file +38 -0
@@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _DIRTY_H_ +#define _DIRTY_H_ + +struct pds_vfio_bmp_info { +unsigned long *bmp; +u32 bmp_bytes; +struct pds_lm_sg_elem *sgl; +dma_addr_t sgl_addr; +u16 num_sge; +}; + +struct pds_vfio_dirty { +struct pds_vfio_bmp_info host_seq; +struct pds_vfio_bmp_info host_ack; +u64 region_size; +u64 region_start; +u64 region_page_size; +bool is_enabled; +}; + +struct pds_vfio_pci_device ; + +bool pds_vfio_dirty_is_enabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_set_enabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_set_disabled (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_dirty_disable (struct pds_vfio_pci_device *pds_vfio); + +int pds_vfio_dma_logging_report (struct vfio_device *vdev,unsigned long iova, +unsigned long length, +struct iova_bitmap *dirty); +int pds_vfio_dma_logging_start (struct vfio_device *vdev, +struct rb_root_cached *ranges,u32 nnodes, +u64 *page_size); +int pds_vfio_dma_logging_stop (struct vfio_device *vdev); +#endif /* _DIRTY_H_ */
drivers/vfio/pci/pds/lm.c +1 -1
@@ -371,7 +371,7 @@ pds_vfio_step_device_state_locked(struct pds_vfio_pci_device *pds_vfio,if (cur ==VFIO_DEVICE_STATE_STOP_COPY && next ==VFIO_DEVICE_STATE_STOP ) {pds_vfio_put_save_file (pds_vfio); -pds_vfio_send_host_vf_lm_status_cmd (pds_vfio,PDS_LM_STA_NONE ); +pds_vfio_dirty_disable (pds_vfio);return NULL ; }
drivers/vfio/pci/pds/vfio_dev.c +10 -1
@@ -5,6 +5,7 @@#include <linux/vfio_pci_core.h> #include "lm.h" +#include "dirty.h" #include "vfio_dev.h" struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio)
@@ -25,7 +26,7 @@ struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev) vfio_coredev); } -static void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) +void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) { again:spin_lock (&pds_vfio->reset_lock);
@@ -117,6 +118,12 @@ static const struct vfio_migration_ops pds_vfio_lm_ops = { .migration_get_data_size = pds_vfio_get_device_state_size }; +static const struct vfio_log_ops pds_vfio_log_ops = { + .log_start = pds_vfio_dma_logging_start, + .log_stop = pds_vfio_dma_logging_stop, + .log_read_and_clear = pds_vfio_dma_logging_report, +}; +static int pds_vfio_init_device (struct vfio_device *vdev) { struct pds_vfio_pci_device *pds_vfio =
@@ -134,6 +141,7 @@ static int pds_vfio_init_device(struct vfio_device *vdev) vdev->migration_flags =VFIO_MIGRATION_STOP_COPY |VFIO_MIGRATION_P2P ; vdev->mig_ops = &pds_vfio_lm_ops; + vdev->log_ops = &pds_vfio_log_ops; dev_dbg(&pdev->dev, "%s: PF %#04x VF %#04x (%d) vf_id %d domain %d pds_vfio %p\n",
@@ -172,6 +180,7 @@ static void pds_vfio_close_device(struct vfio_device *vdev)mutex_lock (&pds_vfio->state_mutex);pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); +pds_vfio_dirty_disable (pds_vfio);mutex_unlock (&pds_vfio->state_mutex);mutex_destroy (&pds_vfio->state_mutex);vfio_pci_core_close_device (vdev);
drivers/vfio/pci/pds/vfio_dev.h +4 -0
@@ -7,6 +7,7 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +#include "dirty.h" #include "lm.h" struct pdsc ;
@@ -17,6 +18,7 @@ struct pds_vfio_pci_device {struct pds_vfio_lm_file *save_file;struct pds_vfio_lm_file *restore_file; +struct pds_vfio_dirty dirty;struct mutex state_mutex;/* protect migration state */ enum vfio_device_mig_state state;spinlock_t reset_lock;/* protect reset_done flow */
@@ -27,6 +29,8 @@ struct pds_vfio_pci_device {u16 client_id; }; +void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio); +const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev);void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio);
include/linux/pds/pds_adminq.h +178 -0
202 lines not on this page · raw message
Signature · 1 line
2.17.1
Brett Creeley
maintainerPATCH v10 6/7 vfio/pds: Add support for firmware recovery
In reply to Brett CreeleyIt's possible that the device firmware crashes and is able to recover due to some configuration and/or other issue. If a live migration is in progress while the firmware crashes, the live migration will fail. However, the VF PCI device should still be functional post crash recovery and subsequent migrations should go through as expected.
When the pds_core device notices that firmware crashes it sends an event to all its client drivers. When the pds_vfio driver receives this event while migration is in progress it will request a deferred reset on the next migration state transition. This state transition will report failure as well as any subsequent state transition requests from the VMM/VFIO. Based on uapi/vfio.h the only way out of VFIO_DEVICE_STATE_ERROR is by issuing VFIO_DEVICE_RESET. Once this reset is done, the migration state will be reset to VFIO_DEVICE_STATE_RUNNING and migration can be performed.
If the event is received while no migration is in progress (i.e. the VM is in normal operating mode), then no actions are taken and the migration state remains VFIO_DEVICE_STATE_RUNNING.
--- drivers/vfio/pci/pds/pci_drv.c | 105 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.c | 28 ++++++++- drivers/vfio/pci/pds/vfio_dev.h | 4 ++ 3 files changed, 135 insertions(+), 2 deletions(-)
drivers/vfio/pci/pds/pci_drv.c +105 -0
@@ -19,6 +19,104 @@#define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO Device Driver"#define PCI_VENDOR_ID_PENSANDO 0x1dd8 +static void pds_vfio_recovery (struct pds_vfio_pci_device *pds_vfio) +{ +bool deferred_reset_needed = false; + +/* +* Documentation states that the kernel migration driver must not +* generate asynchronous device state transitions outside of +* manipulation by the user or the VFIO_DEVICE_RESET ioctl. +* +* Since recovery is an asynchronous event received from the device, +* initiate a deferred reset. Only issue the deferred reset if a +* migration is in progress, which will cause the next step of the +* migration to fail. Also, if the device is in a state that will +* be set to VFIO_DEVICE_STATE_RUNNING on the next action (i.e. VM is +* shutdown and device is in VFIO_DEVICE_STATE_STOP) as that will clear +* the VFIO_DEVICE_STATE_ERROR when the VM starts back up. +*/ +mutex_lock (&pds_vfio->state_mutex); +if ((pds_vfio->state !=VFIO_DEVICE_STATE_RUNNING && + pds_vfio->state !=VFIO_DEVICE_STATE_ERROR ) || + (pds_vfio->state ==VFIO_DEVICE_STATE_RUNNING && +pds_vfio_dirty_is_enabled (pds_vfio))) + deferred_reset_needed = true; +mutex_unlock (&pds_vfio->state_mutex); + +/* +* On the next user initiated state transition, the device will +* transition to the VFIO_DEVICE_STATE_ERROR. At this point it's the user's +* responsibility to reset the device. +* +* If a VFIO_DEVICE_RESET is requested post recovery and before the next +* state transition, then the deferred reset state will be set to +* VFIO_DEVICE_STATE_RUNNING. +*/ +if (deferred_reset_needed) +pds_vfio_deferred_reset (pds_vfio,VFIO_DEVICE_STATE_ERROR ); +} + +static int pds_vfio_pci_notify_handler (struct notifier_block *nb, +unsigned long ecode,void *data) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(nb, struct pds_vfio_pci_device, nb); +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_notifyq_comp *event = data; + +dev_dbg (dev,"%s: event code %lu\n" , __func__, ecode); + +/* +* We don't need to do anything for RESET state==0 as there is no notify +* or feedback mechanism available, and it is possible that we won't +* even see a state==0 event. +* +* Any requests from VFIO while state==0 will fail, which will return +* error and may cause migration to fail. +*/ +if (ecode ==PDS_EVENT_RESET ) { +dev_info (dev,"%s: PDS_EVENT_RESET event received, state==%d\n" , + __func__, event->reset.state); +if (event->reset.state ==1 ) +pds_vfio_recovery (pds_vfio); + } + +return 0 ; +} + +static int +pds_vfio_pci_register_event_handler (struct pds_vfio_pci_device *pds_vfio) +{ +struct device *dev =pds_vfio_to_dev (pds_vfio); +struct notifier_block *nb = &pds_vfio->nb; +int err; + +if (!nb->notifier_call) { + nb->notifier_call = pds_vfio_pci_notify_handler; + err =pdsc_register_notify (nb); +if (err) { + nb->notifier_call =NULL ; +dev_err (dev, +"failed to register pds event handler: %pe\n" , +ERR_PTR (err)); +return -EINVAL ; + } +dev_dbg (dev,"pds event handler registered\n" ); + } + +return 0 ; +} + +static void +pds_vfio_pci_unregister_event_handler (struct pds_vfio_pci_device *pds_vfio) +{ +if (pds_vfio->nb.notifier_call) { +pdsc_unregister_notify (&pds_vfio->nb); + pds_vfio->nb.notifier_call =NULL ; + } +} +static int pds_vfio_pci_probe (struct pci_dev *pdev,const struct pci_device_id *id) {
@@ -48,8 +146,14 @@ static int pds_vfio_pci_probe(struct pci_dev *pdev, goto out_unregister_coredev; } + err =pds_vfio_pci_register_event_handler (pds_vfio); +if (err) + goto out_unregister_client; +return 0 ; +out_unregister_client: +pds_vfio_unregister_client_cmd (pds_vfio); out_unregister_coredev:vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); out_put_vdev:
@@ -61,6 +165,7 @@ static void pds_vfio_pci_remove(struct pci_dev *pdev) {struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); +pds_vfio_pci_unregister_event_handler (pds_vfio);pds_vfio_unregister_client_cmd (pds_vfio);vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev);vfio_put_device (&pds_vfio->vfio_coredev.vdev);
drivers/vfio/pci/pds/vfio_dev.c +26 -2
@@ -33,10 +33,13 @@ void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio)if (pds_vfio->deferred_reset) { pds_vfio->deferred_reset = false;if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) { - pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; + pds_vfio->state = pds_vfio->deferred_reset_state;pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); + }else if (pds_vfio->deferred_reset_state ==VFIO_DEVICE_STATE_ERROR ) { + pds_vfio->state =VFIO_DEVICE_STATE_ERROR ; } + pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;spin_unlock (&pds_vfio->reset_lock); goto again; }
@@ -48,6 +51,7 @@ void pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio) {spin_lock (&pds_vfio->reset_lock); pds_vfio->deferred_reset = true; + pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;if (!mutex_trylock (&pds_vfio->state_mutex)) {spin_unlock (&pds_vfio->reset_lock);return ;
@@ -56,6 +60,15 @@ void pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio)pds_vfio_state_mutex_unlock (pds_vfio); } +void pds_vfio_deferred_reset (struct pds_vfio_pci_device *pds_vfio, +enum vfio_device_mig_state reset_state) +{ +spin_lock (&pds_vfio->reset_lock); + pds_vfio->deferred_reset = true; + pds_vfio->deferred_reset_state = reset_state; +spin_unlock (&pds_vfio->reset_lock); +} +static struct file *pds_vfio_set_device_state (struct vfio_device *vdev,enum vfio_device_mig_state new_state)
@@ -66,7 +79,14 @@ pds_vfio_set_device_state(struct vfio_device *vdev,struct file *res =NULL ;mutex_lock (&pds_vfio->state_mutex); -while (new_state != pds_vfio->state) { +/* +* only way to transition out of VFIO_DEVICE_STATE_ERROR is via +* VFIO_DEVICE_RESET, so prevent the state machine from running since +* vfio_mig_get_next_state() will throw a WARN_ON() when transitioning +* from VFIO_DEVICE_STATE_ERROR to any other state +*/ +while (pds_vfio->state !=VFIO_DEVICE_STATE_ERROR && + new_state != pds_vfio->state) {enum vfio_device_mig_state next_state; int err = vfio_mig_get_next_state(vdev, pds_vfio->state,
@@ -88,6 +108,9 @@ pds_vfio_set_device_state(struct vfio_device *vdev, } }pds_vfio_state_mutex_unlock (pds_vfio); +/* still waiting on a deferred_reset */ +if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) + res =ERR_PTR (-EIO );return res; }
@@ -165,6 +188,7 @@ static int pds_vfio_open_device(struct vfio_device *vdev)mutex_init (&pds_vfio->state_mutex); pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; + pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;vfio_pci_core_finish_enable (&pds_vfio->vfio_coredev);
drivers/vfio/pci/pds/vfio_dev.h +4 -0
@@ -23,6 +23,8 @@ struct pds_vfio_pci_device {enum vfio_device_mig_state state;spinlock_t reset_lock;/* protect reset_done flow */ u8 deferred_reset; +enum vfio_device_mig_state deferred_reset_state; +struct notifier_block nb;int vf_id;int pci_id;
@@ -34,6 +36,8 @@ void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio);const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev);void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_deferred_reset (struct pds_vfio_pci_device *pds_vfio, +enum vfio_device_mig_state reset_state);struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio);struct device *pds_vfio_to_dev (struct pds_vfio_pci_device *pds_vfio);
Signature · 1 line
2.17.1
Tian, Kevin
reviewerRE: [PATCH v10 vfio 6/7] vfio/pds: Add support for firmware recovery
Quoted 20 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMdrivers/vfio/pci/pds/pci_drv.c +14
-0+static void pds_vfio_recovery (struct pds_vfio_pci_device *pds_vfio) +{ +bool deferred_reset_needed = false; + +/* +* Documentation states that the kernel migration driver must not +* generate asynchronous device state transitions outside of +* manipulation by the user or the VFIO_DEVICE_RESET ioctl. +* +* Since recovery is an asynchronous event received from the device, +* initiate a deferred reset. Only issue the deferred reset if a +* migration is in progress, which will cause the next step of the +* migration to fail. Also, if the device is in a state that will +* be set to VFIO_DEVICE_STATE_RUNNING on the next action (i.e. VM is
(quoted diff) +1
-0+ * shutdown and device is in VFIO_DEVICE_STATE_STOP) as that willclear
drivers/vfio/pci/pds/pci_drv.c line 37
Quoted 1 line
drivers/vfio/pci/pds/pci_drv.c +1
-0+* the VFIO_DEVICE_STATE_ERROR when the VM starts back up.
the last sentence after "Also, ..." is incomplete?
drivers/vfio/pci/pds/pci_drv.c line 44
Quoted 7 lines
drivers/vfio/pci/pds/pci_drv.c +7
-0+*/ +mutex_lock (&pds_vfio->state_mutex); +if ((pds_vfio->state !=VFIO_DEVICE_STATE_RUNNING && + pds_vfio->state !=VFIO_DEVICE_STATE_ERROR ) || + (pds_vfio->state ==VFIO_DEVICE_STATE_RUNNING && +pds_vfio_dirty_is_enabled (pds_vfio))) + deferred_reset_needed = true;
any unwind to be done in the dirty tracking path? When firmware crashes presumably the cmd to retrieve dirty pages is also blocked...
Quoted 15 lines
drivers/vfio/pci/pds/pci_drv.c +5
-0+mutex_unlock (&pds_vfio->state_mutex); + +/* +* On the next user initiated state transition, the device will +* transition to the VFIO_DEVICE_STATE_ERROR. At this point it's the user's
drivers/vfio/pci/pds/pci_drv.c +3
-0+* responsibility to reset the device. +* +* If a VFIO_DEVICE_RESET is requested post recovery and before the next
drivers/vfio/pci/pds/pci_drv.c +5
-0+* state transition, then the deferred reset state will be set to +* VFIO_DEVICE_STATE_RUNNING. +*/ +if (deferred_reset_needed) + pds_vfio_deferred_reset(pds_vfio,
Quoted 1 line
VFIO_DEVICE_STATE_ERROR);
open-code as here is the only caller.
Quoted 24 lines
drivers/vfio/pci/pds/pci_drv.c +14
-0+} + +static int pds_vfio_pci_notify_handler (struct notifier_block *nb, +unsigned long ecode,void *data) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(nb, struct pds_vfio_pci_device, nb); +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_notifyq_comp *event = data; + +dev_dbg (dev,"%s: event code %lu\n" , __func__, ecode); + +/* +* We don't need to do anything for RESET state==0 as there is no notify
drivers/vfio/pci/pds/pci_drv.c +8
-0+* or feedback mechanism available, and it is possible that we won't +* even see a state==0 event. +* +* Any requests from VFIO while state==0 will fail, which will return +* error and may cause migration to fail. +*/ +if (ecode ==PDS_EVENT_RESET ) { + dev_info(dev, "%s: PDS_EVENT_RESET event received,state==%d\n",
drivers/vfio/pci/pds/pci_drv.c line 83
Quoted 4 lines
drivers/vfio/pci/pds/pci_drv.c +4
-0+ __func__, event->reset.state); +if (event->reset.state ==1 ) +pds_vfio_recovery (pds_vfio); + }
Please explain what state==0 is, and why state==1 is handled while state==2 is not.
Quoted 5 lines
drivers/vfio/pci/pds/vfio_dev.c +0
-0@@ -33,10 +33,13 @@ void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio)if (pds_vfio->deferred_reset) { pds_vfio->deferred_reset = false;if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) {
drivers/vfio/pci/pds/vfio_dev.c void pds_vfio_state_mutex_unlock(struct · line 45
Quoted 10 lines
drivers/vfio/pci/pds/vfio_dev.c +4
-1- pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; + pds_vfio->state = pds_vfio->deferred_reset_state;pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); + }else if (pds_vfio->deferred_reset_state ==VFIO_DEVICE_STATE_ERROR ) { + pds_vfio->state =VFIO_DEVICE_STATE_ERROR ; } + pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;
this is not required. 'deferred_reset_state' should be set only when deferred_reset is true. Currently only in the notify path and reset path.
So the last assignment is pointless.
It's simpler to be:
if (pds_vfio->deferred_reset) {
pds_vfio->deferred_reset = false;
if (pds_vfio->state == VFIO_DEVICE_STATE_ERROR) {
pds_vfio_put_restore_file(pds_vfio);
pds_vfio_put_save_file(pds_vfio);
}
pds_vfio->state = pds_vfio->deferred_reset_state;
...
}Re: [PATCH v10 vfio 6/7] vfio/pds: Add support for firmware recovery
Quoted 18 lines · Kevin
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 15 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMdrivers/vfio/pci/pds/pci_drv.c +12
-0+static void pds_vfio_recovery (struct pds_vfio_pci_device *pds_vfio) +{ +bool deferred_reset_needed = false; + +/* +* Documentation states that the kernel migration driver must not +* generate asynchronous device state transitions outside of +* manipulation by the user or the VFIO_DEVICE_RESET ioctl. +* +* Since recovery is an asynchronous event received from the device, +* initiate a deferred reset. Only issue the deferred reset if a +* migration is in progress, which will cause the next step of the
Quoted 8 lines
Quoted 6 lines
(quoted diff) +2
-0+ * migration to fail. Also, if the device is in a state that will + * be set to VFIO_DEVICE_STATE_RUNNING on the next action (i.e.VM is
+ * shutdown and device is inVFIO_DEVICE_STATE_STOP) as that will
clear
+ * theVFIO_DEVICE_STATE_ERRORwhen the VM starts back up.the last sentence after "Also, ..." is incomplete?
Yeah, not sure what happened there. Will fix. Thanks.
Quoted 11 lines
drivers/vfio/pci/pds/pci_drv.c line 44
Quoted 8 lines
drivers/vfio/pci/pds/pci_drv.c +7
-0+*/ +mutex_lock (&pds_vfio->state_mutex); +if ((pds_vfio->state !=VFIO_DEVICE_STATE_RUNNING && + pds_vfio->state !=VFIO_DEVICE_STATE_ERROR ) || + (pds_vfio->state ==VFIO_DEVICE_STATE_RUNNING && +pds_vfio_dirty_is_enabled (pds_vfio))) + deferred_reset_needed = true;any unwind to be done in the dirty tracking path? When firmware crashes presumably the cmd to retrieve dirty pages is also blocked...
Hmm. I'll double check this. Thanks.
Quoted 43 lines
Quoted 16 lines
drivers/vfio/pci/pds/pci_drv.c +5
-0+mutex_unlock (&pds_vfio->state_mutex); + +/* +* On the next user initiated state transition, the device will +* transition to the VFIO_DEVICE_STATE_ERROR. At this point it's the user's
drivers/vfio/pci/pds/pci_drv.c +3
-0+* responsibility to reset the device. +* +* If a VFIO_DEVICE_RESET is requested post recovery and before the next
drivers/vfio/pci/pds/pci_drv.c +5
-0+* state transition, then the deferred reset state will be set to +* VFIO_DEVICE_STATE_RUNNING. +*/ +if (deferred_reset_needed) + pds_vfio_deferred_reset(pds_vfio,Quoted 1 line
VFIO_DEVICE_STATE_ERROR);open-code as here is the only caller.
Quoted 23 lines
drivers/vfio/pci/pds/pci_drv.c +14
-0+} + +static int pds_vfio_pci_notify_handler (struct notifier_block *nb, +unsigned long ecode,void *data) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(nb, struct pds_vfio_pci_device, nb); +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_notifyq_comp *event = data; + +dev_dbg (dev,"%s: event code %lu\n" , __func__, ecode); + +/* +* We don't need to do anything for RESET state==0 as there is no notify
drivers/vfio/pci/pds/pci_drv.c +8
-0+* or feedback mechanism available, and it is possible that we won't +* even see a state==0 event. +* +* Any requests from VFIO while state==0 will fail, which will return +* error and may cause migration to fail. +*/ +if (ecode ==PDS_EVENT_RESET ) { + dev_info(dev, "%s: PDS_EVENT_RESET event received,
Quoted 8 lines
drivers/vfio/pci/pds/pci_drv.c line 83
Quoted 5 lines
state==%d\n",
drivers/vfio/pci/pds/pci_drv.c +4
-0+ __func__, event->reset.state); +if (event->reset.state ==1 ) +pds_vfio_recovery (pds_vfio); + }Please explain what state==0 is, and why state==1 is handled while state==2 is not.
Sure, will clarify. Thanks.
Quoted 24 lines
Quoted 6 lines
drivers/vfio/pci/pds/vfio_dev.c +0
-0@@ -33,10 +33,13 @@ void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio)if (pds_vfio->deferred_reset) { pds_vfio->deferred_reset = false;if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) {drivers/vfio/pci/pds/vfio_dev.c void pds_vfio_state_mutex_unlock(struct · line 45
Quoted 10 lines
drivers/vfio/pci/pds/vfio_dev.c +4
-1- pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; + pds_vfio->state = pds_vfio->deferred_reset_state;pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); + }else if (pds_vfio->deferred_reset_state ==VFIO_DEVICE_STATE_ERROR ) { + pds_vfio->state =VFIO_DEVICE_STATE_ERROR ; } + pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;this is not required. '
deferred_reset_state' should be set only whendeferred_resetis true. Currently only in the notify path and reset path.So the last assignment is pointless.
It's simpler to be:
Quoted 9 lines
if (pds_vfio->deferred_reset) {
pds_vfio->deferred_reset = false;
if (pds_vfio->state == VFIO_DEVICE_STATE_ERROR) {
pds_vfio_put_restore_file(pds_vfio);
pds_vfio_put_save_file(pds_vfio);
}
pds_vfio->state = pds_vfio->deferred_reset_state;
...
}I think that makes sense. I will take another look and fix/improve this on the next version.
Quoted 1 line
Brett Creeley
maintainerPATCH v10 7/7 vfio/pds: Add Kconfig and documentation
In reply to Brett CreeleyAdd Kconfig entries and pds_vfio.rst. Also, add an entry in the MAINTAINERS file for this new driver.
It's not clear where documentation for vendor specific VFIO drivers should live, so just re-use the current amd ethernet location.
--- .../device_drivers/ethernet/amd/pds_vfio.rst | 79 +++++++++++++++++++ .../device_drivers/ethernet/index.rst | 1 + MAINTAINERS | 7 ++ drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/pds/Kconfig | 20 +++++ 5 files changed, 109 insertions(+) create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst create mode 100644 drivers/vfio/pci/pds/Kconfig
Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst new file +79 -0
@@ -0,0 +1,79 @@ +.. SPDX-License-Identifier: GPL-2.0+ +.. note: can be edited and viewed with /usr/bin/formiko-vim + +========================================================== +PCI VFIO driver for the AMD/Pensando(R) DSC adapter family +========================================================== + +AMD/Pensando Linux VFIO PCI Device Driver +Copyright(c) 2023 Advanced Micro Devices, Inc. + +Overview +======== + +The ``pds_vfio`` module is a PCI driver that supports Live Migration +capable Virtual Function (VF) devices in the DSC hardware. + +Using the device +================ + +The pds_vfio device is enabled via multiple configuration steps and +depends on the ``pds_core`` driver to create and enable SR-IOV Virtual +Function devices. + +Shown below are the steps to bind the driver to a VF and also to the +associated auxiliary device created by the ``pds_core`` driver. This +example assumes the pds_core and pds_vfio modules are already +loaded. + +.. code-block:: bash + :name: example-setup-script + + #!/bin/bash + + PF_BUS="0000:60" + PF_BDF="0000:60:00.0" + VF_BDF="0000:60:00.1" + + # Prevent non-vfio VF driver from probing the VF device + echo 0 > /sys/class/pci_bus/$PF_BUS/device/$PF_BDF/sriov_drivers_autoprobe + + # Create single VF for Live Migration via VFIO + echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs + + # Allow the VF to be bound to the pds_vfio driver + echo "pds_vfio" > /sys/class/pci_bus/$PF_BUS/device/$VF_BDF/driver_override + + # Bind the VF to the pds_vfio driver + echo "$VF_BDF" > /sys/bus/pci/drivers/pds_vfio/bind + +After performing the steps above, a file in /dev/vfio/<iommu_group> +should have been created. + + +Enabling the driver +=================== + +The driver is enabled via the standard kernel configuration system, +using the make command:: + + make oldconfig/menuconfig/etc. + +The driver is located in the menu structure at: + + -> Device Drivers + -> VFIO Non-Privileged userspace driver framework + -> VFIO support for PDS PCI devices + +Support +======= + +For general Linux networking support, please use the netdev mailing +list, which is monitored by Pensando personnel:: + + [email protected] + +For more specific support needs, please use the Pensando driver support +email:: + + [email protected]
Documentation/networking/device_drivers/ethernet/index.rst +1 -0
@@ -15,6 +15,7 @@ Contents: amazon/ena altera/altera_tse amd/pds_core + amd/pds_vfio aquantia/atlantic chelsio/cxgb cirrus/cs89x0
MAINTAINERS +7 -0
@@ -22140,6 +22140,13 @@ S: Maintained P: Documentation/driver-api/vfio-pci-device-specific-driver-acceptance.rst F: drivers/vfio/pci/*/ +VFIO PDS PCI DRIVER +M: Brett Creeley <[email protected]> +L: [email protected] +S: Maintained +F: Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst +F: drivers/vfio/pci/pds/ + VFIO PLATFORM DRIVER M: Eric Auger <[email protected]> L: [email protected]
drivers/vfio/pci/Kconfig +2 -0
@@ -59,4 +59,6 @@ source "drivers/vfio/pci/mlx5/Kconfig" source "drivers/vfio/pci/hisilicon/Kconfig" +source "drivers/vfio/pci/pds/Kconfig" + endif
drivers/vfio/pci/pds/Kconfig new file +20 -0
@@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Advanced Micro Devices, Inc. + +config PDS_VFIO_PCI + tristate "VFIO support for PDS PCI devices" + depends on PDS_CORE + depends on VFIO_PCI_CORE + help + This provides generic PCI support for PDS devices using the VFIO + framework. + + More specific information on this driver can be + found in + <file:Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst>. + + To compile this driver as a module, choose M here. The module + will be called pds_vfio. + + If you don't know what to do here, say N. +
Signature · 1 line
2.17.1
Tian, Kevin
reviewerRE: [PATCH v10 vfio 7/7] vfio/pds: Add Kconfig and documentation
Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst line 42
Quoted 9 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMDocumentation/networking/device_drivers/ethernet/amd/pds_vfio.rst new file +3
-0+ + # Prevent non-vfio VF driver from probing the VF device + echo 0 >
/sys/class/pci_bus/$PF_BUS/device/$PF_BDF/sriov_drivers_autoprobeDocumentation/networking/device_drivers/ethernet/amd/pds_vfio.rst new file +3
-0+ + # Create single VF for Live Migration via VFIO + echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs
s/via VFIO/via pds_core/
drivers/vfio/pci/pds/Kconfig line 7
Quoted 5 lines
drivers/vfio/pci/pds/Kconfig new file +5
-0+ +config PDS_VFIO_PCI + tristate "VFIO support for PDS PCI devices" + depends on PDS_CORE + depends on VFIO_PCI_CORE
this should be rebased on Alex's Kconfig change.
Re: [PATCH v10 vfio 7/7] vfio/pds: Add Kconfig and documentation
Quoted 15 lines · Kevin
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst line 42
Quoted 9 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMDocumentation/networking/device_drivers/ethernet/amd/pds_vfio.rst new file +3
-0+ + # Prevent non-vfio VF driver from probing the VF device + echo 0 >
/sys/class/pci_bus/$PF_BUS/device/$PF_BDF/sriov_drivers_autoprobeDocumentation/networking/device_drivers/ethernet/amd/pds_vfio.rst new file +3
-0+ + # Create single VF for Live Migration via VFIO + echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfss/via VFIO/via pds_core/
Quoted 7 lines
drivers/vfio/pci/pds/Kconfig line 7
Quoted 5 lines
drivers/vfio/pci/pds/Kconfig new file +5
-0+ +config PDS_VFIO_PCI + tristate "VFIO support for PDS PCI devices" + depends on PDS_CORE + depends on VFIO_PCI_COREthis should be rebased on Alex's
Kconfigchange.
I will fix these issues, thanks for the review.
Brett
[sorry, not-sorry for the top post]
Thanks Jason and Shameer for prior review comments, I hope you both can find time to check v10 as well.
Others that previously stepped up to be reviewers for new vfio-pci variant drivers, please jump in. Thanks,
Alex
Quoted 227 lines · On Fri, 2 Jun 2023 15:03:11 -0700 Brett Creeley <[email protected]> wrote:
This is a patchset for a new vendor specific VFIO driver (
pds_vfio) for use with the AMD/Pensando Distributed Services Card (DSC). This driver makes use of thepds_coredriver.This driver will use the
pds_coredevice's adminq as the VFIO control path to the DSC. In order to make adminq calls, the VFIO instance makes use of functions exported by thepds_coredriver.In order to receive events from
pds_core, thepds_vfiodriver registers to a private notifier. This is needed for various events that come from the device.An ASCII diagram of a VFIO instance looks something like this and can be used with the VFIO subsystem to provide the VF device VFIO and live migration support.
.------. .-----------------------. | QEMU |--| VM .-------------. | '......' | | Eth VF | | | | .-------------. | | | | SR-IOV VF | | | | '-------------' | | '------------||---------' .--------------. || |/dev/<vfio_fd>| || '--------------' || Host Userspace | || =================================================== || Host Kernel | || .--------. || |vfio-pci| || '--------' || .------------------. || || | | exported API |<----+ || || | '--------------| | || || | | .-------------. || | pds_core |--->| pds_vfio | || '------------------' | '-------------' || || | || || 09:00.0 notifier 09:00.1 || == PCI ===============================================||===== || || || .----------. .----------. || ,-----| PF |----------| VF |-------------------, | '----------' '----------' | VF | | DSC | data/control | | | path | -----------------------------------------------------------The
pds_vfiodriver is targeted to reside indrivers/vfio/pci/pds. It makes use of and introduces new files in the commoninclude/linux/pdsinclude directory.Changes:
v10:
- Various fixes/suggestions by Jason Gunthorpe
- Simplifypds_vfio_get_lm_file()based onfpga_mgr_buf_load()
- Clean-ups/fixes based on clang-format
- Remove any double goto labels
- Name goto labels baesed on what needs to be cleaned/freed instead of a "call from" scheme
- Fix any goto unwind ordering issues
- Make sure calldma_map_single()after data is written to memory inpds_vfio_dma_map_lm_file()
- Don't usebitmap_zalloc()for the dirty bitmaps
- Usevzalloc()for dirty bitmaps and refactor how the bitmaps are DMA'd to and from the device inpds_vfio_dirty_seq_ack()
- Remove unnecessary goto inpds_vfio_dirty_disable()v9:
https://lore.kernel.org/netdev/[email protected]/
- Various fixes/suggestions by Alex Williamson
- Fix how ID is generated in client registration
- Add helper functions to get the VF's struct device and structpci_devpointers instead of caching the struct pci dev
- Remove redundantpds_vfio_lm_state()function and remove any places this was being called
- Fix multi-line comments to follow standard convention
- Remove confusing comments inpds_vfio_step_device_state_locked()since the driver's migration states align with the VFIO documentation
- Validate pdsc returned frompdsc_get_pf_struct()
- Various fixes/suggestions by Jason Gunthorpe
- Use struct pdsc instead of void *
- Use {} instead of {0} for structure initialization
- Use unions on the stack instead of casting to the union when sending AQ commands, which required includingpds_lm.hinpds_adminq.h
- Replace use ofdma_alloc_coherent()when creating the sgl DMA entries for the LM file
- Remove cached struct device *coredev and instead usepci_physfn()to get the pds_core's struct device pointer
- Drop the recovery work item and callpds_vfio_recovery()directly from the notifier callback
- Remove unnecessary #define for "pds_vfio_lm" and just use the string inline to theanon_inode_getfile()argument
- Fix LM file reference counting
- Move initialization of some struct members to when the struct is being initialized for AQ commands
- Make use ofGFP_KERNEL_ACCOUNTwhere it makes sense
- ReplacePDS_VFIO_DRV_NAMEwithKBUILD_MODNAME
- Update to latestpds_coreexported functions
- Remove duplicated prototypes for pds_vfio_dma_logging_[start|stop|report] fromlm.h
- Holdpds_vfio->state_mutexwhile starting, stopping, and reporting dirty page tracking in pds_vfio_dma_logging_[start|stop|report]
- Remove duplicatePDS_DEV_TYPE_LM_STRdefine frompds_lm.hthat's already included inpds_common.h
- Replace use ofdma_alloc_coherent()when creating the sgl DMA entries for the dirty bitmapsv8:
https://lore.kernel.org/netdev/[email protected]/
- provide default iommufd callbacks forbind_iommufd,unbind_iommufd, andattach_ioasfor the VFIO device as suggested by Shameerali Kolothum Thodiv7:
https://lore.kernel.org/netdev/[email protected]/
- Disable and clean up dirty page tracking when the VFIO device is closed
- Various improvements suggested by Simon Horman:
- Fix RCT invfio_combine_iova_ranges()
- Simplify function exit paths by removing unnecessary goto labels
- Cleanuppds_vifo_print_guest_region_info()by adding a goto label for freeing memory, which allowed for reduced indentation on a for loop
- Where possible use C99 style for loopsv6:
https://lore.kernel.org/netdev/[email protected]/
- As suggested by Alex Williamson, usepci_domain_nr()macro to make sure thepds_vfioclient's devname is unique
- Remove unnecessary forward declaration and include
- Fix copyright comment to use correct company name
- Remove "." from struct documentation for consistencyv5:
https://lore.kernel.org/netdev/[email protected]/
- Fix SPDX comments in .h files
- Remove adminqcq argument frompdsc_post_adminq()uses
- Unregister client onvfio_pci_core_register_device()failure
- Other minor checkpatch issuesv4:
https://lore.kernel.org/netdev/[email protected]/
- Update cover letter ASCII diagram to reflect new driver architecture
- Remove auxiliary driver implementation
- Use pds_core's exported functions to communicate with the device
- Implement and register notifier for events from the device/pds_core
- Usemodule_pci_driver()macro since auxiliary driver configuration is no longer needed in __init/__exitv3:
https://lore.kernel.org/netdev/[email protected]/
- Update copyright year to 2023 and use "Advanced Micro Devices, Inc." for the company name
- Clarify the fact that AMD/Pensando's VFIO solution is device type agnostic, which aligns with other current VFIO solutions
- Add line indrivers/vfio/pci/Makefileto buildpds_vfio
- Move documentation to amd sub-directory
- Remove some dead code due to thepds_coreimplementation of listening to BIND/UNBIND events
- Move adev_dbg()to a previous patch in the series
- Add implementation for vfio_migration_ops.migration_get_data_size to return the maximum possible device state sizeRFC to v2:
https://lore.kernel.org/all/[email protected]/
- Implement state transitions forVFIO_MIGRATION_P2Pflag
- Improve auxiliary driver probe by returningEPROBE_DEFERwhen the PCI driver is not set up correctly
- Add pointer to docs inDocumentation/networking/device_drivers/ethernet/index.rstRFC:
https://lore.kernel.org/all/[email protected]/Brett Creeley (7):
vfio: Commonizecombine_rangesfor use in other VFIO drivers vfio/pds: Initial support forpds_vfioVFIO driver vfio/pds: register with thepds_corePF
vfio/pds: Add VFIO live migration support vfio/pds: Add support for dirty page tracking vfio/pds: Add support for firmware recovery vfio/pds: AddKconfigand documentation.../device_drivers/ethernet/amd/pds_vfio.rst | 79 +++ .../device_drivers/ethernet/index.rst | 1 + MAINTAINERS | 7 + drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/mlx5/cmd.c | 48 +- drivers/vfio/pci/pds/Kconfig | 20 + drivers/vfio/pci/pds/Makefile | 11 + drivers/vfio/pci/pds/cmds.c | 487 +++++++++++++++ drivers/vfio/pci/pds/cmds.h | 25 + drivers/vfio/pci/pds/dirty.c | 577 ++++++++++++++++++ drivers/vfio/pci/pds/dirty.h | 38 ++ drivers/vfio/pci/pds/lm.c | 421 +++++++++++++ drivers/vfio/pci/pds/lm.h | 41 ++ drivers/vfio/pci/pds/pci_drv.c | 206 +++++++ drivers/vfio/pci/pds/pci_drv.h | 9 + drivers/vfio/pci/pds/vfio_dev.c | 234 +++++++ drivers/vfio/pci/pds/vfio_dev.h | 45 ++ drivers/vfio/vfio_main.c | 47 ++ include/linux/pds/pds_adminq.h | 395 ++++++++++++ include/linux/pds/pds_common.h | 2 + include/linux/vfio.h | 3 + 22 files changed, 2653 insertions(+), 47 deletions(-) create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vfio.rst create mode 100644 drivers/vfio/pci/pds/Kconfig create mode 100644 drivers/vfio/pci/pds/Makefile create mode 100644 drivers/vfio/pci/pds/cmds.c create mode 100644 drivers/vfio/pci/pds/cmds.h create mode 100644 drivers/vfio/pci/pds/dirty.c create mode 100644 drivers/vfio/pci/pds/dirty.h create mode 100644 drivers/vfio/pci/pds/lm.c create mode 100644 drivers/vfio/pci/pds/lm.h create mode 100644 drivers/vfio/pci/pds/pci_drv.c create mode 100644 drivers/vfio/pci/pds/pci_drv.h create mode 100644 drivers/vfio/pci/pds/vfio_dev.c create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
Tian, Kevin
reviewerQuoted 43 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMThis is a patchset for a new vendor specific VFIO driver (
pds_vfio) for use with the AMD/Pensando Distributed Services Card (DSC). This driver makes use of thepds_coredriver.This driver will use the
pds_coredevice's adminq as the VFIO control path to the DSC. In order to make adminq calls, the VFIO instance makes use of functions exported by thepds_coredriver.In order to receive events from
pds_core, thepds_vfiodriver registers to a private notifier. This is needed for various events that come from the device.An ASCII diagram of a VFIO instance looks something like this and can be used with the VFIO subsystem to provide the VF device VFIO and live migration support.
.------. .-----------------------. | QEMU |--| VM .-------------. | '......' | | Eth VF | | | | .-------------. | | | | SR-IOV VF | | | | '-------------' | | '------------||---------' .--------------. || |/dev/<vfio_fd>| || '--------------' || Host Userspace | || =================================================== || Host Kernel | || .--------. || |vfio-pci| || '--------' || .------------------. || || | | exported API |<----+ || || | '--------------| | || || | | .-------------. || | pds_core |--->| pds_vfio | || '------------------' | '-------------' || || | || || 09:00.0 notifier 09:00.1 ||
Quoted 9 lines
== PCI ===============================================||=====
|| || ||
.----------. .----------. ||
,-----| PF |----------| VF |-------------------,
| '----------' '----------' | VF |
| DSC | data/control |
| | path |
-----------------------------------------------------------why is "VF data/control path" drawn out of the VF box?
Quoted 56 lines · Kevin
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 43 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, June 3, 2023 6:03 AMThis is a patchset for a new vendor specific VFIO driver (
pds_vfio) for use with the AMD/Pensando Distributed Services Card (DSC). This driver makes use of thepds_coredriver.This driver will use the
pds_coredevice's adminq as the VFIO control path to the DSC. In order to make adminq calls, the VFIO instance makes use of functions exported by thepds_coredriver.In order to receive events from
pds_core, thepds_vfiodriver registers to a private notifier. This is needed for various events that come from the device.An ASCII diagram of a VFIO instance looks something like this and can be used with the VFIO subsystem to provide the VF device VFIO and live migration support.
.------. .-----------------------. | QEMU |--| VM .-------------. | '......' | | Eth VF | | | | .-------------. | | | | SR-IOV VF | | | | '-------------' | | '------------||---------' .--------------. || |/dev/<vfio_fd>| || '--------------' || Host Userspace | || =================================================== || Host Kernel | || .--------. || |vfio-pci| || '--------' || .------------------. || || | | exported API |<----+ || || | '--------------| | || || | | .-------------. || | pds_core |--->| pds_vfio | || '------------------' | '-------------' || || | || || 09:00.0 notifier 09:00.1 ||Quoted 9 lines
== PCI ===============================================||===== || || || .----------. .----------. || ,-----| PF |----------| VF |-------------------, | '----------' '----------' | VF | | DSC | data/control | | | path | -----------------------------------------------------------
Quoted 1 line
why is "VF data/control path" drawn out of the VF box?
Just a mistake in the drawing. I can fix it. Thanks.
Quoted 1 line
Quoted 50 lines · Brett Creeley
This is a patchset for a new vendor specific VFIO driver (
pds_vfio) for use with the AMD/Pensando Distributed Services Card (DSC). This driver makes use of thepds_coredriver.This driver will use the
pds_coredevice's adminq as the VFIO control path to the DSC. In order to make adminq calls, the VFIO instance makes use of functions exported by thepds_coredriver.In order to receive events from
pds_core, thepds_vfiodriver registers to a private notifier. This is needed for various events that come from the device.An ASCII diagram of a VFIO instance looks something like this and can be used with the VFIO subsystem to provide the VF device VFIO and live migration support.
.------. .-----------------------. | QEMU |--| VM .-------------. | '......' | | Eth VF | | | | .-------------. | | | | SR-IOV VF | | | | '-------------' | | '------------||---------' .--------------. || |/dev/<vfio_fd>| || '--------------' || Host Userspace | || =================================================== || Host Kernel | || .--------. || |vfio-pci| || '--------' || .------------------. || || | | exported API |<----+ || || | '--------------| | || || | | .-------------. || | pds_core |--->| pds_vfio | || '------------------' | '-------------' || || | || || 09:00.0 notifier 09:00.1 || == PCI ===============================================||===== || || || .----------. .----------. || ,-----| PF |----------| VF |-------------------, | '----------' '----------' | VF | | DSC | data/control | | | path | -----------------------------------------------------------
Quoted 6 lines
The
pds_vfiodriver is targeted to reside indrivers/vfio/pci/pds. It makes use of and introduces new files in the commoninclude/linux/pdsinclude directory.Changes:
Quoted 1 line
v10:
Just as a quick note, I don't plan to push v11 next week as I am on vacation. I appreciate all the reviews and hope to have updates when I get back on the week of 6/26.
Thanks,
Brett
[...]