pds-vfio-pci driver
Subsystems (9)
Brett Creeley
maintainerv13 cover pds-vfio-pci driver
This is a patchset for a new vendor specific VFIO driver (pds-vfio-pci) for use with the AMD/Pensando Distributed Services Card (DSC). This driver makes use of the pds_core driver.
This driver will use the pds_core device's adminq as the VFIO control path to the DSC. In order to make adminq calls, the VFIO instance makes use of functions exported by the pds_core driver.
In order to receive events from pds_core, the pds-vfio-pci driver registers to a private notifier. This is needed for various events that come from the device.
An ASCII diagram of a VFIO instance looks something like this and can be used with the VFIO subsystem to provide the VF device VFIO and live migration support.
.------. .-----------------------.
| QEMU |--| VM .-------------. |
'......' | | Eth VF | |
| | .-------------. |
| | | SR-IOV VF | |
| | '-------------' |
| '------------||---------'
.--------------. ||
|/dev/<vfio_fd>| ||
'--------------' ||
Host Userspace | ||
=================================================== ||
Host Kernel | ||
.--------. ||
|vfio-pci| ||
'--------' ||
.------------------. || ||
| | exported API |<----+ || ||
| '--------------| | || ||
| | .--------------. ||
| pds_core |--->| pds-vfio-pci | ||
'------------------' | '--------------' ||
|| | || ||
09:00.0 notifier 09:00.1 ||
== PCI ===============================================||=====
|| || ||
.----------. .----------. ||
,-----| PF |----------| VF |-------------------,
| '----------' |'----------' VF |
| DSC | data/control |
| | path |
-----------------------------------------------------------The pds-vfio-pci driver is targeted to reside in drivers/vfio/pci/pds. It makes use of and introduces new files in the common include/linux/pds include directory.
Note: This series is based on the latest linux-next tree. I did not base it on the Alex Williamson's vfio/next because it has not yet pulled in the latest changes which include the pds_vdpa driver. The pds_vdpa driver has conflicts with the pds-vfio-pci driver that needed to be resolved, which is why this series is based on the latest linux-next tree.
Changes:
v13:
- Various fixes/suggestions by Kevin Tian:
- Fix commit message for patch 1/7
- Re-order pds_vfio_init_device() so no unrolling of vfio_pci_core_init_dev() is necessary
- Update #define to align with live migration via VFIO
- Change dev_info() to dev_err() on a failed AQ command
- Rename AQ command from PDS_LM_CMD_STATUS to PDS_LM_CMD_STATE_SIZE and rename the associated structures
- Various fixes/suggestions by Simon Horman:
- Fix RCT issue
- Don't check for null before calling vfree()
v12:
https://lore.kernel.org/netdev/[email protected]/
- Fix htmldocs issue reported by Simon Horman
v11:
https://lore.kernel.org/netdev/[email protected]/
- Various fixes/suggestions by Kevin Tian:
- Update commit message on patch 1/7 to mention ethernet VF
- Fix formatting
- Check return of pci_iov_vf_id() for error
- Update MODULE_AUTHOR to be myself
- Remove double print of pci_id
- Set fast_poll=true for PDS_LM_CMD_RESUME
- Update comments around SUSPEND and SUSPEND_STATUS operations
- Get rid of pds_vfio_deferred_reset() and just perform the operations inline to the only caller
- Rework pds_vfio_state_mutex_unlock() to be simpler
- Rework comments in pds_vfio_recovery()
- Append "vfio" to the end of PDS_LM_DEV_NAME
- Remove req_len from pds_vfio_client_adminq_cmd()
- Remove unused completion structures in pds_adminq.h
- Fix comment above creating VFs to specify pds_core as the PF device
- Various fixes/suggestions by Alex Williamson:
- Removed cached pci_id from pds_vfio_device structure
- Rename from pds_vfio to pds-vfio-pci
- Various fixes/suggestions by Shameerali Kolothum Thodi:
- Fix checking return of pds_client_register()
- Use bool argument for flags instead of u64 on pds_vfio_client_adminq_cmd()
- Fix support for VFIO_MIGRATION_P2P
- Improve member name alignment in pds_adminq.h
- Rework Kconfig based on latest changes from Alex Williamson
v10:
https://lore.kernel.org/netdev/[email protected]/
- Various fixes/suggestions by Jason Gunthorpe
- Simplify pds_vfio_get_lm_file() based on fpga_mgr_buf_load()
- Clean-ups/fixes based on clang-format
- Remove any double goto labels
- Name goto labels baesed on what needs to be cleaned/freed instead of a "call from" scheme
- Fix any goto unwind ordering issues
- Make sure call dma_map_single() after data is written to memory in pds_vfio_dma_map_lm_file()
- Don't use bitmap_zalloc() for the dirty bitmaps
- Use vzalloc() for dirty bitmaps and refactor how the bitmaps are DMA'd to and from the device in pds_vfio_dirty_seq_ack()
- Remove unnecessary goto in pds_vfio_dirty_disable()
v9:
https://lore.kernel.org/netdev/[email protected]/
- Various fixes/suggestions by Alex Williamson
- Fix how ID is generated in client registration
- Add helper functions to get the VF's struct device and struct pci_dev pointers instead of caching the struct pci dev
- Remove redundant pds_vfio_lm_state() function and remove any places this was being called
- Fix multi-line comments to follow standard convention
- Remove confusing comments in pds_vfio_step_device_state_locked() since the driver's migration states align with the VFIO documentation
- Validate pdsc returned from pdsc_get_pf_struct()
- Various fixes/suggestions by Jason Gunthorpe
- Use struct pdsc instead of void *
- Use {} instead of {0} for structure initialization
- Use unions on the stack instead of casting to the union when sending AQ commands, which required including pds_lm.h in pds_adminq.h
- Replace use of dma_alloc_coherent() when creating the sgl DMA entries for the LM file
- Remove cached struct device *coredev and instead use pci_physfn() to get the pds_core's struct device pointer
- Drop the recovery work item and call pds_vfio_recovery() directly from the notifier callback
- Remove unnecessary #define for "pds_vfio_lm" and just use the string inline to the anon_inode_getfile() argument
- Fix LM file reference counting
- Move initialization of some struct members to when the struct is being initialized for AQ commands
- Make use of GFP_KERNEL_ACCOUNT where it makes sense
- Replace PDS_VFIO_DRV_NAME with KBUILD_MODNAME
- Update to latest pds_core exported functions
- Remove duplicated prototypes for pds_vfio_dma_logging_[start|stop|report] from lm.h
- Hold pds_vfio->state_mutex while starting, stopping, and reporting dirty page tracking in pds_vfio_dma_logging_[start|stop|report]
- Remove duplicate PDS_DEV_TYPE_LM_STR define from pds_lm.h that's already included in pds_common.h
- Replace use of dma_alloc_coherent() when creating the sgl DMA entries for the dirty bitmaps
v8:
https://lore.kernel.org/netdev/[email protected]/
- provide default iommufd callbacks for bind_iommufd, unbind_iommufd, and attach_ioas for the VFIO device as suggested by Shameerali Kolothum Thodi
v7:
https://lore.kernel.org/netdev/[email protected]/
- Disable and clean up dirty page tracking when the VFIO device is closed
- Various improvements suggested by Simon Horman:
- Fix RCT in vfio_combine_iova_ranges()
- Simplify function exit paths by removing unnecessary goto labels
- Cleanup pds_vifo_print_guest_region_info() by adding a goto label for freeing memory, which allowed for reduced indentation on a for loop
- Where possible use C99 style for loops
v6:
https://lore.kernel.org/netdev/[email protected]/
- As suggested by Alex Williamson, use pci_domain_nr() macro to make sure the pds_vfio client's devname is unique
- Remove unnecessary forward declaration and include
- Fix copyright comment to use correct company name
- Remove "." from struct documentation for consistency
v5:
https://lore.kernel.org/netdev/[email protected]/
- Fix SPDX comments in .h files
- Remove adminqcq argument from pdsc_post_adminq() uses
- Unregister client on vfio_pci_core_register_device() failure
- Other minor checkpatch issues
v4:
https://lore.kernel.org/netdev/[email protected]/
- Update cover letter ASCII diagram to reflect new driver architecture
- Remove auxiliary driver implementation
- Use pds_core's exported functions to communicate with the device
- Implement and register notifier for events from the device/pds_core
- Use module_pci_driver() macro since auxiliary driver configuration is no longer needed in __init/__exit
v3:
https://lore.kernel.org/netdev/[email protected]/
- Update copyright year to 2023 and use "Advanced Micro Devices, Inc." for the company name
- Clarify the fact that AMD/Pensando's VFIO solution is device type agnostic, which aligns with other current VFIO solutions
- Add line in drivers/vfio/pci/Makefile to build pds_vfio
- Move documentation to amd sub-directory
- Remove some dead code due to the pds_core implementation of listening to BIND/UNBIND events
- Move a dev_dbg() to a previous patch in the series
- Add implementation for vfio_migration_ops.migration_get_data_size to return the maximum possible device state size
RFC to v2:
https://lore.kernel.org/all/[email protected]/
- Implement state transitions for VFIO_MIGRATION_P2P flag
- Improve auxiliary driver probe by returning EPROBE_DEFER when the PCI driver is not set up correctly
- Add pointer to docs in Documentation/networking/device_drivers/ethernet/index.rst
RFC:
https://lore.kernel.org/all/[email protected]/
Brett Creeley (7):
vfio: Commonize combine_ranges for use in other VFIO drivers vfio/pds: Initial support for pds VFIO driver vfio/pds: register with the pds_core PF
vfio/pds: Add VFIO live migration support vfio/pds: Add support for dirty page tracking vfio/pds: Add support for firmware recovery vfio/pds: Add Kconfig and documentation
.../ethernet/amd/pds_vfio_pci.rst | 79 +++ .../device_drivers/ethernet/index.rst | 1 + MAINTAINERS | 7 + drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/mlx5/cmd.c | 48 +- drivers/vfio/pci/pds/Kconfig | 19 + drivers/vfio/pci/pds/Makefile | 11 + drivers/vfio/pci/pds/cmds.c | 493 +++++++++++++++ drivers/vfio/pci/pds/cmds.h | 25 + drivers/vfio/pci/pds/dirty.c | 573 ++++++++++++++++++ drivers/vfio/pci/pds/dirty.h | 39 ++ drivers/vfio/pci/pds/lm.c | 434 +++++++++++++ drivers/vfio/pci/pds/lm.h | 41 ++ drivers/vfio/pci/pds/pci_drv.c | 214 +++++++ drivers/vfio/pci/pds/pci_drv.h | 9 + drivers/vfio/pci/pds/vfio_dev.c | 227 +++++++ drivers/vfio/pci/pds/vfio_dev.h | 42 ++ drivers/vfio/vfio_main.c | 47 ++ include/linux/pds/pds_adminq.h | 375 ++++++++++++ include/linux/pds/pds_common.h | 3 +- include/linux/vfio.h | 3 + 22 files changed, 2646 insertions(+), 48 deletions(-) create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vfio_pci.rst create mode 100644 drivers/vfio/pci/pds/Kconfig create mode 100644 drivers/vfio/pci/pds/Makefile create mode 100644 drivers/vfio/pci/pds/cmds.c create mode 100644 drivers/vfio/pci/pds/cmds.h create mode 100644 drivers/vfio/pci/pds/dirty.c create mode 100644 drivers/vfio/pci/pds/dirty.h create mode 100644 drivers/vfio/pci/pds/lm.c create mode 100644 drivers/vfio/pci/pds/lm.h create mode 100644 drivers/vfio/pci/pds/pci_drv.c create mode 100644 drivers/vfio/pci/pds/pci_drv.h create mode 100644 drivers/vfio/pci/pds/vfio_dev.c create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
Signature · 1 line
2.17.1
Brett Creeley
maintainerPATCH v13 1/7 vfio: Commonize combine_ranges for use in other VFIO drivers
Currently only Mellanox uses the combine_ranges function. The new pds_vfio driver also needs this function. So, move it to a common location for other vendor drivers to use.
Also, fix RCT ordering while moving/renaming the function.
--- drivers/vfio/pci/mlx5/cmd.c | 48 +------------------------------------ drivers/vfio/vfio_main.c | 47 ++++++++++++++++++++++++++++++++++++ include/linux/vfio.h | 3 +++ 3 files changed, 51 insertions(+), 47 deletions(-)
drivers/vfio/pci/mlx5/cmd.c +1 -47
@@ -732,52 +732,6 @@ void mlx5fv_cmd_clean_migf_resources(struct mlx5_vf_migration_file *migf)mlx5vf_cmd_dealloc_pd (migf); } -static void combine_ranges (struct rb_root_cached *root,u32 cur_nodes, -u32 req_nodes) -{ -struct interval_tree_node *prev, *curr, *comb_start, *comb_end; -unsigned long min_gap; -unsigned long curr_gap; - -/* Special shortcut when a single range is required */ -if (req_nodes ==1 ) { -unsigned long last; - - curr = comb_start =interval_tree_iter_first (root,0 ,ULONG_MAX ); -while (curr) { - last = curr->last; - prev = curr; - curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); -if (prev != comb_start) -interval_tree_remove (prev, root); - } - comb_start->last = last; -return ; - } - -/* Combine ranges which have the smallest gap */ -while (cur_nodes > req_nodes) { - prev =NULL ; - min_gap =ULONG_MAX ; - curr =interval_tree_iter_first (root,0 ,ULONG_MAX ); -while (curr) { -if (prev) { - curr_gap = curr->start - prev->last; -if (curr_gap < min_gap) { - min_gap = curr_gap; - comb_start = prev; - comb_end = curr; - } - } - prev = curr; - curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); - } - comb_start->last = comb_end->last; -interval_tree_remove (comb_end, root); - cur_nodes--; - } -} -static int mlx5vf_create_tracker (struct mlx5_core_dev *mdev,struct mlx5vf_pci_core_device *mvdev,struct rb_root_cached *ranges,u32 nnodes)
@@ -800,7 +754,7 @@ static int mlx5vf_create_tracker(struct mlx5_core_dev *mdev,int i;if (num_ranges > max_num_range) { -combine_ranges (ranges, nnodes, max_num_range); +vfio_combine_iova_ranges (ranges, nnodes, max_num_range); num_ranges = max_num_range; }
drivers/vfio/vfio_main.c +47 -0
53 lines not on this page · raw message
include/linux/vfio.h +3 -0
@@ -245,6 +245,9 @@ int vfio_mig_get_next_state(struct vfio_device *device,enum vfio_device_mig_state new_fsm, enum vfio_device_mig_state *next_fsm); +void vfio_combine_iova_ranges (struct rb_root_cached *root,u32 cur_nodes, +u32 req_nodes); +/* * External user API */
Signature · 1 line
2.17.1
Re: [PATCH v13 vfio 1/7] vfio: Commonize combine_ranges for use in other VFIO drivers
Quoted 87 lines · Brett Creeley
Currently only Mellanox uses the
combine_rangesfunction. The newpds_vfiodriver also needs this function. So, move it to a common location for other vendor drivers to use.Also, fix RCT ordering while moving/renaming the function.
Cc Yishai Hadas <[email protected]>Signed-off-by Brett Creeley <[email protected]>Signed-off-by Shannon Nelson <[email protected]>Reviewed-by Simon Horman <[email protected]>--- drivers/vfio/pci/mlx5/cmd.c | 48 +------------------------------------ drivers/vfio/vfio_main.c | 47 ++++++++++++++++++++++++++++++++++++ include/linux/vfio.h | 3 +++ 3 files changed, 51 insertions(+), 47 deletions(-)drivers/vfio/pci/mlx5/cmd.c +1
-47@@ -732,52 +732,6 @@ void mlx5fv_cmd_clean_migf_resources(struct mlx5_vf_migration_file *migf)mlx5vf_cmd_dealloc_pd (migf); } -static void combine_ranges (struct rb_root_cached *root,u32 cur_nodes, -u32 req_nodes) -{ -struct interval_tree_node *prev, *curr, *comb_start, *comb_end; -unsigned long min_gap; -unsigned long curr_gap; - -/* Special shortcut when a single range is required */ -if (req_nodes ==1 ) { -unsigned long last; - - curr = comb_start =interval_tree_iter_first (root,0 ,ULONG_MAX ); -while (curr) { - last = curr->last; - prev = curr; - curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); -if (prev != comb_start) -interval_tree_remove (prev, root); - } - comb_start->last = last; -return ; - } - -/* Combine ranges which have the smallest gap */ -while (cur_nodes > req_nodes) { - prev =NULL ; - min_gap =ULONG_MAX ; - curr =interval_tree_iter_first (root,0 ,ULONG_MAX ); -while (curr) { -if (prev) { - curr_gap = curr->start - prev->last; -if (curr_gap < min_gap) { - min_gap = curr_gap; - comb_start = prev; - comb_end = curr; - } - } - prev = curr; - curr =interval_tree_iter_next (curr,0 ,ULONG_MAX ); - } - comb_start->last = comb_end->last; -interval_tree_remove (comb_end, root); - cur_nodes--; - } -} -static int mlx5vf_create_tracker (struct mlx5_core_dev *mdev,struct mlx5vf_pci_core_device *mvdev,struct rb_root_cached *ranges,u32 nnodes)@@ -800,7 +754,7 @@ static int mlx5vf_create_tracker(struct mlx5_core_dev *mdev,int i;if (num_ranges > max_num_range) { -combine_ranges (ranges, nnodes, max_num_range); +vfio_combine_iova_ranges (ranges, nnodes, max_num_range); num_ranges = max_num_range; }drivers/vfio/vfio_main.c +0
-0
drivers/vfio/vfio_main.c static int vfio_ioctl_device_feature_migration(struct vfio_device *device, · line 870
Quoted 6 lines
drivers/vfio/vfio_main.c +3
-0@@ -865,6 +865,53 @@ static int vfio_ioctl_device_feature_migration(struct vfio_device *device,return 0 ; } +void vfio_combine_iova_ranges (struct rb_root_cached *root,u32 cur_nodes, +u32 req_nodes) +{
It should really gain a kdoc now.
But the code is fine
Jason
Brett Creeley
maintainerPATCH v13 2/7 vfio/pds: Initial support for pds VFIO driver
In reply to Brett CreeleyThis is the initial framework for the new pds-vfio-pci device driver. This does the very basics of registering the PDS PCI device and configuring it as a VFIO PCI device.
With this change, the VF device can be bound to the pds-vfio-pci driver on the host and presented to the VM as an ethernet VF.
--- drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/pds/Makefile | 8 ++++ drivers/vfio/pci/pds/pci_drv.c | 68 ++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.c | 75 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.h | 19 +++++++++ 5 files changed, 172 insertions(+) create mode 100644 drivers/vfio/pci/pds/Makefile create mode 100644 drivers/vfio/pci/pds/pci_drv.c create mode 100644 drivers/vfio/pci/pds/vfio_dev.c create mode 100644 drivers/vfio/pci/pds/vfio_dev.h
drivers/vfio/pci/Makefile +2 -0
@@ -11,3 +11,5 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o obj-$(CONFIG_MLX5_VFIO_PCI) += mlx5/ obj-$(CONFIG_HISI_ACC_VFIO_PCI) += hisilicon/ + +obj-$(CONFIG_PDS_VFIO_PCI) += pds/
drivers/vfio/pci/pds/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Advanced Micro Devices, Inc. + +obj-$(CONFIG_PDS_VFIO_PCI) += pds-vfio-pci.o + +pds-vfio-pci-y := \ + pci_drv.o \ + vfio_dev.o
drivers/vfio/pci/pds/pci_drv.c new file +68 -0
68 lines not on this page · raw message
drivers/vfio/pci/pds/vfio_dev.c new file +75 -0
75 lines not on this page · raw message
drivers/vfio/pci/pds/vfio_dev.h new file +19 -0
@@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _VFIO_DEV_H_ +#define _VFIO_DEV_H_ + +#include <linux/pci.h> +#include <linux/vfio_pci_core.h> + +struct pds_vfio_pci_device { +struct vfio_pci_core_device vfio_coredev; + +int vf_id; +}; + +const struct vfio_device_ops *pds_vfio_ops_info (void ); +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); + +#endif /* _VFIO_DEV_H_ */
Signature · 1 line
2.17.1
Brett Creeley
maintainerPATCH v13 3/7 vfio/pds: register with the pds_core PF
In reply to Brett CreeleyThe pds_core driver will supply adminq services, so find the PF and register with the DSC services.
Use the following commands to enable a VF:
echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs
--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 44 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 10 ++++++++ drivers/vfio/pci/pds/pci_drv.c | 19 ++++++++++++++ drivers/vfio/pci/pds/pci_drv.h | 9 +++++++ drivers/vfio/pci/pds/vfio_dev.c | 13 +++++++++- drivers/vfio/pci/pds/vfio_dev.h | 6 +++++ include/linux/pds/pds_common.h | 3 ++- 8 files changed, 103 insertions(+), 2 deletions(-) create mode 100644 drivers/vfio/pci/pds/cmds.c create mode 100644 drivers/vfio/pci/pds/cmds.h create mode 100644 drivers/vfio/pci/pds/pci_drv.h
drivers/vfio/pci/pds/Makefile +1 -0
@@ -4,5 +4,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds-vfio-pci.o pds-vfio-pci-y := \ + cmds.o \ pci_drv.o \ vfio_dev.o
drivers/vfio/pci/pds/cmds.c new file +44 -0
44 lines not on this page · raw message
drivers/vfio/pci/pds/cmds.h new file +10 -0
@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _CMDS_H_ +#define _CMDS_H_ + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); + +#endif /* _CMDS_H_ */
drivers/vfio/pci/pds/pci_drv.c +19 -0
@@ -8,9 +8,13 @@#include <linux/types.h> #include <linux/vfio.h> +#include <linux/pds/pds_common.h> #include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> #include "vfio_dev.h" +#include "pci_drv.h" +#include "cmds.h" #define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO Device Driver"#define PCI_VENDOR_ID_PENSANDO 0x1dd8
@@ -27,13 +31,27 @@ static int pds_vfio_pci_probe(struct pci_dev *pdev,return PTR_ERR (pds_vfio);dev_set_drvdata (&pdev->dev, &pds_vfio->vfio_coredev); + pds_vfio->pdsc =pdsc_get_pf_struct (pdev); +if (IS_ERR_OR_NULL (pds_vfio->pdsc)) { + err =PTR_ERR (pds_vfio->pdsc) ?: -ENODEV ; + goto out_put_vdev; + } err =vfio_pci_core_register_device (&pds_vfio->vfio_coredev);if (err) goto out_put_vdev; + err =pds_vfio_register_client_cmd (pds_vfio); +if (err) { +dev_err (&pdev->dev,"failed to register as client: %pe\n" , +ERR_PTR (err)); + goto out_unregister_coredev; + } +return 0 ; +out_unregister_coredev: +vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); out_put_vdev:vfio_put_device (&pds_vfio->vfio_coredev.vdev);return err;
@@ -43,6 +61,7 @@ static void pds_vfio_pci_remove(struct pci_dev *pdev) {struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); +pds_vfio_unregister_client_cmd (pds_vfio);vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev);vfio_put_device (&pds_vfio->vfio_coredev.vdev); }
drivers/vfio/pci/pds/pci_drv.h new file +9 -0
@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _PCI_DRV_H +#define _PCI_DRV_H + +#include <linux/pci.h> + +#endif /* _PCI_DRV_H */
drivers/vfio/pci/pds/vfio_dev.c +12 -1
@@ -6,6 +6,11 @@#include "vfio_dev.h" +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return pds_vfio->vfio_coredev.pdev; +} +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) {struct vfio_pci_core_device *core_device =dev_get_drvdata (&pdev->dev);
@@ -20,7 +25,7 @@ static int pds_vfio_init_device(struct vfio_device *vdev) container_of(vdev, struct pds_vfio_pci_device, vfio_coredev.vdev);struct pci_dev *pdev =to_pci_dev (vdev->dev); -int err, vf_id; +int err, vf_id, pci_id; vf_id =pci_iov_vf_id (pdev);if (vf_id <0 )
@@ -32,6 +37,12 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id = vf_id; + pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); +dev_dbg (&pdev->dev, +"%s: PF %#04x VF %#04x vf_id %d domain %d pds_vfio %p\n" , + __func__,pci_dev_id (pdev->physfn), pci_id, vf_id, +pci_domain_nr (pdev->bus), pds_vfio); +return 0 ; }
drivers/vfio/pci/pds/vfio_dev.h +6 -0
@@ -7,13 +7,19 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +struct pdsc ; +struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev; +struct pdsc *pdsc;int vf_id; +u16 client_id; };const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +#endif /* _VFIO_DEV_H_ */
include/linux/pds/pds_common.h +2 -1
@@ -34,12 +34,13 @@ enum pds_core_vif_types {#define PDS_DEV_TYPE_CORE_STR "Core"#define PDS_DEV_TYPE_VDPA_STR "vDPA" -#define PDS_DEV_TYPE_VFIO_STR "VFio" +#define PDS_DEV_TYPE_VFIO_STR "vfio"#define PDS_DEV_TYPE_ETH_STR "Eth"#define PDS_DEV_TYPE_RDMA_STR "RDMA"#define PDS_DEV_TYPE_LM_STR "LM"#define PDS_VDPA_DEV_NAME PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_VDPA_STR +#define PDS_VFIO_LM_DEV_NAME PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_LM_STR "." PDS_DEV_TYPE_VFIO_STRint pdsc_register_notify (struct notifier_block *nb);void pdsc_unregister_notify (struct notifier_block *nb);
Signature · 1 line
2.17.1
Re: [PATCH v13 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 57 lines · On Tue, 25 Jul 2023 14:40:21 -0700 Brett Creeley <[email protected]> wrote:
The
pds_coredriver will supply adminq services, so find the PF and register with the DSC services.Use the following commands to enable a VF:
echo 1 >/sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfsSigned-off-by Brett Creeley <[email protected]>Signed-off-by Shannon Nelson <[email protected]>--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 44 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 10 ++++++++ drivers/vfio/pci/pds/pci_drv.c | 19 ++++++++++++++ drivers/vfio/pci/pds/pci_drv.h | 9 +++++++ drivers/vfio/pci/pds/vfio_dev.c | 13 +++++++++- drivers/vfio/pci/pds/vfio_dev.h | 6 +++++ include/linux/pds/pds_common.h | 3 ++- 8 files changed, 103 insertions(+), 2 deletions(-) create mode 100644 drivers/vfio/pci/pds/cmds.c create mode 100644 drivers/vfio/pci/pds/cmds.h create mode 100644 drivers/vfio/pci/pds/pci_drv.hdrivers/vfio/pci/pds/Makefile +1
-0@@ -4,5 +4,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds-vfio-pci.o pds-vfio-pci-y := \ + cmds.o \ pci_drv.o \ vfio_dev.odrivers/vfio/pci/pds/cmds.c new file +18
-0@@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/io.h> +#include <linux/types.h> + +#include <linux/pds/pds_common.h> +#include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> + +#include "vfio_dev.h" +#include "cmds.h" + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +char devname[PDS_DEVNAME_LEN ]; +int ci;
drivers/vfio/pci/pds/cmds.c line 28
Quoted 10 lines
drivers/vfio/pci/pds/cmds.c new file +10
-0+ +snprintf (devname,sizeof (devname),"%s.%d-%u" ,PDS_VFIO_LM_DEV_NAME , +pci_domain_nr (pdev->bus), +PCI_DEVID (pdev->bus->number, pdev->devfn)); + + ci =pds_client_register (pci_physfn (pdev), devname); +if (ci <0 ) +return ci; + + pds_vfio->client_id = ci;
Not to be solved in this series, but the documentation is wrong:
/**
* pds_client_register - Link the client to the firmware
* @pf_pdev: ptr to the PF driver struct
* @devname: name that includes service into, e.g. pds_core.vDPA
*
* Return: 0 on success, or
* negative for error
*/
But obviously it does return the client ID and cannot return 0. Thanks,
Alex
Quoted 182 lines
drivers/vfio/pci/pds/cmds.c new file +16
-0+ +return 0 ; +} + +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +int err; + + err =pds_client_unregister (pci_physfn (pdev), pds_vfio->client_id); +if (err) +dev_err (&pdev->dev,"unregister from DSC failed: %pe\n" , +ERR_PTR (err)); + + pds_vfio->client_id =0 ; +}drivers/vfio/pci/pds/cmds.h new file +10
-0@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _CMDS_H_ +#define _CMDS_H_ + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); + +#endif /* _CMDS_H_ */ drivers/vfio/pci/pds/pci_drv.c +19
-0@@ -8,9 +8,13 @@#include <linux/types.h> #include <linux/vfio.h> +#include <linux/pds/pds_common.h> #include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> #include "vfio_dev.h" +#include "pci_drv.h" +#include "cmds.h" #define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO Device Driver"#define PCI_VENDOR_ID_PENSANDO 0x1dd8@@ -27,13 +31,27 @@ static int pds_vfio_pci_probe(struct pci_dev *pdev,return PTR_ERR (pds_vfio);dev_set_drvdata (&pdev->dev, &pds_vfio->vfio_coredev); + pds_vfio->pdsc =pdsc_get_pf_struct (pdev); +if (IS_ERR_OR_NULL (pds_vfio->pdsc)) { + err =PTR_ERR (pds_vfio->pdsc) ?: -ENODEV ; + goto out_put_vdev; + } err =vfio_pci_core_register_device (&pds_vfio->vfio_coredev);if (err) goto out_put_vdev; + err =pds_vfio_register_client_cmd (pds_vfio); +if (err) { +dev_err (&pdev->dev,"failed to register as client: %pe\n" , +ERR_PTR (err)); + goto out_unregister_coredev; + } +return 0 ; +out_unregister_coredev: +vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); out_put_vdev:vfio_put_device (&pds_vfio->vfio_coredev.vdev);return err;@@ -43,6 +61,7 @@ static void pds_vfio_pci_remove(struct pci_dev *pdev) {struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); +pds_vfio_unregister_client_cmd (pds_vfio);vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev);vfio_put_device (&pds_vfio->vfio_coredev.vdev); }drivers/vfio/pci/pds/pci_drv.h new file +9
-0@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _PCI_DRV_H +#define _PCI_DRV_H + +#include <linux/pci.h> + +#endif /* _PCI_DRV_H */ drivers/vfio/pci/pds/vfio_dev.c +12
-1@@ -6,6 +6,11 @@#include "vfio_dev.h" +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return pds_vfio->vfio_coredev.pdev; +} +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) {struct vfio_pci_core_device *core_device =dev_get_drvdata (&pdev->dev);@@ -20,7 +25,7 @@ static int pds_vfio_init_device(struct vfio_device *vdev) container_of(vdev, struct pds_vfio_pci_device, vfio_coredev.vdev);struct pci_dev *pdev =to_pci_dev (vdev->dev); -int err, vf_id; +int err, vf_id, pci_id; vf_id =pci_iov_vf_id (pdev);if (vf_id <0 )@@ -32,6 +37,12 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id = vf_id; + pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); +dev_dbg (&pdev->dev, +"%s: PF %#04x VF %#04x vf_id %d domain %d pds_vfio %p\n" , + __func__,pci_dev_id (pdev->physfn), pci_id, vf_id, +pci_domain_nr (pdev->bus), pds_vfio); +return 0 ; }drivers/vfio/pci/pds/vfio_dev.h +6
-0@@ -7,13 +7,19 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +struct pdsc ; +struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev; +struct pdsc *pdsc;int vf_id; +u16 client_id; };const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +#endif /* _VFIO_DEV_H_ */ include/linux/pds/pds_common.h +2
-1@@ -34,12 +34,13 @@ enum pds_core_vif_types {#define PDS_DEV_TYPE_CORE_STR "Core"#define PDS_DEV_TYPE_VDPA_STR "vDPA" -#define PDS_DEV_TYPE_VFIO_STR "VFio" +#define PDS_DEV_TYPE_VFIO_STR "vfio"#define PDS_DEV_TYPE_ETH_STR "Eth"#define PDS_DEV_TYPE_RDMA_STR "RDMA"#define PDS_DEV_TYPE_LM_STR "LM"#define PDS_VDPA_DEV_NAME PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_VDPA_STR +#define PDS_VFIO_LM_DEV_NAME PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_LM_STR "." PDS_DEV_TYPE_VFIO_STRint pdsc_register_notify (struct notifier_block *nb);void pdsc_unregister_notify (struct notifier_block *nb);
Re: [PATCH v13 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 85 lines · Alex Williamson
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 57 lines · On Tue, 25 Jul 2023 14:40:21 -0700 Brett Creeley <[email protected]> wrote:
The
pds_coredriver will supply adminq services, so find the PF and register with the DSC services.Use the following commands to enable a VF:
echo 1 >/sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfsSigned-off-by Brett Creeley <[email protected]>Signed-off-by Shannon Nelson <[email protected]>--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 44 +++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 10 ++++++++ drivers/vfio/pci/pds/pci_drv.c | 19 ++++++++++++++ drivers/vfio/pci/pds/pci_drv.h | 9 +++++++ drivers/vfio/pci/pds/vfio_dev.c | 13 +++++++++- drivers/vfio/pci/pds/vfio_dev.h | 6 +++++ include/linux/pds/pds_common.h | 3 ++- 8 files changed, 103 insertions(+), 2 deletions(-)create mode 100644
drivers/vfio/pci/pds/cmds.ccreate mode 100644drivers/vfio/pci/pds/cmds.hcreate mode 100644drivers/vfio/pci/pds/pci_drv.hdrivers/vfio/pci/pds/Makefile +1
-0@@ -4,5 +4,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds-vfio-pci.o pds-vfio-pci-y := \ + cmds.o \ pci_drv.o \ vfio_dev.odrivers/vfio/pci/pds/cmds.c new file +18
-0@@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/io.h> +#include <linux/types.h> + +#include <linux/pds/pds_common.h> +#include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> + +#include "vfio_dev.h" +#include "cmds.h" + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +char devname[PDS_DEVNAME_LEN ]; +int ci;drivers/vfio/pci/pds/cmds.c line 28
Quoted 10 lines
drivers/vfio/pci/pds/cmds.c new file +10
-0+ +snprintf (devname,sizeof (devname),"%s.%d-%u" ,PDS_VFIO_LM_DEV_NAME , +pci_domain_nr (pdev->bus), +PCI_DEVID (pdev->bus->number, pdev->devfn)); + + ci =pds_client_register (pci_physfn (pdev), devname); +if (ci <0 ) +return ci; + + pds_vfio->client_id = ci;Not to be solved in this series, but the documentation is wrong:
/**
*pds_client_register- Link the client to the firmware
* @pf_pdev: ptr to the PF driver struct
* @devname: name that includes service into, e.g. pds_core.vDPA
*
* Return: 0 on success, or
* negative for error
*/
Quoted 1 line
But obviously it does return the client ID and cannot return 0. Thanks,
Let me push out a separate patch today that fixes this.
Thanks,
Brett
Quoted 186 lines
Alex
Quoted 182 lines
drivers/vfio/pci/pds/cmds.c new file +16
-0+ +return 0 ; +} + +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +int err; + + err =pds_client_unregister (pci_physfn (pdev), pds_vfio->client_id); +if (err) +dev_err (&pdev->dev,"unregister from DSC failed: %pe\n" , +ERR_PTR (err)); + + pds_vfio->client_id =0 ; +}drivers/vfio/pci/pds/cmds.h new file +10
-0@@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _CMDS_H_ +#define _CMDS_H_ + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); + +#endif /* _CMDS_H_ */ drivers/vfio/pci/pds/pci_drv.c +19
-0@@ -8,9 +8,13 @@#include <linux/types.h> #include <linux/vfio.h> +#include <linux/pds/pds_common.h> #include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> #include "vfio_dev.h" +#include "pci_drv.h" +#include "cmds.h" #define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO Device Driver"#define PCI_VENDOR_ID_PENSANDO 0x1dd8@@ -27,13 +31,27 @@ static int pds_vfio_pci_probe(struct pci_dev *pdev,return PTR_ERR (pds_vfio);dev_set_drvdata (&pdev->dev, &pds_vfio->vfio_coredev); + pds_vfio->pdsc =pdsc_get_pf_struct (pdev); +if (IS_ERR_OR_NULL (pds_vfio->pdsc)) { + err =PTR_ERR (pds_vfio->pdsc) ?: -ENODEV ; + goto out_put_vdev; + } err =vfio_pci_core_register_device (&pds_vfio->vfio_coredev);if (err) goto out_put_vdev; + err =pds_vfio_register_client_cmd (pds_vfio); +if (err) { +dev_err (&pdev->dev,"failed to register as client: %pe\n" , +ERR_PTR (err)); + goto out_unregister_coredev; + } +return 0 ; +out_unregister_coredev: +vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); out_put_vdev:vfio_put_device (&pds_vfio->vfio_coredev.vdev);return err;@@ -43,6 +61,7 @@ static void pds_vfio_pci_remove(struct pci_dev *pdev) {struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); +pds_vfio_unregister_client_cmd (pds_vfio);vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev);vfio_put_device (&pds_vfio->vfio_coredev.vdev); }drivers/vfio/pci/pds/pci_drv.h new file +9
-0@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _PCI_DRV_H +#define _PCI_DRV_H + +#include <linux/pci.h> + +#endif /* _PCI_DRV_H */ drivers/vfio/pci/pds/vfio_dev.c +12
-1@@ -6,6 +6,11 @@#include "vfio_dev.h" +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio) +{ +return pds_vfio->vfio_coredev.pdev; +} +struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev) {struct vfio_pci_core_device *core_device =dev_get_drvdata (&pdev->dev);@@ -20,7 +25,7 @@ static int pds_vfio_init_device(struct vfio_device *vdev) container_of(vdev, struct pds_vfio_pci_device, vfio_coredev.vdev);struct pci_dev *pdev =to_pci_dev (vdev->dev); -int err, vf_id; +int err, vf_id, pci_id; vf_id =pci_iov_vf_id (pdev);if (vf_id <0 )@@ -32,6 +37,12 @@ static int pds_vfio_init_device(struct vfio_device *vdev) pds_vfio->vf_id = vf_id; + pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); +dev_dbg (&pdev->dev, +"%s: PF %#04x VF %#04x vf_id %d domain %d pds_vfio %p\n" , + __func__,pci_dev_id (pdev->physfn), pci_id, vf_id, +pci_domain_nr (pdev->bus), pds_vfio); +return 0 ; }drivers/vfio/pci/pds/vfio_dev.h +6
-0@@ -7,13 +7,19 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +struct pdsc ; +struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev; +struct pdsc *pdsc;int vf_id; +u16 client_id; };const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +#endif /* _VFIO_DEV_H_ */ include/linux/pds/pds_common.h +2
-1@@ -34,12 +34,13 @@ enum pds_core_vif_types {#define PDS_DEV_TYPE_CORE_STR "Core"#define PDS_DEV_TYPE_VDPA_STR "vDPA" -#define PDS_DEV_TYPE_VFIO_STR "VFio" +#define PDS_DEV_TYPE_VFIO_STR "vfio"#define PDS_DEV_TYPE_ETH_STR "Eth"#define PDS_DEV_TYPE_RDMA_STR "RDMA"#define PDS_DEV_TYPE_LM_STR "LM"#define PDS_VDPA_DEV_NAME PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_VDPA_STR +#define PDS_VFIO_LM_DEV_NAME PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_LM_STR "." PDS_DEV_TYPE_VFIO_STRint pdsc_register_notify (struct notifier_block *nb);void pdsc_unregister_notify (struct notifier_block *nb);
Jason Gunthorpe
reviewerRe: [PATCH v13 vfio 3/7] vfio/pds: register with the pds_core PF
In reply to Brett CreeleyQuoted 22 lines · Brett Creeley
drivers/vfio/pci/pds/cmds.c new file +16
-0@@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/io.h> +#include <linux/types.h> + +#include <linux/pds/pds_common.h> +#include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> + +#include "vfio_dev.h" +#include "cmds.h" + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio);
drivers/vfio/pci/pds/cmds.c line 26
Quoted 10 lines
drivers/vfio/pci/pds/cmds.c new file +10
-0+char devname[PDS_DEVNAME_LEN ]; +int ci; + +snprintf (devname,sizeof (devname),"%s.%d-%u" ,PDS_VFIO_LM_DEV_NAME , +pci_domain_nr (pdev->bus), +PCI_DEVID (pdev->bus->number, pdev->devfn)); + + ci =pds_client_register (pci_physfn (pdev), devname); +if (ci <0 ) +return ci;
This is not the right way to get the drvdata of a PCI PF from a VF, you must call pci_iov_get_pf_drvdata().
Jason
Re: [PATCH v13 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 41 lines · Jason Gunthorpe
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 22 lines · Brett Creeley
drivers/vfio/pci/pds/cmds.c new file +16
-0@@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/io.h> +#include <linux/types.h> + +#include <linux/pds/pds_common.h> +#include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> + +#include "vfio_dev.h" +#include "cmds.h" + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio);drivers/vfio/pci/pds/cmds.c line 26
Quoted 10 lines
drivers/vfio/pci/pds/cmds.c new file +10
-0+char devname[PDS_DEVNAME_LEN ]; +int ci; + +snprintf (devname,sizeof (devname),"%s.%d-%u" ,PDS_VFIO_LM_DEV_NAME , +pci_domain_nr (pdev->bus), +PCI_DEVID (pdev->bus->number, pdev->devfn)); + + ci =pds_client_register (pci_physfn (pdev), devname); +if (ci <0 ) +return ci;This is not the right way to get the drvdata of a PCI PF from a VF, you must call
pci_iov_get_pf_drvdata().
Quoted 1 line
Jason
Okay, I will look at this and fix it up on the next version.
Thanks,
Brett
Re: [PATCH v13 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 43 lines · Brett Creeley
Quoted 42 lines · Jason Gunthorpe
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 26 lines · Brett Creeley
drivers/vfio/pci/pds/cmds.c new file +20
-0@@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#include <linux/io.h> +#include <linux/types.h> + +#include <linux/pds/pds_common.h> +#include <linux/pds/pds_core_if.h> +#include <linux/pds/pds_adminq.h> + +#include "vfio_dev.h" +#include "cmds.h" + +int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ + struct pci_dev *pdev = pds_vfio_to_pci_dev(pds_vfio); + char devname[PDS_DEVNAME_LEN]; + int ci; + + snprintf(devname, sizeof(devname), "%s.%d-%u",Quoted 7 lines
PDS_VFIO_LM_DEV_NAME,
+pci_domain_nr(pdev->bus),
+PCI_DEVID(pdev->bus->number, pdev->devfn)); +
+ ci =pds_client_register(pci_physfn(pdev), devname); + if (ci < 0)
+ return ci;This is not the right way to get the drvdata of a PCI PF from a VF, you must call
pci_iov_get_pf_drvdata().
Quoted 7 lines
Quoted 1 line
Jason
Okay, I will look at this and fix it up on the next version.
Thanks,
Brett
After taking another look this was intentional. I'm getting the PF pci_dev, not the PF's drvdata.
Thanks,
Brett
Jason Gunthorpe
reviewerRe: [PATCH v13 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 18 lines · Brett Creeley
Quoted 18 lines
Quoted 18 lines
Quoted 7 lines
drivers/vfio/pci/pds/cmds.c new file +7
-0+int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ + struct pci_dev *pdev = pds_vfio_to_pci_dev(pds_vfio); + char devname[PDS_DEVNAME_LEN]; + int ci; + + snprintf(devname, sizeof(devname), "%s.%d-%u",Quoted 7 lines
PDS_VFIO_LM_DEV_NAME,
+pci_domain_nr(pdev->bus),
+PCI_DEVID(pdev->bus->number, pdev->devfn)); +
+ ci =pds_client_register(pci_physfn(pdev), devname); + if (ci < 0)
+ return ci;This is not the right way to get the drvdata of a PCI PF from a VF, you must call
pci_iov_get_pf_drvdata().
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Jason
Okay, I will look at this and fix it up on the next version.
After taking another look this was intentional. I'm getting the PF
pci_dev, not the PF's drvdata.
pds_client_register() gets the drvdata from the passed PCI function, you have to use pci_iov_get_pf_drvdata() to do this. You can't shortcut it like this.
This is all nonsensical, the existing callers start with aa pdsc:
int pdsc_auxbus_dev_add(struct pdsc *cf, struct pdsc *pf)
Then they call
client_id = pds_client_register(pf->pdev, devname);
Which then does
struct pdsc *pf; pf = pci_get_drvdata(pf_pdev);
Just pass in the pdsc you already had
Add another wrapper to get the pdsc from a VF using pci_iov_get_pf_drvdata() like you are supposed to.
Jason
Re: [PATCH v13 vfio 3/7] vfio/pds: register with the pds_core PF
Quoted 46 lines · Jason Gunthorpe
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 18 lines · Brett Creeley
Quoted 18 lines
Quoted 18 lines
Quoted 8 lines
drivers/vfio/pci/pds/cmds.c new file +7
-0+int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio) +{ +struct pci_dev *pdev =pds_vfio_to_pci_dev (pds_vfio); +char devname[PDS_DEVNAME_LEN ]; +int ci; + + snprintf(devname, sizeof(devname), "%s.%d-%u",
PDS_VFIO_LM_DEV_NAME,drivers/vfio/pci/pds/cmds.c line 26
Quoted 6 lines
drivers/vfio/pci/pds/cmds.c new file +6
-0+pci_domain_nr (pdev->bus), + PCI_DEVID(pdev->bus->number, pdev->devfn)); + + ci =pds_client_register (pci_physfn (pdev), devname); +if (ci <0 ) +return ci;This is not the right way to get the drvdata of a PCI PF from a VF, you must call
pci_iov_get_pf_drvdata().Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Jason
Okay, I will look at this and fix it up on the next version.
After taking another look this was intentional. I'm getting the PF
pci_dev, not the PF's drvdata.
pds_client_register()gets the drvdata from the passed PCI function, you have to usepci_iov_get_pf_drvdata()to do this. You can't shortcut it like this.This is all nonsensical, the existing callers start with aa pdsc:
int
pdsc_auxbus_dev_add(struct pdsc *cf, struct pdsc *pf)Then they call
client_id = pds_client_register(pf->pdev, devname);Which then does
struct pdsc *pf; pf = pci_get_drvdata(pf_pdev);
Quoted 6 lines
Just pass in the pdsc you already had
Add another wrapper to get the pdsc from a VF using
pci_iov_get_pf_drvdata()like you are supposed to.Jason
Yeah this makes sense. Thanks for pointing this out. We will rework this in pds_core and I will add another small patch to the series that handles this. Then I can rework this patch to align. Good catch. I should have a v14 out later today with this update and other pending review comments.
Thanks for the review,
Brett
Brett Creeley
maintainerPATCH v13 4/7 vfio/pds: Add VFIO live migration support
In reply to Brett CreeleyAdd live migration support via the VFIO subsystem. The migration implementation aligns with the definition from uapi/vfio.h and uses the pds_core PF's adminq for device configuration.
The ability to suspend, resume, and transfer VF device state data is included along with the required admin queue command structures and implementations.
PDS_LM_CMD_SUSPEND and PDS_LM_CMD_SUSPEND_STATUS are added to support the VF device suspend operation.
PDS_LM_CMD_RESUME is added to support the VF device resume operation.
PDS_LM_CMD_STATE_SIZE is added to determine the exact size of the VF device state data.
PDS_LM_CMD_SAVE is added to get the VF device state data.
PDS_LM_CMD_RESTORE is added to restore the VF device with the previously saved data from PDS_LM_CMD_SAVE.
PDS_LM_CMD_HOST_VF_STATUS is added to notify the DSC/firmware when a migration is in/not-in progress from the host's perspective. The DSC/firmware can use this to clear/setup any necessary state related to a migration.
--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 324 ++++++++++++++++++++++++ drivers/vfio/pci/pds/cmds.h | 8 +- drivers/vfio/pci/pds/lm.c | 434 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/lm.h | 41 +++ drivers/vfio/pci/pds/pci_drv.c | 13 + drivers/vfio/pci/pds/vfio_dev.c | 120 ++++++++- drivers/vfio/pci/pds/vfio_dev.h | 11 + include/linux/pds/pds_adminq.h | 197 +++++++++++++++ 9 files changed, 1147 insertions(+), 2 deletions(-) create mode 100644 drivers/vfio/pci/pds/lm.c create mode 100644 drivers/vfio/pci/pds/lm.h
drivers/vfio/pci/pds/Makefile +1 -0
@@ -5,5 +5,6 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds-vfio-pci.o pds-vfio-pci-y := \ cmds.o \ + lm.o \ pci_drv.o \ vfio_dev.o
drivers/vfio/pci/pds/cmds.c +324 -0
339 lines not on this page · raw message
drivers/vfio/pci/pds/cmds.h +7 -1
@@ -6,5 +6,11 @@int pds_vfio_register_client_cmd (struct pds_vfio_pci_device *pds_vfio);void pds_vfio_unregister_client_cmd (struct pds_vfio_pci_device *pds_vfio); - +int pds_vfio_suspend_device_cmd (struct pds_vfio_pci_device *pds_vfio,u8 type); +int pds_vfio_resume_device_cmd (struct pds_vfio_pci_device *pds_vfio,u8 type); +int pds_vfio_get_lm_state_size_cmd (struct pds_vfio_pci_device *pds_vfio,u64 *size); +int pds_vfio_get_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_set_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_send_host_vf_lm_status_cmd (struct pds_vfio_pci_device *pds_vfio, +enum pds_lm_host_vf_status vf_status);#endif /* _CMDS_H_ */
drivers/vfio/pci/pds/lm.c new file +434 -0
434 lines not on this page · raw message
drivers/vfio/pci/pds/lm.h new file +41 -0
41 lines not on this page · raw message
drivers/vfio/pci/pds/pci_drv.c +13 -0
@@ -72,11 +72,24 @@ static const struct pci_device_id pds_vfio_pci_table[] = { };MODULE_DEVICE_TABLE (pci, pds_vfio_pci_table); +static void pds_vfio_pci_aer_reset_done (struct pci_dev *pdev) +{ +struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); + +pds_vfio_reset (pds_vfio); +} + +static const struct pci_error_handlers pds_vfio_pci_err_handlers = { + .reset_done = pds_vfio_pci_aer_reset_done, + .error_detected = vfio_pci_core_aer_err_detected, +}; +static struct pci_driver pds_vfio_pci_driver = { .name =KBUILD_MODNAME , .id_table = pds_vfio_pci_table, .probe = pds_vfio_pci_probe, .remove = pds_vfio_pci_remove, + .err_handler = &pds_vfio_pci_err_handlers, .driver_managed_dma = true, };
drivers/vfio/pci/pds/vfio_dev.c +119 -1
160 lines not on this page · raw message
drivers/vfio/pci/pds/vfio_dev.h +11 -0
@@ -7,19 +7,30 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +#include "lm.h" +struct pdsc ;struct pds_vfio_pci_device {struct vfio_pci_core_device vfio_coredev;struct pdsc *pdsc; +struct pds_vfio_lm_file *save_file; +struct pds_vfio_lm_file *restore_file; +struct mutex state_mutex;/* protect migration state */ +enum vfio_device_mig_state state; +spinlock_t reset_lock;/* protect reset_done flow */ +u8 deferred_reset; +int vf_id;u16 client_id; };const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev); +void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio);struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio); +struct device *pds_vfio_to_dev (struct pds_vfio_pci_device *pds_vfio);#endif /* _VFIO_DEV_H_ */
include/linux/pds/pds_adminq.h +197 -0
215 lines not on this page · raw message
Signature · 1 line
2.17.1
Brett Creeley
maintainerPATCH v13 5/7 vfio/pds: Add support for dirty page tracking
In reply to Brett CreeleyIn order to support dirty page tracking, the driver has to implement the VFIO subsystem's vfio_log_ops. This includes log_start, log_stop, and log_read_and_clear.
All of the tracker resources are allocated and dirty tracking on the device is started during log_start. The resources are cleaned up and dirty tracking on the device is stopped during log_stop. The dirty pages are determined and reported during log_read_and_clear.
In order to support these callbacks admin queue commands are used. All of the adminq queue command structures and implementations are included as part of this patch.
PDS_LM_CMD_DIRTY_STATUS is added to query the current status of dirty tracking on the device. This includes if it's enabled (i.e. number of regions being tracked from the device's perspective) and the maximum number of regions supported from the device's perspective.
PDS_LM_CMD_DIRTY_ENABLE is added to enable dirty tracking on the specified number of regions and their iova ranges.
PDS_LM_CMD_DIRTY_DISABLE is added to disable dirty tracking for all regions on the device.
PDS_LM_CMD_READ_SEQ and PDS_LM_CMD_DIRTY_WRITE_ACK are added to support reading and acknowledging the currently dirtied pages.
--- drivers/vfio/pci/pds/Makefile | 1 + drivers/vfio/pci/pds/cmds.c | 125 +++++++ drivers/vfio/pci/pds/cmds.h | 9 + drivers/vfio/pci/pds/dirty.c | 573 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/dirty.h | 39 +++ drivers/vfio/pci/pds/lm.c | 2 +- drivers/vfio/pci/pds/vfio_dev.c | 12 +- drivers/vfio/pci/pds/vfio_dev.h | 4 + include/linux/pds/pds_adminq.h | 178 ++++++++++ 9 files changed, 941 insertions(+), 2 deletions(-) create mode 100644 drivers/vfio/pci/pds/dirty.c create mode 100644 drivers/vfio/pci/pds/dirty.h
drivers/vfio/pci/pds/Makefile +1 -0
@@ -5,6 +5,7 @@ obj-$(CONFIG_PDS_VFIO_PCI) += pds-vfio-pci.o pds-vfio-pci-y := \ cmds.o \ + dirty.o \ lm.o \ pci_drv.o \ vfio_dev.o
drivers/vfio/pci/pds/cmds.c +125 -0
128 lines not on this page · raw message
drivers/vfio/pci/pds/cmds.h +9 -0
@@ -13,4 +13,13 @@ int pds_vfio_get_lm_state_cmd(struct pds_vfio_pci_device *pds_vfio);int pds_vfio_set_lm_state_cmd (struct pds_vfio_pci_device *pds_vfio);void pds_vfio_send_host_vf_lm_status_cmd (struct pds_vfio_pci_device *pds_vfio,enum pds_lm_host_vf_status vf_status); +int pds_vfio_dirty_status_cmd (struct pds_vfio_pci_device *pds_vfio, +u64 regions_dma,u8 *max_regions, +u8 *num_regions); +int pds_vfio_dirty_enable_cmd (struct pds_vfio_pci_device *pds_vfio, +u64 regions_dma,u8 num_regions); +int pds_vfio_dirty_disable_cmd (struct pds_vfio_pci_device *pds_vfio); +int pds_vfio_dirty_seq_ack_cmd (struct pds_vfio_pci_device *pds_vfio, +u64 sgl_dma,u16 num_sge,u32 offset, +u32 total_len,bool read_seq);#endif /* _CMDS_H_ */
drivers/vfio/pci/pds/dirty.c new file +573 -0
573 lines not on this page · raw message
drivers/vfio/pci/pds/dirty.h new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _DIRTY_H_ +#define _DIRTY_H_ + +struct pds_vfio_bmp_info { +unsigned long *bmp; +u32 bmp_bytes; +struct pds_lm_sg_elem *sgl; +dma_addr_t sgl_addr; +u16 num_sge; +}; + +struct pds_vfio_dirty { +struct pds_vfio_bmp_info host_seq; +struct pds_vfio_bmp_info host_ack; +u64 region_size; +u64 region_start; +u64 region_page_size; +bool is_enabled; +}; + +struct pds_vfio_pci_device ; + +bool pds_vfio_dirty_is_enabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_set_enabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_set_disabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_disable (struct pds_vfio_pci_device *pds_vfio, +bool send_cmd); + +int pds_vfio_dma_logging_report (struct vfio_device *vdev,unsigned long iova, +unsigned long length, +struct iova_bitmap *dirty); +int pds_vfio_dma_logging_start (struct vfio_device *vdev, +struct rb_root_cached *ranges,u32 nnodes, +u64 *page_size); +int pds_vfio_dma_logging_stop (struct vfio_device *vdev); +#endif /* _DIRTY_H_ */
drivers/vfio/pci/pds/lm.c +1 -1
@@ -371,7 +371,7 @@ pds_vfio_step_device_state_locked(struct pds_vfio_pci_device *pds_vfio,if (cur ==VFIO_DEVICE_STATE_STOP_COPY && next ==VFIO_DEVICE_STATE_STOP ) {pds_vfio_put_save_file (pds_vfio); -pds_vfio_send_host_vf_lm_status_cmd (pds_vfio,PDS_LM_STA_NONE ); +pds_vfio_dirty_disable (pds_vfio, true);return NULL ; }
drivers/vfio/pci/pds/vfio_dev.c +11 -1
@@ -5,6 +5,7 @@#include <linux/vfio_pci_core.h> #include "lm.h" +#include "dirty.h" #include "vfio_dev.h" struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio)
@@ -25,7 +26,7 @@ struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev) vfio_coredev); } -static void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) +void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) { again:spin_lock (&pds_vfio->reset_lock);
@@ -35,6 +36,7 @@ static void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio) pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ;pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); +pds_vfio_dirty_disable (pds_vfio, false); }spin_unlock (&pds_vfio->reset_lock); goto again;
@@ -117,6 +119,12 @@ static const struct vfio_migration_ops pds_vfio_lm_ops = { .migration_get_data_size = pds_vfio_get_device_state_size }; +static const struct vfio_log_ops pds_vfio_log_ops = { + .log_start = pds_vfio_dma_logging_start, + .log_stop = pds_vfio_dma_logging_stop, + .log_read_and_clear = pds_vfio_dma_logging_report, +}; +static int pds_vfio_init_device (struct vfio_device *vdev) { struct pds_vfio_pci_device *pds_vfio =
@@ -137,6 +145,7 @@ static int pds_vfio_init_device(struct vfio_device *vdev) vdev->migration_flags =VFIO_MIGRATION_STOP_COPY |VFIO_MIGRATION_P2P ; vdev->mig_ops = &pds_vfio_lm_ops; + vdev->log_ops = &pds_vfio_log_ops; pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); dev_dbg(&pdev->dev,
@@ -175,6 +184,7 @@ static void pds_vfio_close_device(struct vfio_device *vdev)mutex_lock (&pds_vfio->state_mutex);pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); +pds_vfio_dirty_disable (pds_vfio, true);mutex_unlock (&pds_vfio->state_mutex);mutex_destroy (&pds_vfio->state_mutex);vfio_pci_core_close_device (vdev);
drivers/vfio/pci/pds/vfio_dev.h +4 -0
@@ -7,6 +7,7 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +#include "dirty.h" #include "lm.h" struct pdsc ;
@@ -17,6 +18,7 @@ struct pds_vfio_pci_device {struct pds_vfio_lm_file *save_file;struct pds_vfio_lm_file *restore_file; +struct pds_vfio_dirty dirty;struct mutex state_mutex;/* protect migration state */ enum vfio_device_mig_state state;spinlock_t reset_lock;/* protect reset_done flow */
@@ -26,6 +28,8 @@ struct pds_vfio_pci_device {u16 client_id; }; +void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio); +const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev);void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio);
include/linux/pds/pds_adminq.h +178 -0
202 lines not on this page · raw message
Signature · 1 line
2.17.1
RE: [PATCH v13 vfio 5/7] vfio/pds: Add support for dirty page tracking
Hi Brett,
Quoted 11 lines
-----Original Message-----
From: Brett Creeley [mailto:[email protected]]
Sent: 25 July 2023 22:40
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Shameerali Kolothum Thodi <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: [PATCH v13 vfio 5/7] vfio/pds: Add support for dirty page tracking
[...]
Quoted 14 lines
drivers/vfio/pci/pds/dirty.c new file +14
-0+static int pds_vfio_dirty_enable (struct pds_vfio_pci_device *pds_vfio, +struct rb_root_cached *ranges,u32 nnodes, +u64 *page_size) +{ +struct pci_dev *pdev = pds_vfio->vfio_coredev.pdev; +struct device *pdsc_dev = &pci_physfn (pdev)->dev; +struct pds_vfio_dirty *dirty = &pds_vfio->dirty; +u64 region_start, region_size, region_page_size; +struct pds_lm_dirty_region_info *region_info; +struct interval_tree_node *node =NULL ; +u8 max_regions =0 , num_regions; +dma_addr_t regions_dma =0 ; +u32 num_ranges = nnodes; +u32 page_count;
drivers/vfio/pci/pds/dirty.c line 201
Quoted 10 lines
drivers/vfio/pci/pds/dirty.c new file +10
-0+u16 len; +int err; + +dev_dbg (&pdev->dev,"vf%u: Start dirty page tracking\n" , + pds_vfio->vf_id); + +if (pds_vfio_dirty_is_enabled (pds_vfio)) +return -EINVAL ; + +pds_vfio_dirty_set_enabled (pds_vfio);
Any reason why this is set here? It looks to me you could set this at the end if everything goes well and avoid below goto out_set_disabled s. Not sure I am missing anything obvious here.
Thanks,
Shameer.
Quoted 771 lines
drivers/vfio/pci/pds/dirty.c new file +25
-0+ +/* find if dirty tracking is disabled, i.e. num_regions == 0 */ + err =pds_vfio_dirty_status_cmd (pds_vfio,0 , &max_regions, + &num_regions); +if (err <0 ) { +dev_err (&pdev->dev,"Failed to get dirty status, err %pe\n" , +ERR_PTR (err)); + goto out_set_disabled; + }else if (num_regions) { +dev_err (&pdev->dev, +"Dirty tracking already enabled for %d regions\n" , + num_regions); + err = -EEXIST ; + goto out_set_disabled; + }else if (!max_regions) { +dev_err (&pdev->dev, +"Device doesn't support dirty tracking, max_regions %d\n" , + max_regions); + err = -EOPNOTSUPP ; + goto out_set_disabled; + } + +/* +* Only support 1 region for now. If there are any large gaps in the +* VM's address regions, then this would be a waste of memory as we are
drivers/vfio/pci/pds/dirty.c new file +55
-0+* generating 2 bitmaps (ack/seq) from the min address to the max +* address of the VM's address regions. In the future, if we support +* more than one region in the device/driver we can split the bitmaps +* on the largest address region gaps. We can do this split up to the +* max_regions times returned from the dirty_status command. +*/ + max_regions =1 ; +if (num_ranges > max_regions) { +vfio_combine_iova_ranges (ranges, nnodes, max_regions); + num_ranges = max_regions; + } + + node =interval_tree_iter_first (ranges,0 ,ULONG_MAX ); +if (!node) { + err = -EINVAL ; + goto out_set_disabled; + } + + region_size = node->last - node->start +1 ; + region_start = node->start; + region_page_size = *page_size; + + len =sizeof (*region_info); + region_info =kzalloc (len,GFP_KERNEL ); +if (!region_info) { + err = -ENOMEM ; + goto out_set_disabled; + } + + page_count =DIV_ROUND_UP (region_size, region_page_size); + + region_info->dma_base =cpu_to_le64 (region_start); + region_info->page_count =cpu_to_le32 (page_count); + region_info->page_size_log2 =ilog2 (region_page_size); + + regions_dma =dma_map_single (pdsc_dev, (void *)region_info, len, +DMA_BIDIRECTIONAL ); +if (dma_mapping_error (pdsc_dev, regions_dma)) { + err = -ENOMEM ; + goto out_free_region_info; + } + + err =pds_vfio_dirty_enable_cmd (pds_vfio, regions_dma, max_regions); +dma_unmap_single (pdsc_dev, regions_dma, len,DMA_BIDIRECTIONAL ); +if (err) + goto out_free_region_info; + +/* +* page_count might be adjusted by the device, +* update it before freeing region_info DMA +*/ + page_count =le32_to_cpu (region_info->page_count); + + dev_dbg(&pdev->dev, + "region_info: regions_dma 0x%llx dma_base 0x%llx page_count %u
page_size_log2%u\n",drivers/vfio/pci/pds/dirty.c new file +37
-0+ regions_dma, region_start, page_count, + (u8)ilog2(region_page_size)); + + err =pds_vfio_dirty_alloc_bitmaps (dirty, page_count /BITS_PER_BYTE ); +if (err) { +dev_err (&pdev->dev,"Failed to alloc dirty bitmaps: %pe\n" , +ERR_PTR (err)); + goto out_free_region_info; + } + + err =pds_vfio_dirty_alloc_sgl (pds_vfio, page_count); +if (err) { +dev_err (&pdev->dev,"Failed to alloc dirty sg lists: %pe\n" , +ERR_PTR (err)); + goto out_free_bitmaps; + } + + dirty->region_start = region_start; + dirty->region_size = region_size; + dirty->region_page_size = region_page_size; + +pds_vfio_print_guest_region_info (pds_vfio, max_regions); + +kfree (region_info); + +return 0 ; + +out_free_bitmaps: +pds_vfio_dirty_free_bitmaps (dirty); +out_free_region_info: +kfree (region_info); +out_set_disabled: +pds_vfio_dirty_set_disabled (pds_vfio); +return err; +} + +void pds_vfio_dirty_disable(struct pds_vfio_pci_device *pds_vfio, bool
send_cmd)drivers/vfio/pci/pds/dirty.c new file +11
-0+{ +if (pds_vfio_dirty_is_enabled (pds_vfio)) { +pds_vfio_dirty_set_disabled (pds_vfio); +if (send_cmd) +pds_vfio_dirty_disable_cmd (pds_vfio); +pds_vfio_dirty_free_sgl (pds_vfio); +pds_vfio_dirty_free_bitmaps (&pds_vfio->dirty); + } + +if (send_cmd) + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_NONE);drivers/vfio/pci/pds/dirty.c new file +66
-0+} + +static int pds_vfio_dirty_seq_ack (struct pds_vfio_pci_device *pds_vfio, +struct pds_vfio_bmp_info *bmp_info, +u32 offset,u32 bmp_bytes,bool read_seq) +{ +const char *bmp_type_str = read_seq ?"read_seq" :"write_ack" ; +u8 dma_dir = read_seq ?DMA_FROM_DEVICE :DMA_TO_DEVICE ; +struct pci_dev *pdev = pds_vfio->vfio_coredev.pdev; +struct device *pdsc_dev = &pci_physfn (pdev)->dev; +unsigned long long npages; +struct sg_table sg_table; +struct scatterlist *sg; +struct page **pages; +u32 page_offset; +const void *bmp; +size_t size; +u16 num_sge; +int err; +int i; + + bmp = (void *)((u64 )bmp_info->bmp + offset); + page_offset =offset_in_page (bmp); + bmp -= page_offset; + +/* +* Start and end of bitmap section to seq/ack might not be page +* aligned, so use the page_offset to account for that so there +* will be enough pages to represent the bmp_bytes +*/ + npages =DIV_ROUND_UP_ULL (bmp_bytes + page_offset,PAGE_SIZE ); + pages =kmalloc_array (npages,sizeof (*pages),GFP_KERNEL ); +if (!pages) +return -ENOMEM ; + +for (unsigned long long i =0 ; i < npages; i++) { +struct page *page =vmalloc_to_page (bmp); + +if (!page) { + err = -EFAULT ; + goto out_free_pages; + } + + pages[i] = page; + bmp +=PAGE_SIZE ; + } + + err =sg_alloc_table_from_pages (&sg_table, pages, npages, page_offset, + bmp_bytes,GFP_KERNEL ); +if (err) + goto out_free_pages; + + err =dma_map_sgtable (pdsc_dev, &sg_table, dma_dir,0 ); +if (err) + goto out_free_sg_table; + +for_each_sgtable_dma_sg (&sg_table, sg, i) { +struct pds_lm_sg_elem *sg_elem = &bmp_info->sgl[i]; + + sg_elem->addr =cpu_to_le64 (sg_dma_address (sg)); + sg_elem->len =cpu_to_le32 (sg_dma_len (sg)); + } + + num_sge = sg_table.nents; + size = num_sge *sizeof (struct pds_lm_sg_elem ); + dma_sync_single_for_device(pdsc_dev, bmp_info->sgl_addr, size,
dma_dir);(quoted diff) +1
-0+ err = pds_vfio_dirty_seq_ack_cmd(pds_vfio, bmp_info->sgl_addr,
num_sge,drivers/vfio/pci/pds/dirty.c new file +4
-0+ offset, bmp_bytes, read_seq); +if (err) + dev_err(&pdev->dev, + "Dirty bitmap %s failed offset %u bmp_bytes %u num_sge %uDMA 0x%llx: %pe\n",
drivers/vfio/pci/pds/dirty.c new file +3
-0+ bmp_type_str, offset, bmp_bytes, + num_sge, bmp_info->sgl_addr, ERR_PTR(err)); + dma_sync_single_for_cpu(pdsc_dev, bmp_info->sgl_addr, size,
dma_dir);drivers/vfio/pci/pds/dirty.c new file +25
-0+ +dma_unmap_sgtable (pdsc_dev, &sg_table, dma_dir,0 ); +out_free_sg_table: +sg_free_table (&sg_table); +out_free_pages: +kfree (pages); + +return err; +} + +static int pds_vfio_dirty_write_ack (struct pds_vfio_pci_device *pds_vfio, +u32 offset,u32 len) +{ +return pds_vfio_dirty_seq_ack (pds_vfio, &pds_vfio->dirty.host_ack, + offset, len,WRITE_ACK ); +} + +static int pds_vfio_dirty_read_seq (struct pds_vfio_pci_device *pds_vfio, +u32 offset,u32 len) +{ +return pds_vfio_dirty_seq_ack (pds_vfio, &pds_vfio->dirty.host_seq, + offset, len,READ_SEQ ); +} + +static int pds_vfio_dirty_process_bitmaps(struct pds_vfio_pci_device
*pds_vfio,drivers/vfio/pci/pds/dirty.c new file +58
-0+struct iova_bitmap *dirty_bitmap, + u32 bmp_offset, u32 len_bytes) +{ + u64 page_size = pds_vfio->dirty.region_page_size; +u64 region_start = pds_vfio->dirty.region_start; +u32 bmp_offset_bit; +__le64 *seq, *ack; +int dword_count; + + dword_count = len_bytes /sizeof (u64); + seq = (__le64 *)((u64 )pds_vfio->dirty.host_seq.bmp + bmp_offset); + ack = (__le64 *)((u64 )pds_vfio->dirty.host_ack.bmp + bmp_offset); + bmp_offset_bit = bmp_offset *8 ; + +for (int i =0 ; i < dword_count; i++) { +u64 xor =le64_to_cpu (seq[i]) ^le64_to_cpu (ack[i]); + +/* prepare for next write_ack call */ + ack[i] = seq[i]; + +for (u8 bit_i =0 ; bit_i <BITS_PER_TYPE (u64); ++bit_i) { +if (xor &BIT (bit_i)) { +u64 abs_bit_i = bmp_offset_bit + + i *BITS_PER_TYPE (u64) + bit_i; +u64 addr = abs_bit_i * page_size + region_start; + +iova_bitmap_set (dirty_bitmap, addr, page_size); + } + } + } + +return 0 ; +} + +static int pds_vfio_dirty_sync (struct pds_vfio_pci_device *pds_vfio, +struct iova_bitmap *dirty_bitmap, +unsigned long iova,unsigned long length) +{ +struct device *dev = &pds_vfio->vfio_coredev.pdev->dev; +struct pds_vfio_dirty *dirty = &pds_vfio->dirty; +u64 bmp_offset, bmp_bytes; +u64 bitmap_size, pages; +int err; + +dev_dbg (dev,"vf%u: Get dirty page bitmap\n" , pds_vfio->vf_id); + +if (!pds_vfio_dirty_is_enabled (pds_vfio)) { +dev_err (dev,"vf%u: Sync failed, dirty tracking is disabled\n" , + pds_vfio->vf_id); +return -EINVAL ; + } + + pages =DIV_ROUND_UP (length, pds_vfio->dirty.region_page_size); + bitmap_size = +round_up (pages,sizeof (u64) *BITS_PER_BYTE ) /BITS_PER_BYTE ; + + dev_dbg(dev, + "vf%u: iova 0x%lx length %lu page_size %llu pages %llu
bitmap_size%llu\n",drivers/vfio/pci/pds/dirty.c new file +17
-0+ pds_vfio->vf_id, iova, length, pds_vfio->dirty.region_page_size, + pages, bitmap_size); + +if (!length || ((dirty->region_start + iova + length) > + (dirty->region_start + dirty->region_size))) { +dev_err (dev,"Invalid iova 0x%lx and/or length 0x%lx to sync\n" , + iova, length); +return -EINVAL ; + } + +/* bitmap is modified in 64 bit chunks */ + bmp_bytes =ALIGN (DIV_ROUND_UP (length / dirty->region_page_size, +sizeof (u64)), +sizeof (u64)); +if (bmp_bytes != bitmap_size) { + dev_err(dev, + "Calculated bitmap bytes %llu not equal to bitmapsize %llu\n",
drivers/vfio/pci/pds/dirty.c new file +5
-0+ bmp_bytes, bitmap_size); +return -EINVAL ; + } + + bmp_offset = DIV_ROUND_UP(iova / dirty->region_page_size,sizeof(u64));
drivers/vfio/pci/pds/cmds.c +3
-0+ + dev_dbg(dev, + "Syncing dirty bitmap, iova 0x%lx length 0x%lx, bmp_offset %llu
bmp_bytes%llu\n",drivers/vfio/pci/pds/dirty.c new file +7
-0+ iova, length, bmp_offset, bmp_bytes); + + err =pds_vfio_dirty_read_seq (pds_vfio, bmp_offset, bmp_bytes); +if (err) +return err; + + err = pds_vfio_dirty_process_bitmaps(pds_vfio, dirty_bitmap,
bmp_offset,drivers/vfio/pci/pds/dirty.c new file +12
-0+ bmp_bytes); +if (err) +return err; + + err =pds_vfio_dirty_write_ack (pds_vfio, bmp_offset, bmp_bytes); +if (err) +return err; + +return 0 ; +} + +int pds_vfio_dma_logging_report(struct vfio_device *vdev, unsigned longiova,
drivers/vfio/pci/pds/dirty.c new file +25
-0+ unsigned long length, struct iova_bitmap *dirty) +{ + struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); +int err; + +mutex_lock (&pds_vfio->state_mutex); + err =pds_vfio_dirty_sync (pds_vfio, dirty, iova, length); +pds_vfio_state_mutex_unlock (pds_vfio); + +return err; +} + +int pds_vfio_dma_logging_start (struct vfio_device *vdev, +struct rb_root_cached *ranges,u32 nnodes, +u64 *page_size) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); +int err; + +mutex_lock (&pds_vfio->state_mutex); + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_IN_PROGRESS);drivers/vfio/pci/pds/dirty.c new file +18
-0+ err =pds_vfio_dirty_enable (pds_vfio, ranges, nnodes, page_size); +pds_vfio_state_mutex_unlock (pds_vfio); + +return err; +} + +int pds_vfio_dma_logging_stop (struct vfio_device *vdev) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); + +mutex_lock (&pds_vfio->state_mutex); +pds_vfio_dirty_disable (pds_vfio, true); +pds_vfio_state_mutex_unlock (pds_vfio); + +return 0 ; +}drivers/vfio/pci/pds/dirty.h new file +32
-0@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _DIRTY_H_ +#define _DIRTY_H_ + +struct pds_vfio_bmp_info { +unsigned long *bmp; +u32 bmp_bytes; +struct pds_lm_sg_elem *sgl; +dma_addr_t sgl_addr; +u16 num_sge; +}; + +struct pds_vfio_dirty { +struct pds_vfio_bmp_info host_seq; +struct pds_vfio_bmp_info host_ack; +u64 region_size; +u64 region_start; +u64 region_page_size; +bool is_enabled; +}; + +struct pds_vfio_pci_device ; + +bool pds_vfio_dirty_is_enabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_set_enabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_set_disabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_disable (struct pds_vfio_pci_device *pds_vfio, +bool send_cmd); + +int pds_vfio_dma_logging_report(struct vfio_device *vdev, unsigned longiova,
drivers/vfio/pci/pds/dirty.h new file +7
-0+unsigned long length, + struct iova_bitmap *dirty); +int pds_vfio_dma_logging_start (struct vfio_device *vdev, +struct rb_root_cached *ranges,u32 nnodes, +u64 *page_size); +int pds_vfio_dma_logging_stop (struct vfio_device *vdev); +#endif /* _DIRTY_H_ */ drivers/vfio/pci/pds/lm.c +1
-1@@ -371,7 +371,7 @@ pds_vfio_step_device_state_locked(struct pds_vfio_pci_device *pds_vfio, if (cur == VFIO_DEVICE_STATE_STOP_COPY && next == VFIO_DEVICE_STATE_STOP) {pds_vfio_put_save_file (pds_vfio); -pds_vfio_send_host_vf_lm_status_cmd (pds_vfio, PDS_LM_STA_NONE); +pds_vfio_dirty_disable (pds_vfio, true);drivers/vfio/pci/pds/lm.c +0
-0return NULL ; }drivers/vfio/pci/pds/vfio_dev.c +2
-1@@ -5,6 +5,7 @@#include <linux/vfio_pci_core.h> #include "lm.h" +#include "dirty.h" #include "vfio_dev.h" struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio)@@ -25,7 +26,7 @@ struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev) vfio_coredev); } -static void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) +void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) {again:
drivers/vfio/pci/pds/vfio_dev.c +1
-0spin_lock (&pds_vfio->reset_lock);@@ -35,6 +36,7 @@ static void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio) pds_vfio->state = VFIO_DEVICE_STATE_RUNNING;pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); +pds_vfio_dirty_disable (pds_vfio, false); }spin_unlock (&pds_vfio->reset_lock);drivers/vfio/pci/pds/vfio_dev.c +6
-0goto again;@@ -117,6 +119,12 @@ static const struct vfio_migration_ops pds_vfio_lm_ops = { .migration_get_data_size = pds_vfio_get_device_state_size }; +static const struct vfio_log_ops pds_vfio_log_ops = { + .log_start = pds_vfio_dma_logging_start, + .log_stop = pds_vfio_dma_logging_stop, + .log_read_and_clear = pds_vfio_dma_logging_report, +}; +static int pds_vfio_init_device (struct vfio_device *vdev) {drivers/vfio/pci/pds/vfio_dev.c +1
-0struct pds_vfio_pci_device *pds_vfio =@@ -137,6 +145,7 @@ static int pds_vfio_init_device(struct vfio_device *vdev) vdev->migration_flags =VFIO_MIGRATION_STOP_COPY |VFIO_MIGRATION_P2P ; vdev->mig_ops = &pds_vfio_lm_ops; + vdev->log_ops = &pds_vfio_log_ops;drivers/vfio/pci/pds/vfio_dev.c +1
-0pci_id =PCI_DEVID (pdev->bus->number, pdev->devfn); dev_dbg(&pdev->dev,@@ -175,6 +184,7 @@ static void pds_vfio_close_device(struct vfio_device *vdev)mutex_lock (&pds_vfio->state_mutex);pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); +pds_vfio_dirty_disable (pds_vfio, true);mutex_unlock (&pds_vfio->state_mutex);mutex_destroy (&pds_vfio->state_mutex);drivers/vfio/pci/pds/vfio_dev.c +0
-0vfio_pci_core_close_device (vdev);drivers/vfio/pci/pds/vfio_dev.h +4
-0@@ -7,6 +7,7 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +#include "dirty.h" #include "lm.h" struct pdsc ;@@ -17,6 +18,7 @@ struct pds_vfio_pci_device {struct pds_vfio_lm_file *save_file;struct pds_vfio_lm_file *restore_file; +struct pds_vfio_dirty dirty;struct mutex state_mutex;/* protect migration state */ enum vfio_device_mig_state state;spinlock_t reset_lock;/* protect reset_done flow */ @@ -26,6 +28,8 @@ struct pds_vfio_pci_device {u16 client_id; }; +void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio); +const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev);void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio);include/linux/pds/pds_adminq.h +9
-0@@ -835,6 +835,13 @@ enum pds_lm_cmd_opcode {PDS_LM_CMD_RESUME =20 ,PDS_LM_CMD_SAVE =21 ,PDS_LM_CMD_RESTORE =22 , + +/* Dirty page tracking commands */ +PDS_LM_CMD_DIRTY_STATUS =32 , +PDS_LM_CMD_DIRTY_ENABLE =33 , +PDS_LM_CMD_DIRTY_DISABLE =34 , +PDS_LM_CMD_DIRTY_READ_SEQ =35 , + PDS_LM_CMD_DIRTY_WRITE_ACK = 36, };/** @@ -992,6 +999,172 @@ enum pds_lm_host_vf_status { PDS_LM_STA_MAX, }; +/** +* struct pds_lm_dirty_region_info - Memory region info for STATUS and ENABLE
+ * @dma_base: Base address of the DMA-contiguous memory region
+ * @page_count: Number of pages in the memory region
+ * @page_size_log2: Log2 page size in the memory region
+ * @rsvd: Word boundary padding
+ */include/linux/pds/pds_adminq.h +17
-0+struct pds_lm_dirty_region_info { +__le64 dma_base; +__le32 page_count; +u8 page_size_log2; +u8 rsvd[3 ]; +}; + +/** +* struct pds_lm_dirty_status_cmd - DIRTY_STATUS command +* @opcode: Opcode PDS_LM_CMD_DIRTY_STATUS +* @rsvd: Word boundary padding +* @vf_id: VF id +* @max_regions: Capacity of the region info buffer +* @rsvd2: Word boundary padding +* @regions_dma: DMA address of the region info buffer +* +* The minimum of max_regions (from the command) and num_regions (from the
+ * completion) of structpds_lm_dirty_region_infowill be written to
+ *regions_dma.
+ *
+ * Themax_regionsmay be zero, in which caseregions_dmais ignored. In that
+ * case, the completion will only report the maximum number of regions
+ * supported by the device, and the number of regions currently enabled.
+ */include/linux/pds/pds_adminq.h +24
-0+struct pds_lm_dirty_status_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 max_regions; +u8 rsvd2[3 ]; +__le64 regions_dma; +} __packed; + +/** +* enum pds_lm_dirty_bmp_type - Type of dirty page bitmap +* @PDS_LM_DIRTY_BMP_TYPE_NONE: No bitmap / disabled +* @PDS_LM_DIRTY_BMP_TYPE_SEQ_ACK: Seq/Ack bitmap representation +*/ +enum pds_lm_dirty_bmp_type { +PDS_LM_DIRTY_BMP_TYPE_NONE =0 , +PDS_LM_DIRTY_BMP_TYPE_SEQ_ACK =1 , +}; + +/** +* struct pds_lm_dirty_status_comp - STATUS command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Word boundary padding +* @comp_index: Index in the desc ring for which this is the completion
+ * @max_regions: Maximum number of regions supported by the
device
+ * @num_regions: Number of regions currently enabled
+ * @bmp_type: Type of dirty bitmap representation
+ * @rsvd2: Word boundary padding
+ * @bmp_type_mask: Mask of supported bitmap types, bit index per type
+ * @rsvd3: Word boundary padding
+ * @color: Color bit
+ *
+ * This completion descriptor is used for STATUS, ENABLE, and DISABLE.
+ */include/linux/pds/pds_adminq.h +24
-0+struct pds_lm_dirty_status_comp { +u8 status; +u8 rsvd; +__le16 comp_index; +u8 max_regions; +u8 num_regions; +u8 bmp_type; +u8 rsvd2; +__le32 bmp_type_mask; +u8 rsvd3[3 ]; +u8 color; +}; + +/** +* struct pds_lm_dirty_enable_cmd - DIRTY_ENABLE command +* @opcode: Opcode PDS_LM_CMD_DIRTY_ENABLE +* @rsvd: Word boundary padding +* @vf_id: VF id +* @bmp_type: Type of dirty bitmap representation +* @num_regions: Number of entries in the region info buffer +* @rsvd2: Word boundary padding +* @regions_dma: DMA address of the region info buffer +* +* The num_regions must be nonzero, and less than or equal to the maximum
+ * number of regions supported by the device.
+ *
+ * The memory regions should not overlap.
+ *
+ * The information should be initialized by the driver. The device may modify
+ * the information on successful completion, such as by size-aligning the
+ * number of pages in a region.
+ *
+ * The modified number of pages will be greater than or equal to the page count
+ * given in the enable command, and at least as coarsly aligned as the given
+ * value. For example, the count might be aligned to a multiple of 64, but
+ * if the value is already a multiple of 128 or higher, it will not change.
+ * If the driver requires its own minimum alignment of the number of pages, the
+ * driver should account for that already in the region info of this command.
+ *
+ * This command uses structpds_lm_dirty_status_compfor its completion.
+ */include/linux/pds/pds_adminq.h +21
-0+struct pds_lm_dirty_enable_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 bmp_type; +u8 num_regions; +u8 rsvd2[2 ]; +__le64 regions_dma; +} __packed; + +/** +* struct pds_lm_dirty_disable_cmd - DIRTY_DISABLE command +* @opcode: Opcode PDS_LM_CMD_DIRTY_DISABLE +* @rsvd: Word boundary padding +* @vf_id: VF id +* +* Dirty page tracking will be disabled. This may be called in any state, as +* long as dirty page tracking is supported by the device, to ensure that dirty +* page tracking is disabled. +* +* This command uses struct pds_lm_dirty_status_comp for its completion. On
+ * success,num_regionswill be zero.
+ */include/linux/pds/pds_adminq.h +8
-0+struct pds_lm_dirty_disable_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_dirty_seq_ack_cmd - DIRTY_READ_SEQ or _WRITE_ACK command
+ * @opcode: Opcode PDS_LM_CMD_DIRTY_[READ_SEQ|WRITE_ACK]
+ * @rsvd: Word boundary padding
+ * @vf_id: VF id
+ * @off_bytes: Byte offset in the bitmap
+ * @len_bytes: Number of bytes to transfer
+ * @num_sge: Number of DMA scatter gather elements
+ * @rsvd2: Word boundary padding
+ * @sgl_addr: DMA address of scatter gather list
+ *
+ * Read bytes from the SEQ bitmap, or write bytes into the ACK bitmap.
+ *
+ * This command treats the entire bitmap as a byte buffer. It does not
+ * distinguish between guest memory regions. The driver should refer to the
+ * number of pages in each region, according toPDS_LM_CMD_DIRTY_STATUS, to
+ * determine the region boundaries in the bitmap. Each region will be
+ * represented by exactly the number of bits as the page count for that region,
+ * immediately following the last bit of the previous region.
+ */include/linux/pds/pds_adminq.h +16
-1+struct pds_lm_dirty_seq_ack_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +__le32 off_bytes; +__le32 len_bytes; +__le16 num_sge; +u8 rsvd2[2 ]; +__le64 sgl_addr; +} __packed; +/** * struct pds_lm_host_vf_status_cmd - HOST_VF_STATUS command * @opcode: Opcode PDS_LM_CMD_HOST_VF_STATUS @@ -1039,6 +1212,10 @@ union pds_core_adminq_cmd {struct pds_lm_save_cmd lm_save;struct pds_lm_restore_cmd lm_restore;struct pds_lm_host_vf_status_cmd lm_host_vf_status; +struct pds_lm_dirty_status_cmd lm_dirty_status; +struct pds_lm_dirty_enable_cmd lm_dirty_enable; +struct pds_lm_dirty_disable_cmd lm_dirty_disable; +struct pds_lm_dirty_seq_ack_cmd lm_dirty_seq_ack; };union pds_core_adminq_comp {@@ -1065,6 +1242,7 @@ union pds_core_adminq_comp {struct pds_vdpa_vq_reset_comp vdpa_vq_reset;struct pds_lm_state_size_comp lm_state_size; +struct pds_lm_dirty_status_comp lm_dirty_status; };#ifndef __CHECKER__ --2.17.1
Re: [PATCH v13 vfio 5/7] vfio/pds: Add support for dirty page tracking
Quoted 47 lines · Shameerali Kolothum Thodi
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Hi Brett,
Quoted 11 lines
-----Original Message-----
From: Brett Creeley [mailto:[email protected]]
Sent: 25 July 2023 22:40
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Shameerali Kolothum Thodi <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: [PATCH v13 vfio 5/7] vfio/pds: Add support for dirty page tracking[...]
Quoted 14 lines
drivers/vfio/pci/pds/dirty.c new file +14
-0+static int pds_vfio_dirty_enable (struct pds_vfio_pci_device *pds_vfio, +struct rb_root_cached *ranges,u32 nnodes, +u64 *page_size) +{ +struct pci_dev *pdev = pds_vfio->vfio_coredev.pdev; +struct device *pdsc_dev = &pci_physfn (pdev)->dev; +struct pds_vfio_dirty *dirty = &pds_vfio->dirty; +u64 region_start, region_size, region_page_size; +struct pds_lm_dirty_region_info *region_info; +struct interval_tree_node *node =NULL ; +u8 max_regions =0 , num_regions; +dma_addr_t regions_dma =0 ; +u32 num_ranges = nnodes; +u32 page_count;drivers/vfio/pci/pds/dirty.c line 201
Quoted 10 lines
drivers/vfio/pci/pds/dirty.c new file +10
-0+u16 len; +int err; + +dev_dbg (&pdev->dev,"vf%u: Start dirty page tracking\n" , + pds_vfio->vf_id); + +if (pds_vfio_dirty_is_enabled (pds_vfio)) +return -EINVAL ; + +pds_vfio_dirty_set_enabled (pds_vfio);Any reason why this is set here? It looks to me you could set this at the end if everything goes well and avoid below goto
out_set_disableds. Not sure I am missing anything obvious here.
Quoted 2 lines
Thanks,
Shameer.
No, this is a good catch and simplifies this function a bit. I will respin and fix this.
Thanks for the review,
Brett
Quoted 772 lines
Quoted 771 lines
drivers/vfio/pci/pds/dirty.c new file +25
-0+ +/* find if dirty tracking is disabled, i.e. num_regions == 0 */ + err =pds_vfio_dirty_status_cmd (pds_vfio,0 , &max_regions, + &num_regions); +if (err <0 ) { +dev_err (&pdev->dev,"Failed to get dirty status, err %pe\n" , +ERR_PTR (err)); + goto out_set_disabled; + }else if (num_regions) { +dev_err (&pdev->dev, +"Dirty tracking already enabled for %d regions\n" , + num_regions); + err = -EEXIST ; + goto out_set_disabled; + }else if (!max_regions) { +dev_err (&pdev->dev, +"Device doesn't support dirty tracking, max_regions %d\n" , + max_regions); + err = -EOPNOTSUPP ; + goto out_set_disabled; + } + +/* +* Only support 1 region for now. If there are any large gaps in the +* VM's address regions, then this would be a waste of memory as we are
drivers/vfio/pci/pds/dirty.c new file +55
-0+* generating 2 bitmaps (ack/seq) from the min address to the max +* address of the VM's address regions. In the future, if we support +* more than one region in the device/driver we can split the bitmaps +* on the largest address region gaps. We can do this split up to the +* max_regions times returned from the dirty_status command. +*/ + max_regions =1 ; +if (num_ranges > max_regions) { +vfio_combine_iova_ranges (ranges, nnodes, max_regions); + num_ranges = max_regions; + } + + node =interval_tree_iter_first (ranges,0 ,ULONG_MAX ); +if (!node) { + err = -EINVAL ; + goto out_set_disabled; + } + + region_size = node->last - node->start +1 ; + region_start = node->start; + region_page_size = *page_size; + + len =sizeof (*region_info); + region_info =kzalloc (len,GFP_KERNEL ); +if (!region_info) { + err = -ENOMEM ; + goto out_set_disabled; + } + + page_count =DIV_ROUND_UP (region_size, region_page_size); + + region_info->dma_base =cpu_to_le64 (region_start); + region_info->page_count =cpu_to_le32 (page_count); + region_info->page_size_log2 =ilog2 (region_page_size); + + regions_dma =dma_map_single (pdsc_dev, (void *)region_info, len, +DMA_BIDIRECTIONAL ); +if (dma_mapping_error (pdsc_dev, regions_dma)) { + err = -ENOMEM ; + goto out_free_region_info; + } + + err =pds_vfio_dirty_enable_cmd (pds_vfio, regions_dma, max_regions); +dma_unmap_single (pdsc_dev, regions_dma, len,DMA_BIDIRECTIONAL ); +if (err) + goto out_free_region_info; + +/* +* page_count might be adjusted by the device, +* update it before freeing region_info DMA +*/ + page_count =le32_to_cpu (region_info->page_count); + + dev_dbg(&pdev->dev, + "region_info: regions_dma 0x%llx dma_base 0x%llx page_count %u
page_size_log2%u\n",drivers/vfio/pci/pds/dirty.c new file +37
-0+ regions_dma, region_start, page_count, + (u8)ilog2(region_page_size)); + + err =pds_vfio_dirty_alloc_bitmaps (dirty, page_count /BITS_PER_BYTE ); +if (err) { +dev_err (&pdev->dev,"Failed to alloc dirty bitmaps: %pe\n" , +ERR_PTR (err)); + goto out_free_region_info; + } + + err =pds_vfio_dirty_alloc_sgl (pds_vfio, page_count); +if (err) { +dev_err (&pdev->dev,"Failed to alloc dirty sg lists: %pe\n" , +ERR_PTR (err)); + goto out_free_bitmaps; + } + + dirty->region_start = region_start; + dirty->region_size = region_size; + dirty->region_page_size = region_page_size; + +pds_vfio_print_guest_region_info (pds_vfio, max_regions); + +kfree (region_info); + +return 0 ; + +out_free_bitmaps: +pds_vfio_dirty_free_bitmaps (dirty); +out_free_region_info: +kfree (region_info); +out_set_disabled: +pds_vfio_dirty_set_disabled (pds_vfio); +return err; +} + +void pds_vfio_dirty_disable(struct pds_vfio_pci_device *pds_vfio, bool
send_cmd)drivers/vfio/pci/pds/dirty.c new file +11
-0+{ +if (pds_vfio_dirty_is_enabled (pds_vfio)) { +pds_vfio_dirty_set_disabled (pds_vfio); +if (send_cmd) +pds_vfio_dirty_disable_cmd (pds_vfio); +pds_vfio_dirty_free_sgl (pds_vfio); +pds_vfio_dirty_free_bitmaps (&pds_vfio->dirty); + } + +if (send_cmd) + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_NONE);drivers/vfio/pci/pds/dirty.c new file +66
-0+} + +static int pds_vfio_dirty_seq_ack (struct pds_vfio_pci_device *pds_vfio, +struct pds_vfio_bmp_info *bmp_info, +u32 offset,u32 bmp_bytes,bool read_seq) +{ +const char *bmp_type_str = read_seq ?"read_seq" :"write_ack" ; +u8 dma_dir = read_seq ?DMA_FROM_DEVICE :DMA_TO_DEVICE ; +struct pci_dev *pdev = pds_vfio->vfio_coredev.pdev; +struct device *pdsc_dev = &pci_physfn (pdev)->dev; +unsigned long long npages; +struct sg_table sg_table; +struct scatterlist *sg; +struct page **pages; +u32 page_offset; +const void *bmp; +size_t size; +u16 num_sge; +int err; +int i; + + bmp = (void *)((u64 )bmp_info->bmp + offset); + page_offset =offset_in_page (bmp); + bmp -= page_offset; + +/* +* Start and end of bitmap section to seq/ack might not be page +* aligned, so use the page_offset to account for that so there +* will be enough pages to represent the bmp_bytes +*/ + npages =DIV_ROUND_UP_ULL (bmp_bytes + page_offset,PAGE_SIZE ); + pages =kmalloc_array (npages,sizeof (*pages),GFP_KERNEL ); +if (!pages) +return -ENOMEM ; + +for (unsigned long long i =0 ; i < npages; i++) { +struct page *page =vmalloc_to_page (bmp); + +if (!page) { + err = -EFAULT ; + goto out_free_pages; + } + + pages[i] = page; + bmp +=PAGE_SIZE ; + } + + err =sg_alloc_table_from_pages (&sg_table, pages, npages, page_offset, + bmp_bytes,GFP_KERNEL ); +if (err) + goto out_free_pages; + + err =dma_map_sgtable (pdsc_dev, &sg_table, dma_dir,0 ); +if (err) + goto out_free_sg_table; + +for_each_sgtable_dma_sg (&sg_table, sg, i) { +struct pds_lm_sg_elem *sg_elem = &bmp_info->sgl[i]; + + sg_elem->addr =cpu_to_le64 (sg_dma_address (sg)); + sg_elem->len =cpu_to_le32 (sg_dma_len (sg)); + } + + num_sge = sg_table.nents; + size = num_sge *sizeof (struct pds_lm_sg_elem ); + dma_sync_single_for_device(pdsc_dev, bmp_info->sgl_addr, size,
dma_dir);
+ err = pds_vfio_dirty_seq_ack_cmd(pds_vfio,bmp_info->sgl_addr,num_sge,drivers/vfio/pci/pds/dirty.c new file +4
-0+ offset, bmp_bytes, read_seq); +if (err) + dev_err(&pdev->dev, + "Dirty bitmap %s failed offset %u bmp_bytes %u num_sge %uDMA 0x%llx: %pe\n",
drivers/vfio/pci/pds/dirty.c new file +3
-0+ bmp_type_str, offset, bmp_bytes, + num_sge, bmp_info->sgl_addr, ERR_PTR(err)); + dma_sync_single_for_cpu(pdsc_dev, bmp_info->sgl_addr, size,
dma_dir);drivers/vfio/pci/pds/dirty.c new file +25
-0+ +dma_unmap_sgtable (pdsc_dev, &sg_table, dma_dir,0 ); +out_free_sg_table: +sg_free_table (&sg_table); +out_free_pages: +kfree (pages); + +return err; +} + +static int pds_vfio_dirty_write_ack (struct pds_vfio_pci_device *pds_vfio, +u32 offset,u32 len) +{ +return pds_vfio_dirty_seq_ack (pds_vfio, &pds_vfio->dirty.host_ack, + offset, len,WRITE_ACK ); +} + +static int pds_vfio_dirty_read_seq (struct pds_vfio_pci_device *pds_vfio, +u32 offset,u32 len) +{ +return pds_vfio_dirty_seq_ack (pds_vfio, &pds_vfio->dirty.host_seq, + offset, len,READ_SEQ ); +} + +static int pds_vfio_dirty_process_bitmaps(struct pds_vfio_pci_device
*pds_vfio,drivers/vfio/pci/pds/dirty.c new file +58
-0+struct iova_bitmap *dirty_bitmap, + u32 bmp_offset, u32 len_bytes) +{ + u64 page_size = pds_vfio->dirty.region_page_size; +u64 region_start = pds_vfio->dirty.region_start; +u32 bmp_offset_bit; +__le64 *seq, *ack; +int dword_count; + + dword_count = len_bytes /sizeof (u64); + seq = (__le64 *)((u64 )pds_vfio->dirty.host_seq.bmp + bmp_offset); + ack = (__le64 *)((u64 )pds_vfio->dirty.host_ack.bmp + bmp_offset); + bmp_offset_bit = bmp_offset *8 ; + +for (int i =0 ; i < dword_count; i++) { +u64 xor =le64_to_cpu (seq[i]) ^le64_to_cpu (ack[i]); + +/* prepare for next write_ack call */ + ack[i] = seq[i]; + +for (u8 bit_i =0 ; bit_i <BITS_PER_TYPE (u64); ++bit_i) { +if (xor &BIT (bit_i)) { +u64 abs_bit_i = bmp_offset_bit + + i *BITS_PER_TYPE (u64) + bit_i; +u64 addr = abs_bit_i * page_size + region_start; + +iova_bitmap_set (dirty_bitmap, addr, page_size); + } + } + } + +return 0 ; +} + +static int pds_vfio_dirty_sync (struct pds_vfio_pci_device *pds_vfio, +struct iova_bitmap *dirty_bitmap, +unsigned long iova,unsigned long length) +{ +struct device *dev = &pds_vfio->vfio_coredev.pdev->dev; +struct pds_vfio_dirty *dirty = &pds_vfio->dirty; +u64 bmp_offset, bmp_bytes; +u64 bitmap_size, pages; +int err; + +dev_dbg (dev,"vf%u: Get dirty page bitmap\n" , pds_vfio->vf_id); + +if (!pds_vfio_dirty_is_enabled (pds_vfio)) { +dev_err (dev,"vf%u: Sync failed, dirty tracking is disabled\n" , + pds_vfio->vf_id); +return -EINVAL ; + } + + pages =DIV_ROUND_UP (length, pds_vfio->dirty.region_page_size); + bitmap_size = +round_up (pages,sizeof (u64) *BITS_PER_BYTE ) /BITS_PER_BYTE ; + + dev_dbg(dev, + "vf%u: iova 0x%lx length %lu page_size %llu pages %llu
bitmap_size%llu\n",drivers/vfio/pci/pds/dirty.c new file +17
-0+ pds_vfio->vf_id, iova, length, pds_vfio->dirty.region_page_size, + pages, bitmap_size); + +if (!length || ((dirty->region_start + iova + length) > + (dirty->region_start + dirty->region_size))) { +dev_err (dev,"Invalid iova 0x%lx and/or length 0x%lx to sync\n" , + iova, length); +return -EINVAL ; + } + +/* bitmap is modified in 64 bit chunks */ + bmp_bytes =ALIGN (DIV_ROUND_UP (length / dirty->region_page_size, +sizeof (u64)), +sizeof (u64)); +if (bmp_bytes != bitmap_size) { + dev_err(dev, + "Calculated bitmap bytes %llu not equal to bitmapsize %llu\n",
drivers/vfio/pci/pds/dirty.c new file +5
-0+ bmp_bytes, bitmap_size); +return -EINVAL ; + } + + bmp_offset = DIV_ROUND_UP(iova / dirty->region_page_size,sizeof(u64));
drivers/vfio/pci/pds/cmds.c +3
-0+ + dev_dbg(dev, + "Syncing dirty bitmap, iova 0x%lx length 0x%lx, bmp_offset %llu
bmp_bytes%llu\n",drivers/vfio/pci/pds/dirty.c new file +7
-0+ iova, length, bmp_offset, bmp_bytes); + + err =pds_vfio_dirty_read_seq (pds_vfio, bmp_offset, bmp_bytes); +if (err) +return err; + + err = pds_vfio_dirty_process_bitmaps(pds_vfio, dirty_bitmap,
bmp_offset,drivers/vfio/pci/pds/dirty.c new file +12
-0+ bmp_bytes); +if (err) +return err; + + err =pds_vfio_dirty_write_ack (pds_vfio, bmp_offset, bmp_bytes); +if (err) +return err; + +return 0 ; +} + +int pds_vfio_dma_logging_report(struct vfio_device *vdev, unsigned longiova,
drivers/vfio/pci/pds/dirty.c new file +25
-0+ unsigned long length, struct iova_bitmap *dirty) +{ + struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); +int err; + +mutex_lock (&pds_vfio->state_mutex); + err =pds_vfio_dirty_sync (pds_vfio, dirty, iova, length); +pds_vfio_state_mutex_unlock (pds_vfio); + +return err; +} + +int pds_vfio_dma_logging_start (struct vfio_device *vdev, +struct rb_root_cached *ranges,u32 nnodes, +u64 *page_size) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); +int err; + +mutex_lock (&pds_vfio->state_mutex); + pds_vfio_send_host_vf_lm_status_cmd(pds_vfio,
PDS_LM_STA_IN_PROGRESS);drivers/vfio/pci/pds/dirty.c new file +18
-0+ err =pds_vfio_dirty_enable (pds_vfio, ranges, nnodes, page_size); +pds_vfio_state_mutex_unlock (pds_vfio); + +return err; +} + +int pds_vfio_dma_logging_stop (struct vfio_device *vdev) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(vdev, struct pds_vfio_pci_device, + vfio_coredev.vdev); + +mutex_lock (&pds_vfio->state_mutex); +pds_vfio_dirty_disable (pds_vfio, true); +pds_vfio_state_mutex_unlock (pds_vfio); + +return 0 ; +}drivers/vfio/pci/pds/dirty.h new file +32
-0@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc. */ + +#ifndef _DIRTY_H_ +#define _DIRTY_H_ + +struct pds_vfio_bmp_info { +unsigned long *bmp; +u32 bmp_bytes; +struct pds_lm_sg_elem *sgl; +dma_addr_t sgl_addr; +u16 num_sge; +}; + +struct pds_vfio_dirty { +struct pds_vfio_bmp_info host_seq; +struct pds_vfio_bmp_info host_ack; +u64 region_size; +u64 region_start; +u64 region_page_size; +bool is_enabled; +}; + +struct pds_vfio_pci_device ; + +bool pds_vfio_dirty_is_enabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_set_enabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_set_disabled (struct pds_vfio_pci_device *pds_vfio); +void pds_vfio_dirty_disable (struct pds_vfio_pci_device *pds_vfio, +bool send_cmd); + +int pds_vfio_dma_logging_report(struct vfio_device *vdev, unsigned longiova,
drivers/vfio/pci/pds/dirty.h new file +7
-0+unsigned long length, + struct iova_bitmap *dirty); +int pds_vfio_dma_logging_start (struct vfio_device *vdev, +struct rb_root_cached *ranges,u32 nnodes, +u64 *page_size); +int pds_vfio_dma_logging_stop (struct vfio_device *vdev); +#endif /* _DIRTY_H_ */ drivers/vfio/pci/pds/lm.c +1
-1@@ -371,7 +371,7 @@ pds_vfio_step_device_state_locked(struct pds_vfio_pci_device *pds_vfio, if (cur == VFIO_DEVICE_STATE_STOP_COPY && next == VFIO_DEVICE_STATE_STOP) {pds_vfio_put_save_file (pds_vfio); -pds_vfio_send_host_vf_lm_status_cmd (pds_vfio, PDS_LM_STA_NONE); +pds_vfio_dirty_disable (pds_vfio, true);return NULL; }drivers/vfio/pci/pds/vfio_dev.c +2
-1@@ -5,6 +5,7 @@#include <linux/vfio_pci_core.h> #include "lm.h" +#include "dirty.h" #include "vfio_dev.h" struct pci_dev *pds_vfio_to_pci_dev (struct pds_vfio_pci_device *pds_vfio)@@ -25,7 +26,7 @@ struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev) vfio_coredev); } -static void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) +void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio) {again: spin_lock(&pds_vfio->reset_lock);drivers/vfio/pci/pds/vfio_dev.c +1
-0@@ -35,6 +36,7 @@ static void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio) pds_vfio->state = VFIO_DEVICE_STATE_RUNNING;pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); +pds_vfio_dirty_disable (pds_vfio, false); }spin_unlock (&pds_vfio->reset_lock);goto again;drivers/vfio/pci/pds/vfio_dev.c +6
-0@@ -117,6 +119,12 @@ static const struct vfio_migration_ops pds_vfio_lm_ops = { .migration_get_data_size = pds_vfio_get_device_state_size }; +static const struct vfio_log_ops pds_vfio_log_ops = { + .log_start = pds_vfio_dma_logging_start, + .log_stop = pds_vfio_dma_logging_stop, + .log_read_and_clear = pds_vfio_dma_logging_report, +}; +static int pds_vfio_init_device (struct vfio_device *vdev) {struct
pds_vfio_pci_device*pds_vfio=drivers/vfio/pci/pds/vfio_dev.c +1
-0@@ -137,6 +145,7 @@ static int pds_vfio_init_device(struct vfio_device *vdev) vdev->migration_flags =VFIO_MIGRATION_STOP_COPY |VFIO_MIGRATION_P2P ; vdev->mig_ops = &pds_vfio_lm_ops; + vdev->log_ops = &pds_vfio_log_ops;pci_id = PCI_DEVID(pdev->bus->number, pdev->devfn); dev_dbg(&pdev->dev,drivers/vfio/pci/pds/vfio_dev.c +1
-0@@ -175,6 +184,7 @@ static void pds_vfio_close_device(struct vfio_device *vdev)mutex_lock (&pds_vfio->state_mutex);pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio); +pds_vfio_dirty_disable (pds_vfio, true);mutex_unlock (&pds_vfio->state_mutex);mutex_destroy (&pds_vfio->state_mutex);vfio_pci_core_close_device(vdev);drivers/vfio/pci/pds/vfio_dev.h +4
-0@@ -7,6 +7,7 @@#include <linux/pci.h> #include <linux/vfio_pci_core.h> +#include "dirty.h" #include "lm.h" struct pdsc ;@@ -17,6 +18,7 @@ struct pds_vfio_pci_device {struct pds_vfio_lm_file *save_file;struct pds_vfio_lm_file *restore_file; +struct pds_vfio_dirty dirty;struct mutex state_mutex;/* protect migration state */ enum vfio_device_mig_state state;spinlock_t reset_lock;/* protect reset_done flow */ @@ -26,6 +28,8 @@ struct pds_vfio_pci_device {u16 client_id; }; +void pds_vfio_state_mutex_unlock (struct pds_vfio_pci_device *pds_vfio); +const struct vfio_device_ops *pds_vfio_ops_info (void );struct pds_vfio_pci_device *pds_vfio_pci_drvdata (struct pci_dev *pdev);void pds_vfio_reset (struct pds_vfio_pci_device *pds_vfio);include/linux/pds/pds_adminq.h +9
-0@@ -835,6 +835,13 @@ enum pds_lm_cmd_opcode {PDS_LM_CMD_RESUME =20 ,PDS_LM_CMD_SAVE =21 ,PDS_LM_CMD_RESTORE =22 , + +/* Dirty page tracking commands */ +PDS_LM_CMD_DIRTY_STATUS =32 , +PDS_LM_CMD_DIRTY_ENABLE =33 , +PDS_LM_CMD_DIRTY_DISABLE =34 , +PDS_LM_CMD_DIRTY_READ_SEQ =35 , + PDS_LM_CMD_DIRTY_WRITE_ACK = 36, };/** @@ -992,6 +999,172 @@ enum pds_lm_host_vf_status { PDS_LM_STA_MAX, }; +/** +* struct pds_lm_dirty_region_info - Memory region info for STATUS and ENABLE
+ * @dma_base: Base address of the DMA-contiguous memory region
+ * @page_count: Number of pages in the memory region
+ * @page_size_log2: Log2 page size in the memory region
+ * @rsvd: Word boundary padding
+ */include/linux/pds/pds_adminq.h +17
-0+struct pds_lm_dirty_region_info { +__le64 dma_base; +__le32 page_count; +u8 page_size_log2; +u8 rsvd[3 ]; +}; + +/** +* struct pds_lm_dirty_status_cmd - DIRTY_STATUS command +* @opcode: Opcode PDS_LM_CMD_DIRTY_STATUS +* @rsvd: Word boundary padding +* @vf_id: VF id +* @max_regions: Capacity of the region info buffer +* @rsvd2: Word boundary padding +* @regions_dma: DMA address of the region info buffer +* +* The minimum of max_regions (from the command) and num_regions (from the
+ * completion) of structpds_lm_dirty_region_infowill be written to
+ *regions_dma.
+ *
+ * Themax_regionsmay be zero, in which caseregions_dmais ignored. In that
+ * case, the completion will only report the maximum number of regions
+ * supported by the device, and the number of regions currently enabled.
+ */include/linux/pds/pds_adminq.h +24
-0+struct pds_lm_dirty_status_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 max_regions; +u8 rsvd2[3 ]; +__le64 regions_dma; +} __packed; + +/** +* enum pds_lm_dirty_bmp_type - Type of dirty page bitmap +* @PDS_LM_DIRTY_BMP_TYPE_NONE: No bitmap / disabled +* @PDS_LM_DIRTY_BMP_TYPE_SEQ_ACK: Seq/Ack bitmap representation +*/ +enum pds_lm_dirty_bmp_type { +PDS_LM_DIRTY_BMP_TYPE_NONE =0 , +PDS_LM_DIRTY_BMP_TYPE_SEQ_ACK =1 , +}; + +/** +* struct pds_lm_dirty_status_comp - STATUS command completion +* @status: Status of the command (enum pds_core_status_code) +* @rsvd: Word boundary padding +* @comp_index: Index in the desc ring for which this is the completion
+ * @max_regions: Maximum number of regions supported by the
device
+ * @num_regions: Number of regions currently enabled
+ * @bmp_type: Type of dirty bitmap representation+ * @rsvd2: Word boundary padding + * @bmp_type_mask: Mask of supported bitmap types, bit index per type + * @rsvd3: Word boundary padding + * @color: Color bit+ *
+ * This completion descriptor is used for STATUS, ENABLE, and DISABLE.
+ */include/linux/pds/pds_adminq.h +24
-0+struct pds_lm_dirty_status_comp { +u8 status; +u8 rsvd; +__le16 comp_index; +u8 max_regions; +u8 num_regions; +u8 bmp_type; +u8 rsvd2; +__le32 bmp_type_mask; +u8 rsvd3[3 ]; +u8 color; +}; + +/** +* struct pds_lm_dirty_enable_cmd - DIRTY_ENABLE command +* @opcode: Opcode PDS_LM_CMD_DIRTY_ENABLE +* @rsvd: Word boundary padding +* @vf_id: VF id +* @bmp_type: Type of dirty bitmap representation +* @num_regions: Number of entries in the region info buffer +* @rsvd2: Word boundary padding +* @regions_dma: DMA address of the region info buffer +* +* The num_regions must be nonzero, and less than or equal to the maximum
+ * number of regions supported by the device.
+ *
+ * The memory regions should not overlap.
+ *
+ * The information should be initialized by the driver. The device may modify
+ * the information on successful completion, such as by size-aligning the
+ * number of pages in a region.
+ *
+ * The modified number of pages will be greater than or equal to the page count
+ * given in the enable command, and at least as coarsly aligned as the given
+ * value. For example, the count might be aligned to a multiple of 64, but
+ * if the value is already a multiple of 128 or higher, it will not change.
+ * If the driver requires its own minimum alignment of the number of pages, the
+ * driver should account for that already in the region info of this command.
+ *
+ * This command uses structpds_lm_dirty_status_compfor its completion.
+ */include/linux/pds/pds_adminq.h +21
-0+struct pds_lm_dirty_enable_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +u8 bmp_type; +u8 num_regions; +u8 rsvd2[2 ]; +__le64 regions_dma; +} __packed; + +/** +* struct pds_lm_dirty_disable_cmd - DIRTY_DISABLE command +* @opcode: Opcode PDS_LM_CMD_DIRTY_DISABLE +* @rsvd: Word boundary padding +* @vf_id: VF id +* +* Dirty page tracking will be disabled. This may be called in any state, as +* long as dirty page tracking is supported by the device, to ensure that dirty +* page tracking is disabled. +* +* This command uses struct pds_lm_dirty_status_comp for its completion. On
+ * success,num_regionswill be zero.
+ */include/linux/pds/pds_adminq.h +8
-0+struct pds_lm_dirty_disable_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +}; + +/** +* struct pds_lm_dirty_seq_ack_cmd - DIRTY_READ_SEQ or _WRITE_ACK command
+ * @opcode: Opcode PDS_LM_CMD_DIRTY_[READ_SEQ|WRITE_ACK] + * @rsvd: Word boundary padding + * @vf_id: VF id+ * @off_bytes: Byte offset in the bitmap
+ * @len_bytes: Number of bytes to transfer
+ * @num_sge: Number of DMA scatter gather elements
+ * @rsvd2: Word boundary padding
+ * @sgl_addr: DMA address of scatter gather list
+ *
+ * Read bytes from the SEQ bitmap, or write bytes into the ACK bitmap.
+ *
+ * This command treats the entire bitmap as a byte buffer. It does not
+ * distinguish between guest memory regions. The driver should refer to the
+ * number of pages in each region, according toPDS_LM_CMD_DIRTY_STATUS, to
+ * determine the region boundaries in the bitmap. Each region will be
+ * represented by exactly the number of bits as the page count for that region,
+ * immediately following the last bit of the previous region.
+ */include/linux/pds/pds_adminq.h +16
-1+struct pds_lm_dirty_seq_ack_cmd { +u8 opcode; +u8 rsvd; +__le16 vf_id; +__le32 off_bytes; +__le32 len_bytes; +__le16 num_sge; +u8 rsvd2[2 ]; +__le64 sgl_addr; +} __packed; +/** * struct pds_lm_host_vf_status_cmd - HOST_VF_STATUS command * @opcode: Opcode PDS_LM_CMD_HOST_VF_STATUS @@ -1039,6 +1212,10 @@ union pds_core_adminq_cmd {struct pds_lm_save_cmd lm_save;struct pds_lm_restore_cmd lm_restore;struct pds_lm_host_vf_status_cmd lm_host_vf_status; +struct pds_lm_dirty_status_cmd lm_dirty_status; +struct pds_lm_dirty_enable_cmd lm_dirty_enable; +struct pds_lm_dirty_disable_cmd lm_dirty_disable; +struct pds_lm_dirty_seq_ack_cmd lm_dirty_seq_ack; };union pds_core_adminq_comp {@@ -1065,6 +1242,7 @@ union pds_core_adminq_comp {struct pds_vdpa_vq_reset_comp vdpa_vq_reset;struct pds_lm_state_size_comp lm_state_size; +struct pds_lm_dirty_status_comp lm_dirty_status; };#ifndef __CHECKER__ --2.17.1
Brett Creeley
maintainerPATCH v13 6/7 vfio/pds: Add support for firmware recovery
In reply to Brett CreeleyIt's possible that the device firmware crashes and is able to recover due to some configuration and/or other issue. If a live migration is in progress while the firmware crashes, the live migration will fail. However, the VF PCI device should still be functional post crash recovery and subsequent migrations should go through as expected.
When the pds_core device notices that firmware crashes it sends an event to all its client drivers. When the pds_vfio driver receives this event while migration is in progress it will request a deferred reset on the next migration state transition. This state transition will report failure as well as any subsequent state transition requests from the VMM/VFIO. Based on uapi/vfio.h the only way out of VFIO_DEVICE_STATE_ERROR is by issuing VFIO_DEVICE_RESET. Once this reset is done, the migration state will be reset to VFIO_DEVICE_STATE_RUNNING and migration can be performed.
If the event is received while no migration is in progress (i.e. the VM is in normal operating mode), then no actions are taken and the migration state remains VFIO_DEVICE_STATE_RUNNING.
--- drivers/vfio/pci/pds/pci_drv.c | 114 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/pds/vfio_dev.c | 17 ++++- drivers/vfio/pci/pds/vfio_dev.h | 2 + 3 files changed, 131 insertions(+), 2 deletions(-)
drivers/vfio/pci/pds/pci_drv.c +114 -0
@@ -19,6 +19,113 @@#define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO Device Driver"#define PCI_VENDOR_ID_PENSANDO 0x1dd8 +static void pds_vfio_recovery (struct pds_vfio_pci_device *pds_vfio) +{ +bool deferred_reset_needed = false; + +/* +* Documentation states that the kernel migration driver must not +* generate asynchronous device state transitions outside of +* manipulation by the user or the VFIO_DEVICE_RESET ioctl. +* +* Since recovery is an asynchronous event received from the device, +* initiate a deferred reset. Issue a deferred reset in the following +* situations: +* 1. Migration is in progress, which will cause the next step of +* the migration to fail. +* 2. If the device is in a state that will be set to +* VFIO_DEVICE_STATE_RUNNING on the next action (i.e. VM is +* shutdown and device is in VFIO_DEVICE_STATE_STOP). +*/ +mutex_lock (&pds_vfio->state_mutex); +if ((pds_vfio->state !=VFIO_DEVICE_STATE_RUNNING && + pds_vfio->state !=VFIO_DEVICE_STATE_ERROR ) || + (pds_vfio->state ==VFIO_DEVICE_STATE_RUNNING && +pds_vfio_dirty_is_enabled (pds_vfio))) + deferred_reset_needed = true; +mutex_unlock (&pds_vfio->state_mutex); + +/* +* On the next user initiated state transition, the device will +* transition to the VFIO_DEVICE_STATE_ERROR. At this point it's the user's +* responsibility to reset the device. +* +* If a VFIO_DEVICE_RESET is requested post recovery and before the next +* state transition, then the deferred reset state will be set to +* VFIO_DEVICE_STATE_RUNNING. +*/ +if (deferred_reset_needed) { +spin_lock (&pds_vfio->reset_lock); + pds_vfio->deferred_reset = true; + pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_ERROR ; +spin_unlock (&pds_vfio->reset_lock); + } +} + +static int pds_vfio_pci_notify_handler (struct notifier_block *nb, +unsigned long ecode,void *data) +{ +struct pds_vfio_pci_device *pds_vfio = + container_of(nb, struct pds_vfio_pci_device, nb); +struct device *dev =pds_vfio_to_dev (pds_vfio); +union pds_core_notifyq_comp *event = data; + +dev_dbg (dev,"%s: event code %lu\n" , __func__, ecode); + +/* +* We don't need to do anything for RESET state==0 as there is no notify +* or feedback mechanism available, and it is possible that we won't +* even see a state==0 event since the pds_core recovery is pending. +* +* Any requests from VFIO while state==0 will fail, which will return +* error and may cause migration to fail. +*/ +if (ecode ==PDS_EVENT_RESET ) { +dev_info (dev,"%s: PDS_EVENT_RESET event received, state==%d\n" , + __func__, event->reset.state); +/* +* pds_core device finished recovery and sent us the +* notification (state == 1) to allow us to recover +*/ +if (event->reset.state ==1 ) +pds_vfio_recovery (pds_vfio); + } + +return 0 ; +} + +static int +pds_vfio_pci_register_event_handler (struct pds_vfio_pci_device *pds_vfio) +{ +struct device *dev =pds_vfio_to_dev (pds_vfio); +struct notifier_block *nb = &pds_vfio->nb; +int err; + +if (!nb->notifier_call) { + nb->notifier_call = pds_vfio_pci_notify_handler; + err =pdsc_register_notify (nb); +if (err) { + nb->notifier_call =NULL ; +dev_err (dev, +"failed to register pds event handler: %pe\n" , +ERR_PTR (err)); +return -EINVAL ; + } +dev_dbg (dev,"pds event handler registered\n" ); + } + +return 0 ; +} + +static void +pds_vfio_pci_unregister_event_handler (struct pds_vfio_pci_device *pds_vfio) +{ +if (pds_vfio->nb.notifier_call) { +pdsc_unregister_notify (&pds_vfio->nb); + pds_vfio->nb.notifier_call =NULL ; + } +} +static int pds_vfio_pci_probe (struct pci_dev *pdev,const struct pci_device_id *id) {
@@ -48,8 +155,14 @@ static int pds_vfio_pci_probe(struct pci_dev *pdev, goto out_unregister_coredev; } + err =pds_vfio_pci_register_event_handler (pds_vfio); +if (err) + goto out_unregister_client; +return 0 ; +out_unregister_client: +pds_vfio_unregister_client_cmd (pds_vfio); out_unregister_coredev:vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev); out_put_vdev:
@@ -61,6 +174,7 @@ static void pds_vfio_pci_remove(struct pci_dev *pdev) {struct pds_vfio_pci_device *pds_vfio =pds_vfio_pci_drvdata (pdev); +pds_vfio_pci_unregister_event_handler (pds_vfio);pds_vfio_unregister_client_cmd (pds_vfio);vfio_pci_core_unregister_device (&pds_vfio->vfio_coredev);vfio_put_device (&pds_vfio->vfio_coredev.vdev);
drivers/vfio/pci/pds/vfio_dev.c +15 -2
@@ -33,11 +33,12 @@ void pds_vfio_state_mutex_unlock(struct pds_vfio_pci_device *pds_vfio)if (pds_vfio->deferred_reset) { pds_vfio->deferred_reset = false;if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) { - pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ;pds_vfio_put_restore_file (pds_vfio);pds_vfio_put_save_file (pds_vfio);pds_vfio_dirty_disable (pds_vfio, false); } + pds_vfio->state = pds_vfio->deferred_reset_state; + pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;spin_unlock (&pds_vfio->reset_lock); goto again; }
@@ -49,6 +50,7 @@ void pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio) {spin_lock (&pds_vfio->reset_lock); pds_vfio->deferred_reset = true; + pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;if (!mutex_trylock (&pds_vfio->state_mutex)) {spin_unlock (&pds_vfio->reset_lock);return ;
@@ -67,7 +69,14 @@ pds_vfio_set_device_state(struct vfio_device *vdev,struct file *res =NULL ;mutex_lock (&pds_vfio->state_mutex); -while (new_state != pds_vfio->state) { +/* +* only way to transition out of VFIO_DEVICE_STATE_ERROR is via +* VFIO_DEVICE_RESET, so prevent the state machine from running since +* vfio_mig_get_next_state() will throw a WARN_ON() when transitioning +* from VFIO_DEVICE_STATE_ERROR to any other state +*/ +while (pds_vfio->state !=VFIO_DEVICE_STATE_ERROR && + new_state != pds_vfio->state) {enum vfio_device_mig_state next_state; int err = vfio_mig_get_next_state(vdev, pds_vfio->state,
@@ -89,6 +98,9 @@ pds_vfio_set_device_state(struct vfio_device *vdev, } }pds_vfio_state_mutex_unlock (pds_vfio); +/* still waiting on a deferred_reset */ +if (pds_vfio->state ==VFIO_DEVICE_STATE_ERROR ) + res =ERR_PTR (-EIO );return res; }
@@ -169,6 +181,7 @@ static int pds_vfio_open_device(struct vfio_device *vdev)mutex_init (&pds_vfio->state_mutex); pds_vfio->state =VFIO_DEVICE_STATE_RUNNING ; + pds_vfio->deferred_reset_state =VFIO_DEVICE_STATE_RUNNING ;vfio_pci_core_finish_enable (&pds_vfio->vfio_coredev);
drivers/vfio/pci/pds/vfio_dev.h +2 -0
@@ -23,6 +23,8 @@ struct pds_vfio_pci_device {enum vfio_device_mig_state state;spinlock_t reset_lock;/* protect reset_done flow */ u8 deferred_reset; +enum vfio_device_mig_state deferred_reset_state; +struct notifier_block nb;int vf_id;u16 client_id;
Signature · 1 line
2.17.1
Jason Gunthorpe
reviewerRe: [PATCH v13 vfio 6/7] vfio/pds: Add support for firmware recovery
Quoted 7 lines · Brett Creeley
It's possible that the device firmware crashes and is able to recover due to some configuration and/or other issue. If a live migration is in progress while the firmware crashes, the live migration will fail. However, the VF PCI device should still be functional post crash recovery and subsequent migrations should go through as expected.
Quoted 9 lines
When the
pds_coredevice notices that firmware crashes it sends an event to all its client drivers. When thepds_vfiodriver receives this event while migration is in progress it will request a deferred reset on the next migration state transition. This state transition will report failure as well as any subsequent state transition requests from the VMM/VFIO. Based onuapi/vfio.hthe only way out ofVFIO_DEVICE_STATE_ERRORis by issuingVFIO_DEVICE_RESET. Once this reset is done, the migration state will be reset toVFIO_DEVICE_STATE_RUNNINGand migration can be performed.
Have you actually tested this? Does the qemu side respond properly if this happens during a migration?
Jason
Re: [PATCH v13 vfio 6/7] vfio/pds: Add support for firmware recovery
Quoted 21 lines · Jason Gunthorpe
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 7 lines · Brett Creeley
It's possible that the device firmware crashes and is able to recover due to some configuration and/or other issue. If a live migration is in progress while the firmware crashes, the live migration will fail. However, the VF PCI device should still be functional post crash recovery and subsequent migrations should go through as expected.
Quoted 9 lines
When the
pds_coredevice notices that firmware crashes it sends an event to all its client drivers. When thepds_vfiodriver receives this event while migration is in progress it will request a deferred reset on the next migration state transition. This state transition will report failure as well as any subsequent state transition requests from the VMM/VFIO. Based onuapi/vfio.hthe only way out ofVFIO_DEVICE_STATE_ERRORis by issuingVFIO_DEVICE_RESET. Once this reset is done, the migration state will be reset toVFIO_DEVICE_STATE_RUNNINGand migration can be performed.
Quoted 4 lines
Have you actually tested this? Does the qemu side respond properly if this happens during a migration?
Jason
Yes, this has actually been tested. It's not necessary clean as far as the log messages go because the driver may still be getting requests (i.e. dirty log requests), but the noise should be okay because this is a very rare event.
QEMU does respond properly and in the manner I mentioned above.
Thanks,
Brett
Jason Gunthorpe
reviewerRe: [PATCH v13 vfio 6/7] vfio/pds: Add support for firmware recovery
Quoted 27 lines · Brett Creeley
Quoted 24 lines · Jason Gunthorpe
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 7 lines · Brett Creeley
It's possible that the device firmware crashes and is able to recover due to some configuration and/or other issue. If a live migration is in progress while the firmware crashes, the live migration will fail. However, the VF PCI device should still be functional post crash recovery and subsequent migrations should go through as expected.
Quoted 9 lines
When the
pds_coredevice notices that firmware crashes it sends an event to all its client drivers. When thepds_vfiodriver receives this event while migration is in progress it will request a deferred reset on the next migration state transition. This state transition will report failure as well as any subsequent state transition requests from the VMM/VFIO. Based onuapi/vfio.hthe only way out ofVFIO_DEVICE_STATE_ERRORis by issuingVFIO_DEVICE_RESET. Once this reset is done, the migration state will be reset toVFIO_DEVICE_STATE_RUNNINGand migration can be performed.Have you actually tested this? Does the qemu side respond properly if this happens during a migration?
Quoted 8 lines
Quoted 1 line
Jason
Yes, this has actually been tested. It's not necessary clean as far as the log messages go because the driver may still be getting requests (i.e. dirty log requests), but the noise should be okay because this is a very rare event.
QEMU does respond properly and in the manner I mentioned above.
But what actually happens?
QEMU aborts the migration and FLRs the device and then the VM has a totally trashed PCI function?
Can the VM recover from this?
Jason
Re: [PATCH v13 vfio 6/7] vfio/pds: Add support for firmware recovery
Quoted 40 lines · Jason Gunthorpe
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 27 lines · Brett Creeley
Quoted 24 lines · Jason Gunthorpe
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 7 lines · Brett Creeley
It's possible that the device firmware crashes and is able to recover due to some configuration and/or other issue. If a live migration is in progress while the firmware crashes, the live migration will fail. However, the VF PCI device should still be functional post crash recovery and subsequent migrations should go through as expected.
Quoted 9 lines
When the
pds_coredevice notices that firmware crashes it sends an event to all its client drivers. When thepds_vfiodriver receives this event while migration is in progress it will request a deferred reset on the next migration state transition. This state transition will report failure as well as any subsequent state transition requests from the VMM/VFIO. Based onuapi/vfio.hthe only way out ofVFIO_DEVICE_STATE_ERRORis by issuingVFIO_DEVICE_RESET. Once this reset is done, the migration state will be reset toVFIO_DEVICE_STATE_RUNNINGand migration can be performed.Have you actually tested this? Does the qemu side respond properly if this happens during a migration?
Quoted 8 lines
Quoted 1 line
Jason
Yes, this has actually been tested. It's not necessary clean as far as the log messages go because the driver may still be getting requests (i.e. dirty log requests), but the noise should be okay because this is a very rare event.
QEMU does respond properly and in the manner I mentioned above.
Quoted 8 lines
But what actually happens?
QEMU aborts the migration and FLRs the device and then the VM has a totally trashed PCI function?
Can the VM recover from this?
Jason
As it mentions above, the VM and PCI function do recover from this and the subsequent migration works as expected.
Thanks,
Brett
Tian, Kevin
reviewerRE: [PATCH v13 vfio 6/7] vfio/pds: Add support for firmware recovery
Quoted 52 lines
From: Brett Creeley <[email protected]>
Sent: Saturday, August 5, 2023 2:51 AMQuoted 1 line · Jason Gunthorpe
Caution: This message originated from an External Source. Use proper
caution when opening attachments, clicking links, or responding.
Quoted 7 lines · Brett Creeley
Quoted 4 lines · Jason Gunthorpe
Quoted 1 line
Caution: This message originated from an External Source. Use proper
caution when opening attachments, clicking links, or responding.
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines · Brett Creeley
Quoted 7 lines
It's possible that the device firmware crashes and is able to recover due to some configuration and/or other issue. If a live migration is in progress while the firmware crashes, the live migration will fail. However, the VF PCI device should still be functional post crash recovery and subsequent migrations should go through as expected.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
When the
pds_coredevice notices that firmware crashes it sends an event to all its client drivers. When thepds_vfiodriver receives this event while migration is in progress it will request a deferred reset on the next migration state transition. This state transition will report failure as well as any subsequent state transition requests from the VMM/VFIO. Based onuapi/vfio.hthe only way out ofVFIO_DEVICE_STATE_ERRORis by issuingVFIO_DEVICE_RESET. Oncethis
Quoted 20 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
reset is done, the migration state will be reset to
VFIO_DEVICE_STATE_RUNNINGand migration can be performed.Have you actually tested this? Does the qemu side respond properly if this happens during a migration?
Quoted 8 lines
Quoted 1 line
Jason
Yes, this has actually been tested. It's not necessary clean as far as the log messages go because the driver may still be getting requests (i.e. dirty log requests), but the noise should be okay because this is a very rare event.
QEMU does respond properly and in the manner I mentioned above.
But what actually happens?
QEMU aborts the migration and FLRs the device and then the VM has a totally trashed PCI function?
Quoted 7 lines
Quoted 3 lines
Can the VM recover from this?
Jason
As it mentions above, the VM and PCI function do recover from this and the subsequent migration works as expected.
If reset is requested by the host how is the VM notified to handle this undesired situation? Would it lead to observable application failures inside the guest after the recovery?
Brett Creeley
maintainerPATCH v13 7/7 vfio/pds: Add Kconfig and documentation
In reply to Brett CreeleyAdd Kconfig entries and pds-vfio-pci.rst. Also, add an entry in the MAINTAINERS file for this new driver.
It's not clear where documentation for vendor specific VFIO drivers should live, so just re-use the current amd ethernet location.
--- .../ethernet/amd/pds_vfio_pci.rst | 79 +++++++++++++++++++ .../device_drivers/ethernet/index.rst | 1 + MAINTAINERS | 7 ++ drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/pds/Kconfig | 19 +++++ 5 files changed, 108 insertions(+) create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vfio_pci.rst create mode 100644 drivers/vfio/pci/pds/Kconfig
Documentation/networking/device_drivers/ethernet/amd/pds_vfio_pci.rst new file +79 -0
@@ -0,0 +1,79 @@ +.. SPDX-License-Identifier: GPL-2.0+ +.. note: can be edited and viewed with /usr/bin/formiko-vim + +========================================================== +PCI VFIO driver for the AMD/Pensando(R) DSC adapter family +========================================================== + +AMD/Pensando Linux VFIO PCI Device Driver +Copyright(c) 2023 Advanced Micro Devices, Inc. + +Overview +======== + +The ``pds-vfio-pci`` module is a PCI driver that supports Live Migration +capable Virtual Function (VF) devices in the DSC hardware. + +Using the device +================ + +The pds-vfio-pci device is enabled via multiple configuration steps and +depends on the ``pds_core`` driver to create and enable SR-IOV Virtual +Function devices. + +Shown below are the steps to bind the driver to a VF and also to the +associated auxiliary device created by the ``pds_core`` driver. This +example assumes the pds_core and pds-vfio-pci modules are already +loaded. + +.. code-block:: bash + :name: example-setup-script + + #!/bin/bash + + PF_BUS="0000:60" + PF_BDF="0000:60:00.0" + VF_BDF="0000:60:00.1" + + # Prevent non-vfio VF driver from probing the VF device + echo 0 > /sys/class/pci_bus/$PF_BUS/device/$PF_BDF/sriov_drivers_autoprobe + + # Create single VF for Live Migration via pds_core + echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs + + # Allow the VF to be bound to the pds-vfio-pci driver + echo "pds-vfio-pci" > /sys/class/pci_bus/$PF_BUS/device/$VF_BDF/driver_override + + # Bind the VF to the pds-vfio-pci driver + echo "$VF_BDF" > /sys/bus/pci/drivers/pds-vfio-pci/bind + +After performing the steps above, a file in /dev/vfio/<iommu_group> +should have been created. + + +Enabling the driver +=================== + +The driver is enabled via the standard kernel configuration system, +using the make command:: + + make oldconfig/menuconfig/etc. + +The driver is located in the menu structure at: + + -> Device Drivers + -> VFIO Non-Privileged userspace driver framework + -> VFIO support for PDS PCI devices + +Support +======= + +For general Linux networking support, please use the netdev mailing +list, which is monitored by Pensando personnel:: + + [email protected] + +For more specific support needs, please use the Pensando driver support +email:: + + [email protected]
Documentation/networking/device_drivers/ethernet/index.rst +1 -0
@@ -16,6 +16,7 @@ Contents: altera/altera_tse amd/pds_core amd/pds_vdpa + amd/pds_vfio_pci aquantia/atlantic chelsio/cxgb cirrus/cs89x0
MAINTAINERS +7 -0
@@ -22436,6 +22436,13 @@ S: Maintained P: Documentation/driver-api/vfio-pci-device-specific-driver-acceptance.rst F: drivers/vfio/pci/*/ +VFIO PDS PCI DRIVER +M: Brett Creeley <[email protected]> +L: [email protected] +S: Maintained +F: Documentation/networking/device_drivers/ethernet/amd/pds_vfio_pci.rst +F: drivers/vfio/pci/pds/ + VFIO PLATFORM DRIVER M: Eric Auger <[email protected]> L: [email protected]
drivers/vfio/pci/Kconfig +2 -0
@@ -63,4 +63,6 @@ source "drivers/vfio/pci/mlx5/Kconfig" source "drivers/vfio/pci/hisilicon/Kconfig" +source "drivers/vfio/pci/pds/Kconfig" + endmenu
drivers/vfio/pci/pds/Kconfig new file +19 -0
@@ -0,0 +1,19 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2023 Advanced Micro Devices, Inc. + +config PDS_VFIO_PCI + tristate "VFIO support for PDS PCI devices" + depends on PDS_CORE + select VFIO_PCI_CORE + help + This provides generic PCI support for PDS devices using the VFIO + framework. + + More specific information on this driver can be + found in + <file:Documentation/networking/device_drivers/ethernet/amd/pds_vfio_pci.rst>. + + To compile this driver as a module, choose M here. The module + will be called pds-vfio-pci. + + If you don't know what to do here, say N.
Signature · 1 line
2.17.1
Jason Gunthorpe
reviewerQuoted 6 lines · Brett Creeley
Note: This series is based on the latest linux-next tree. I did not base it on the Alex Williamson's vfio/next because it has not yet pulled in the latest changes which include the
pds_vdpadriver. Thepds_vdpadriver has conflicts with the pds-vfio-pci driver that needed to be resolved, which is why this series is based on the latest linux-next tree.
This is not the right way to handle this, Alex cannot apply a series against linux-next.
If you can't make a shared branch and the conflicts are too significant to forward to Linus then you have to wait for the next cycle.
Jason
Quoted 9 lines · On Wed, 26 Jul 2023 10:35:06 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 6 lines · Brett Creeley
Note: This series is based on the latest linux-next tree. I did not base it on the Alex Williamson's vfio/next because it has not yet pulled in the latest changes which include the
pds_vdpadriver. Thepds_vdpadriver has conflicts with the pds-vfio-pci driver that needed to be resolved, which is why this series is based on the latest linux-next tree.
Quoted 6 lines
This is not the right way to handle this, Alex cannot apply a series against linux-next.
If you can't make a shared branch and the conflicts are too significant to forward to Linus then you have to wait for the next cycle.
Brett, can you elaborate on what's missing from my next branch vs linux-next?
AFAICT the pds_vdpa driver went into mainline via a8d70602b186 ("Merge tag 'for_linus' ofgit://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost") during the v6.5 merge window and I'm not spotting anything in linux-next obviously relevant to pds-vfio-pci since then.
There's a debugfs fix on the list, but that's sufficiently trivial to fixup on merge if necessary. This series also applies cleanly vs my current next branch. Was the issue simply that I hadn't updated my next branch (done yesterday) since the v6.5 merge window? You can always send patches vs mainline. Thanks,
Alex
Quoted 31 lines · Alex Williamson
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 9 lines · On Wed, 26 Jul 2023 10:35:06 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 6 lines · Brett Creeley
Note: This series is based on the latest linux-next tree. I did not base it on the Alex Williamson's vfio/next because it has not yet pulled in the latest changes which include the
pds_vdpadriver. Thepds_vdpadriver has conflicts with the pds-vfio-pci driver that needed to be resolved, which is why this series is based on the latest linux-next tree.Quoted 6 lines
This is not the right way to handle this, Alex cannot apply a series against linux-next.
If you can't make a shared branch and the conflicts are too significant to forward to Linus then you have to wait for the next cycle.
Brett, can you elaborate on what's missing from my next branch vs linux-next?
AFAICT the
pds_vdpadriver went into mainline viaa8d70602b186("Merge tag 'for_linus' ofgit://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost") during the v6.5 merge window and I'm not spotting anything in linux-next obviously relevant to pds-vfio-pci since then.
Quoted 5 lines
There's a debugfs fix on the list, but that's sufficiently trivial to fixup on merge if necessary. This series also applies cleanly vs my current next branch. Was the issue simply that I hadn't updated my next branch (done yesterday) since the v6.5 merge window? You can always send patches vs mainline. Thanks,
Yeah, this was exactly it. Your vfio/next branch didn't have the pds_vdpa series in it yet, which also included some changes to the header files used by the pds-vfio-pci series, which is where the conflicts are.
Should I rebase my series on your vfio/next branch and resend?
Thanks,
Brett
Quoted 3 lines
Alex
Quoted 38 lines · On Wed, 26 Jul 2023 12:05:13 -0700 Brett Creeley <[email protected]> wrote:
Quoted 31 lines · Alex Williamson
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Quoted 9 lines · On Wed, 26 Jul 2023 10:35:06 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 6 lines · Brett Creeley
Note: This series is based on the latest linux-next tree. I did not base it on the Alex Williamson's vfio/next because it has not yet pulled in the latest changes which include the
pds_vdpadriver. Thepds_vdpadriver has conflicts with the pds-vfio-pci driver that needed to be resolved, which is why this series is based on the latest linux-next tree.Quoted 6 lines
This is not the right way to handle this, Alex cannot apply a series against linux-next.
If you can't make a shared branch and the conflicts are too significant to forward to Linus then you have to wait for the next cycle.
Brett, can you elaborate on what's missing from my next branch vs linux-next?
AFAICT the
pds_vdpadriver went into mainline viaa8d70602b186("Merge tag 'for_linus' ofgit://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost") during the v6.5 merge window and I'm not spotting anything in linux-next obviously relevant to pds-vfio-pci since then.Quoted 5 lines
There's a debugfs fix on the list, but that's sufficiently trivial to fixup on merge if necessary. This series also applies cleanly vs my current next branch. Was the issue simply that I hadn't updated my next branch (done yesterday) since the v6.5 merge window? You can always send patches vs mainline. Thanks,
Quoted 4 lines
Yeah, this was exactly it. Your vfio/next branch didn't have the
pds_vdpaseries in it yet, which also included some changes to the header files used by the pds-vfio-pci series, which is where the conflicts are.
Ok, so let's put this back on the table as a candidate for v6.6.
Quoted 1 line
Should I rebase my series on your vfio/next branch and resend?
It doesn't seem necessary, I think rebasing my next branch to v6.5-rc3 made it effectively equivalent to linux-next for the purposes of this driver. It applies cleanly, so I think we can continue review from this. Thanks,
Alex
Quoted 53 lines · Brett Creeley
This is a patchset for a new vendor specific VFIO driver (pds-vfio-pci) for use with the AMD/Pensando Distributed Services Card (DSC). This driver makes use of the
pds_coredriver.This driver will use the
pds_coredevice's adminq as the VFIO control path to the DSC. In order to make adminq calls, the VFIO instance makes use of functions exported by thepds_coredriver.In order to receive events from
pds_core, the pds-vfio-pci driver registers to a private notifier. This is needed for various events that come from the device.An ASCII diagram of a VFIO instance looks something like this and can be used with the VFIO subsystem to provide the VF device VFIO and live migration support.
.------. .-----------------------. | QEMU |--| VM .-------------. | '......' | | Eth VF | | | | .-------------. | | | | SR-IOV VF | | | | '-------------' | | '------------||---------' .--------------. || |/dev/<vfio_fd>| || '--------------' || Host Userspace | || =================================================== || Host Kernel | || .--------. || |vfio-pci| || '--------' || .------------------. || || | | exported API |<----+ || || | '--------------| | || || | | .--------------. || | pds_core |--->| pds-vfio-pci | || '------------------' | '--------------' || || | || || 09:00.0 notifier 09:00.1 || == PCI ===============================================||===== || || || .----------. .----------. || ,-----| PF |----------| VF |-------------------, | '----------' |'----------' VF | | DSC | data/control | | | path | -----------------------------------------------------------The pds-vfio-pci driver is targeted to reside in
drivers/vfio/pci/pds. It makes use of and introduces new files in the commoninclude/linux/pdsinclude directory.
Quoted 6 lines
Note: This series is based on the latest linux-next tree. I did not base it on the Alex Williamson's vfio/next because it has not yet pulled in the latest changes which include the
pds_vdpadriver. Thepds_vdpadriver has conflicts with the pds-vfio-pci driver that needed to be resolved, which is why this series is based on the latest linux-next tree.
For series,