PCI: Add vf reset notification for pf
PATCH 1/1 PCI: Add vf reset notification for pf
When a vf has been reset, the pf wants to get notification to remove the vf out of schedule.
Solution:
Add the callback function in pci_driver sriov_vf_reset_notification. When vf reset happens, then call this callback function.
--- drivers/pci/pci.c | 8 ++++++++ include/linux/pci.h | 1 + 2 files changed, 9 insertions(+)
drivers/pci/pci.c +8 -0
@@ -4780,6 +4780,14 @@ EXPORT_SYMBOL_GPL(pcie_flr);*/ int pcie_reset_flr (struct pci_dev *dev,bool probe) { +struct pci_dev *pf_dev; + +if (dev->is_virtfn) { + pf_dev = dev->physfn; +if (pf_dev->driver->sriov_vf_reset_notification) + pf_dev->driver->sriov_vf_reset_notification (pf_dev, dev); + } +if (dev->dev_flags &PCI_DEV_FLAGS_NO_FLR_RESET )return -ENOTTY ;
include/linux/pci.h +1 -0
@@ -926,6 +926,7 @@ struct pci_driver {int (*sriov_configure)(struct pci_dev *dev,int num_vfs);/* On PF */ int (*sriov_set_msix_vec_count)(struct pci_dev *vf,int msix_vec_count);/* On PF */ u32 (*sriov_get_vf_total_msix)(struct pci_dev *pf); +void (*sriov_vf_reset_notification)(struct pci_dev *pf,struct pci_dev *vf);const struct pci_error_handlers *err_handler;const struct attribute_group **groups;const struct attribute_group **dev_groups;
Signature · 1 line
2.36.1
include/linux/pci.h struct pci_driver { · line 929
Quoted 5 lines · Emily Deng
include/linux/pci.h +1
-0@@ -926,6 +926,7 @@ struct pci_driver {int (*sriov_configure)(struct pci_dev *dev,int num_vfs);/* On PF */ int (*sriov_set_msix_vec_count)(struct pci_dev *vf,int msix_vec_count);/* On PF */ u32 (*sriov_get_vf_total_msix)(struct pci_dev *pf); +void (*sriov_vf_reset_notification)(struct pci_dev *pf,struct pci_dev *vf);
You've created a new callback, but there is no user. Could you resubmit this with an in-kernel use case?
Quoted 2 lines · Emily Deng
When a vf has been reset, the pf wants to get notification to remove the vf out of schedule.
It is very questionable if this is right thing to do. The idea of SR-IOV is that VFs represent a physical device and they should be treated separately from the PF.
In addition to that Keith said, this patch needs better justification.
Thanks
Quoted 46 lines
Solution:
Add the callback function inpci_driversriov_vf_reset_notification. When vf reset happens, then call this callback function.Signed-off-by Emily Deng <[email protected]>--- drivers/pci/pci.c | 8 ++++++++ include/linux/pci.h | 1 + 2 files changed, 9 insertions(+)drivers/pci/pci.c +8
-0@@ -4780,6 +4780,14 @@ EXPORT_SYMBOL_GPL(pcie_flr);*/ int pcie_reset_flr (struct pci_dev *dev,bool probe) { +struct pci_dev *pf_dev; + +if (dev->is_virtfn) { + pf_dev = dev->physfn; +if (pf_dev->driver->sriov_vf_reset_notification) + pf_dev->driver->sriov_vf_reset_notification (pf_dev, dev); + } +if (dev->dev_flags &PCI_DEV_FLAGS_NO_FLR_RESET )return -ENOTTY ;include/linux/pci.h +1
-0@@ -926,6 +926,7 @@ struct pci_driver {int (*sriov_configure)(struct pci_dev *dev,int num_vfs);/* On PF */ int (*sriov_set_msix_vec_count)(struct pci_dev *vf,int msix_vec_count);/* On PF */ u32 (*sriov_get_vf_total_msix)(struct pci_dev *pf); +void (*sriov_vf_reset_notification)(struct pci_dev *pf,struct pci_dev *vf);const struct pci_error_handlers *err_handler;const struct attribute_group **groups;const struct attribute_group **dev_groups;Signature · 1 line
2.36.1
[AMD Official Use Only - General]
The use case is the vfio-pci driver, which is used to support a user mode PF driver. Will also sent out the vfio-pci driver patch. And add more comments in the patch. Thanks.
Emily Deng
Best Wishes
Quoted 69 lines
-----Original Message-----
From: Leon Romanovsky <[email protected]>
Sent: Sunday, February 4, 2024 7:21 PM
To: Deng, Emily <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]
Subject: Re: [PATCH] PCI: Add vf reset notification for pfQuoted 2 lines · Emily Deng
When a vf has been reset, the pf wants to get notification to remove the vf out of schedule.
It is very questionable if this is right thing to do. The idea of SR-IOV is that VFs represent a physical device and they should be treated separately from the PF.
In addition to that Keith said, this patch needs better justification.
Thanks
Quoted 16 lines
Solution:
Add the callback function inpci_driversriov_vf_reset_notification. When vf reset happens, then call this callback function.Signed-off-by Emily Deng <[email protected]>--- drivers/pci/pci.c | 8 ++++++++ include/linux/pci.h | 1 + 2 files changed, 9 insertions(+)drivers/pci/pci.c index renamed from drivers/pci/pci.c +0
-0
60230da957e0..aca937b05531100644drivers/pci/pci.c EXPORT_SYMBOL_GPL(pcie_flr); · line 4787
Quoted 8 lines
drivers/pci/pci.c +6
-0@@ -4780,6 +4780,14 @@ EXPORT_SYMBOL_GPL(pcie_flr);*/ int pcie_reset_flr (struct pci_dev *dev,bool probe) { +struct pci_dev *pf_dev; + +if (dev->is_virtfn) { + pf_dev = dev->physfn; +if (pf_dev->driver->sriov_vf_reset_notification) + pf_dev->driver->sriov_vf_reset_notification(pf_dev,dev);
include/linux/pci.h struct pci_driver { · line 927
Quoted 12 lines
drivers/pci/pci.c +2
-0+ } +if (dev->dev_flags &PCI_DEV_FLAGS_NO_FLR_RESET )return -ENOTTY ;include/linux/pci.h index renamed from include/linux/pci.h +0
-0
c69a2cc1f412..4fa31d9b0aa7100644include/linux/pci.h +0
-0@@ -926,6 +926,7 @@ struct pci_driver {int (*sriov_configure)(struct pci_dev *dev,int num_vfs);/* On PF */ int (*sriov_set_msix_vec_count)(struct pci_dev *vf, int
msix_vec_count); /* On PF */Quoted 10 lines
include/linux/pci.h +2
-1u32 (*sriov_get_vf_total_msix)(struct pci_dev *pf); +void (*sriov_vf_reset_notification)(struct pci_dev *pf,struct +pci_dev *vf);const struct pci_error_handlers *err_handler;const struct attribute_group **groups;const struct attribute_group **dev_groups; --2.36.1
Am 04.02.24 um 07:12 schrieb Emily Deng:
Quoted 6 lines
When a vf has been reset, the pf wants to get notification to remove the vf out of schedule.
Solution:
Add the callback function inpci_driversriov_vf_reset_notification. When vf reset happens, then call this callback function.
Well that doesn't make much sense. As other already noted as well a VF should be an encapsulated representation of a physical devices functionality.
AMD implemented that a bit different with a hypervisor to control which PF functionality a VF exposes, but that doesn't mean that we can leak this AMD specific handling into the common Linux PCI subsystem.
Additional to that a technical blocker is that when a VF is passed into a VM you don't have access to the PF any more to make this reset notification.
Regards,
Christian.
Quoted 38 lines
Signed-off-by Emily Deng <[email protected]>--- drivers/pci/pci.c | 8 ++++++++ include/linux/pci.h | 1 + 2 files changed, 9 insertions(+)drivers/pci/pci.c +8
-0@@ -4780,6 +4780,14 @@ EXPORT_SYMBOL_GPL(pcie_flr);*/ int pcie_reset_flr (struct pci_dev *dev,bool probe) { +struct pci_dev *pf_dev; + +if (dev->is_virtfn) { + pf_dev = dev->physfn; +if (pf_dev->driver->sriov_vf_reset_notification) + pf_dev->driver->sriov_vf_reset_notification (pf_dev, dev); + } +if (dev->dev_flags &PCI_DEV_FLAGS_NO_FLR_RESET )return -ENOTTY ;include/linux/pci.h +1
-0@@ -926,6 +926,7 @@ struct pci_driver {int (*sriov_configure)(struct pci_dev *dev,int num_vfs);/* On PF */ int (*sriov_set_msix_vec_count)(struct pci_dev *vf,int msix_vec_count);/* On PF */ u32 (*sriov_get_vf_total_msix)(struct pci_dev *pf); +void (*sriov_vf_reset_notification)(struct pci_dev *pf,struct pci_dev *vf);const struct pci_error_handlers *err_handler;const struct attribute_group **groups;const struct attribute_group **dev_groups;
Quoted 44 lines · On Sun, 4 Feb 2024 14:12:57 +0800 Emily Deng <[email protected]> wrote:
When a vf has been reset, the pf wants to get notification to remove the vf out of schedule.
Solution:
Add the callback function inpci_driversriov_vf_reset_notification. When vf reset happens, then call this callback function.Signed-off-by Emily Deng <[email protected]>--- drivers/pci/pci.c | 8 ++++++++ include/linux/pci.h | 1 + 2 files changed, 9 insertions(+)drivers/pci/pci.c +8
-0@@ -4780,6 +4780,14 @@ EXPORT_SYMBOL_GPL(pcie_flr);*/ int pcie_reset_flr (struct pci_dev *dev,bool probe) { +struct pci_dev *pf_dev; + +if (dev->is_virtfn) { + pf_dev = dev->physfn; +if (pf_dev->driver->sriov_vf_reset_notification) + pf_dev->driver->sriov_vf_reset_notification (pf_dev, dev); + } +if (dev->dev_flags &PCI_DEV_FLAGS_NO_FLR_RESET )return -ENOTTY ;include/linux/pci.h +0
-0@@ -926,6 +926,7 @@ struct pci_driver {int (*sriov_configure)(struct pci_dev *dev,int num_vfs); /* On PF */ int (*sriov_set_msix_vec_count)(struct pci_dev *vf, int
msix_vec_count); /* On PF */ u32 (*sriov_get_vf_total_msix)(structpci_dev*pf);include/linux/pci.h +1
-0+ void (*sriov_vf_reset_notification)(struct pci_dev *pf,struct
pci_dev*vf); const structpci_error_handlers*err_handler;
include/linux/pci.h struct pci_driver { · line 932
Quoted 2 lines
include/linux/pci.h +0
-0const struct attribute_group **groups;const struct attribute_group **dev_groups;
Hi:
I would suggest you can provide a cover letter including a complete picture that tells the background, detailed problem statement, the solutions and plus the users. As this seems very like a generic change, it needs a better justification to convince folks why this is the best solution. Without a complete picture, the solution just looks like a workaround.
Thanks,
Zhi.