vfio: Add an ioctl to reset the device

PATCH Applied Alex Williamson 4 participants 9 messages Apr 19, 2011 VFIO driver

Share & tools

Alex Williamson

author

2011-04-19 20:33 UTC #0 lore raw

PATCH 1/1 vfio: Add an ioctl to reset the device

When using VFIO to assign a device to a guest, we want to make sure the device is quiesced on VM reset to stop all DMA within the guest mapped memory. Add an ioctl which just calls pci_reset_function() and returns whether it succeeds.

Signed-off-by Alex Williamson <[email protected]>

---

We've recently needed to add this functionality for current KVM based device assignment, VFIO should provide a way to do this too. An example of it being used in the Qemu VFIO driver can be found here:

https://github.com/awilliam/qemu-vfio/blob/vfio/hw/vfio.c

 drivers/vfio/vfio_main.c |    4 ++++
 include/linux/vfio.h     |    3 +++
 2 files changed, 7 insertions(+), 0 deletions(-)

drivers/vfio/vfio_main.c +4 -0

index 7e427fc..b9bb692 100644
@@ -490,6 +490,10 @@ static long vfio_unl_ioctl(struct file *filep,
 		ret = vfio_irq_eoi_eventfd(vdev, fd);
 		break;
 
+	case VFIO_RESET_FUNCTION:
+		ret = pci_reset_function(vdev->pdev);
+		break;
+
 	default:
 		return -EINVAL;
 	}

include/linux/vfio.h +3 -0

index f07d8fe..47d9bb9 100644
@@ -215,6 +215,9 @@ struct vfio_dma_map {
 /* Re-enable INTx via eventfd */
 #define	VFIO_IRQ_EOI_EVENTFD	_IOW(';', 110, int)
 
+/* Reset PCI function */
+#define VFIO_RESET_FUNCTION	_IO(';', 111)
+
 /*
  * Reads, writes, and mmaps determine which PCI BAR (or config space)
  * from the high level bits of the file offset

Randy Dunlap

2011-04-19 21:40 UTC #1 lore raw

Quoted 39 lines · On Tue, 19 Apr 2011 14:32:59 -0600 Alex Williamson wrote:

When using VFIO to assign a device to a guest, we want to make sure the device is quiesced on VM reset to stop all DMA within the guest mapped memory. Add an ioctl which just calls pci_reset_function() and returns whether it succeeds.
Signed-off-by Alex Williamson <[email protected]>
---
We've recently needed to add this functionality for current KVM based device assignment, VFIO should provide a way to do this too. An example of it being used in the Qemu VFIO driver can be found here:
https://github.com/awilliam/qemu-vfio/blob/vfio/hw/vfio.c
 drivers/vfio/vfio_main.c |    4 ++++
 include/linux/vfio.h     |    3 +++
 2 files changed, 7 insertions(+), 0 deletions(-)
drivers/vfio/vfio_main.c +4 -0
index 7e427fc..b9bb692 100644
@@ -490,6 +490,10 @@ static long vfio_unl_ioctl(struct file *filep,
 		ret = vfio_irq_eoi_eventfd(vdev, fd);
 		break;
 
+	case VFIO_RESET_FUNCTION:
+		ret = pci_reset_function(vdev->pdev);
+		break;
+
 	default:
 		return -EINVAL;
 	}
include/linux/vfio.h +0 -0
index f07d8fe..47d9bb9 100644

include/linux/vfio.h struct vfio_dma_map { · line 223

Quoted 9 lines

include/linux/vfio.h +3 -0

@@ -215,6 +215,9 @@ struct vfio_dma_map {
 /* Re-enable INTx via eventfd */
 #define	VFIO_IRQ_EOI_EVENTFD	_IOW(';', 110, int)
 
+/* Reset PCI function */
+#define VFIO_RESET_FUNCTION	_IO(';', 111)
+
 /*
  * Reads, writes, and mmaps determine which PCI BAR (or config space)
  * from the high level bits of the file offset

I can't find include/linux/vfio.h in linux-next or mainline git, but ioctls need to be documented in Documentation/ioctl/ioctl-number.txt....

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

Chris Wright

2011-04-19 21:43 UTC #2 lore raw

Quoted 2 lines · * Randy Dunlap ([email protected]) wrote:

I can't find include/linux/vfio.h in linux-next or mainline git, but ioctls need to be documented in Documentation/ioctl/ioctl-number.txt....

It is in the full patchset: https://github.com/pugs/vfio-linux-2.6

Chris Wright

2011-04-19 22:07 UTC #3 lore raw

In reply to Alex Williamson

Quoted 4 lines · * Alex Williamson ([email protected]) wrote:

When using VFIO to assign a device to a guest, we want to make sure the device is quiesced on VM reset to stop all DMA within the guest mapped memory. Add an ioctl which just calls pci_reset_function() and returns whether it succeeds.

Shouldn't there be a reset when binding/unbinding vfio to/from a pci device?

Alex Williamson

author

2011-04-19 22:22 UTC #4 lore raw

Quoted 8 lines · Chris Wright

Quoted 4 lines · * Alex Williamson ([email protected]) wrote:
When using VFIO to assign a device to a guest, we want to make sure the device is quiesced on VM reset to stop all DMA within the guest mapped memory. Add an ioctl which just calls pci_reset_function() and returns whether it succeeds.
Shouldn't there be a reset when binding/unbinding vfio to/from a pci device?

There's already one when the /dev/vfioX file is opened, we should add another on release, and probably add the same PCI save state store/load that I'm proposing for KVM across those. Thanks,

Alex

Chris Wright

2011-04-19 22:26 UTC #5 lore raw

Quoted 7 lines · * Alex Williamson ([email protected]) wrote:

Quoted 6 lines · Chris Wright
Quoted 4 lines · * Alex Williamson ([email protected]) wrote:
When using VFIO to assign a device to a guest, we want to make sure the device is quiesced on VM reset to stop all DMA within the guest mapped memory. Add an ioctl which just calls pci_reset_function() and returns whether it succeeds.

Quoted 6 lines

Quoted 2 lines
Shouldn't there be a reset when binding/unbinding vfio to/from a pci device?
There's already one when the /dev/vfioX file is opened, we should add another on release, and probably add the same PCI save state store/load that I'm proposing for KVM across those. Thanks,

Hmm, I looked and didn't see it, hence the question.

Alex Williamson

author

2011-04-19 22:30 UTC #6 lore raw

Quoted 8 lines · Chris Wright

Quoted 7 lines · * Alex Williamson ([email protected]) wrote:
Quoted 6 lines · Chris Wright
Quoted 4 lines · * Alex Williamson ([email protected]) wrote:
When using VFIO to assign a device to a guest, we want to make sure the device is quiesced on VM reset to stop all DMA within the guest mapped memory. Add an ioctl which just calls pci_reset_function() and returns whether it succeeds.

Quoted 8 lines

Quoted 6 lines
Quoted 2 lines
Shouldn't there be a reset when binding/unbinding vfio to/from a pci device?
There's already one when the /dev/vfioX file is opened, we should add another on release, and probably add the same PCI save state store/load that I'm proposing for KVM across those. Thanks,
Hmm, I looked and didn't see it, hence the question.

vfio_open() -> pci_reset_function()
https://github.com/pugs/vfio-linux-2.6/blob/vfio/drivers/vfio/vfio_main.c

Chris Wright

2011-04-19 22:37 UTC #7 lore raw

Quoted 12 lines · * Alex Williamson ([email protected]) wrote:

Quoted 11 lines · Chris Wright
Quoted 10 lines · * Alex Williamson ([email protected]) wrote:
Quoted 8 lines · Chris Wright
Quoted 4 lines · * Alex Williamson ([email protected]) wrote:
When using VFIO to assign a device to a guest, we want to make sure the device is quiesced on VM reset to stop all DMA within the guest mapped memory. Add an ioctl which just calls pci_reset_function() and returns whether it succeeds.
Shouldn't there be a reset when binding/unbinding vfio to/from a pci device?

Quoted 8 lines

Quoted 5 lines
Quoted 3 lines
There's already one when the /dev/vfioX file is opened, we should add another on release, and probably add the same PCI save state store/load that I'm proposing for KVM across those. Thanks,
Hmm, I looked and didn't see it, hence the question.
vfio_open() -> pci_reset_function()
https://github.com/pugs/vfio-linux-2.6/blob/vfio/drivers/vfio/vfio_main.c

Got it, thanks Alex.

Tom Lyon

Applied

2011-04-19 23:22 UTC #8 lore raw

In reply to Alex Williamson

Quoted 39 lines · Alex Williamson

When using VFIO to assign a device to a guest, we want to make sure the device is quiesced on VM reset to stop all DMA within the guest mapped memory. Add an ioctl which just calls pci_reset_function() and returns whether it succeeds.
Signed-off-by Alex Williamson <[email protected]>
---
We've recently needed to add this functionality for current KVM based device assignment, VFIO should provide a way to do this too. An example of it being used in the Qemu VFIO driver can be found here:
https://github.com/awilliam/qemu-vfio/blob/vfio/hw/vfio.c
 drivers/vfio/vfio_main.c |    4 ++++
 include/linux/vfio.h     |    3 +++
 2 files changed, 7 insertions(+), 0 deletions(-)
drivers/vfio/vfio_main.c +4 -0
index 7e427fc..b9bb692 100644
@@ -490,6 +490,10 @@ static long vfio_unl_ioctl(struct file *filep,
 		ret = vfio_irq_eoi_eventfd(vdev, fd);
 		break;
 
+	case VFIO_RESET_FUNCTION:
+		ret = pci_reset_function(vdev->pdev);
+		break;
+
 	default:
 		return -EINVAL;
 	}
include/linux/vfio.h +0 -0
index f07d8fe..47d9bb9 100644

include/linux/vfio.h struct vfio_dma_map { · line 223

Quoted 9 lines

include/linux/vfio.h +3 -0

@@ -215,6 +215,9 @@ struct vfio_dma_map {
 /* Re-enable INTx via eventfd */
 #define	VFIO_IRQ_EOI_EVENTFD	_IOW(';', 110, int)
 
+/* Reset PCI function */
+#define VFIO_RESET_FUNCTION	_IO(';', 111)
+
 /*
  * Reads, writes, and mmaps determine which PCI BAR (or config space)
  * from the high level bits of the file offset

Applied.