vfio/pci: Support dynamic allocation of MSI-X interrupts
Subsystems (2)
cover vfio/pci: Support dynamic allocation of MSI-X interrupts
== Cover letter ==
Qemu allocates interrupts incrementally at the time the guest unmasks an interrupt, for example each time a Linux guest runs request_irq().
Dynamic allocation of MSI-X interrupts was not possible until v6.2 [1]. This prompted Qemu to, when allocating a new interrupt, first release all previously allocated interrupts (including disable of MSI-X) followed by re-allocation of all interrupts that includes the new interrupt. Please see [2] for a detailed discussion about this issue.
Releasing and re-allocating interrupts may be acceptable if all interrupts are unmasked during device initialization. If unmasking of interrupts occur during runtime this may result in lost interrupts. For example, consider an accelerator device with multiple work queues, each work queue having a dedicated interrupt. A work queue can be enabled at any time with its associated interrupt unmasked while other work queues are already active. Having all interrupts released and MSI-X disabled to enable the new work queue will impact active work queues.
This series builds on the recent interrupt sub-system core changes that added support for dynamic MSI-X allocation after initial MSI-X enabling.
Add support for dynamic MSI-X allocation to vfio-pci. A flag indicating lack of support for dynamic allocation already exist:VFIO_IRQ_INFO_NORESIZE and has always been set for MSI and MSI-X. With support for dynamic MSI-X the flag is cleared for MSI-X, enabling Qemu to modify its behavior.
== Why is this an RFC ? ==
vfio support for dynamic MSI-X needs to work with existing user space as well as upcoming user space that takes advantage of this feature. I would appreciate guidance on the expectations and requirements surrounding error handling when considering existing user space.
For example, consider the following scenario:
Start: Consider a passthrough device that supports 10 MSI-X interrupts
(0 to 9) and existing Qemu allocated interrupts 0 to 4.
Scenario: Qemu (hypothetically) attempts to associate a new action to
interrupts 0 to 7. Early checking of this range in vfio_set_irqs_validate_and_prepare() will pass since it is a valid range for the device. What happens after the early checking is considered next:
Current behavior (before this series): Since the provided range, 0 - 7,
exceeds the allocated range, no action will be taken on existing allocated interrupts 0 - 4 and the Qemu request will fail without making any state changes.
New behavior (with this series): Since vfio supports dynamic MSI-X new
interrupts will be allocated for vectors 5, 6, and 7. Please note that this changes the behavior encountered by unmodified Qemu: new interrupts are allocated instead of returning an error. Even more, since the range provided by Qemu spans 0 - 7, a failure during allocation of 5, 6, or 7 will result in release of entire range.
This series aims to maintain existing error behavior for MSI (please see "vfio/pci: Remove interrupt context counter") but I would appreciate your guidance on whether existing error behavior should be maintained for MSI-X and how to do so if it is a requirement.
Any feedback is appreciated
Reinette
[1] commit 34026364df8e ("PCI/MSI: Provide post-enable dynamic allocation interfaces for MSI-X")
[2] https://lore.kernel.org/kvm/MWHPR11MB188603D0D809C1079F5817DC8C099@MWHPR11MB1886.namprd11.prod.outlook.com/#t
Reinette Chatre (8):
vfio/pci: Consolidate irq cleanup on MSI/MSI-X disable vfio/pci: Remove negative check on unsigned vector vfio/pci: Prepare for dynamic interrupt context storage vfio/pci: Use xarray for interrupt context storage vfio/pci: Remove interrupt context counter vfio/pci: Move to single error path
vfio/pci: Support dynamic MSI-x
vfio/pci: Clear VFIO_IRQ_INFO_NORESIZE for MSI-X
drivers/vfio/pci/vfio_pci_core.c | 3 +- drivers/vfio/pci/vfio_pci_intrs.c | 376 ++++++++++++++++++++++-------- include/linux/vfio_pci_core.h | 3 +- include/uapi/linux/vfio.h | 3 + 4 files changed, 286 insertions(+), 99 deletions(-)
base-commit: eeac8ede17557680855031c6f305ece2378af326
Signature · 1 line
2.34.1
RFC PATCH 1/8 vfio/pci: Consolidate irq cleanup on MSI/MSI-X disable
vfio_msi_disable() releases all previously allocated state associated with each interrupt before disabling MSI/MSI-X.
vfio_msi_disable() iterates twice over the interrupt state:
first directly with a for loop to do virqfd cleanup, followed by another for loop within vfio_msi_set_block() that releases the interrupt and its state using vfio_msi_set_vector_signal().
Simplify interrupt cleanup by iterating over allocated interrupts once.
--- drivers/vfio/pci/vfio_pci_intrs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/vfio/pci/vfio_pci_intrs.c +1 -2
@@ -426,10 +426,9 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix)for (i =0 ; i < vdev->num_ctx; i++) {vfio_virqfd_disable (&vdev->ctx[i].unmask);vfio_virqfd_disable (&vdev->ctx[i].mask); +vfio_msi_set_vector_signal (vdev, i,-1 , msix); } -vfio_msi_set_block (vdev,0 , vdev->num_ctx,NULL , msix); - cmd =vfio_pci_memory_lock_and_enable (vdev);pci_free_irq_vectors (pdev);vfio_pci_memory_unlock_and_restore (vdev, cmd);
Signature · 1 line
2.34.1
RFC PATCH 2/8 vfio/pci: Remove negative check on unsigned vector
In reply to Reinette ChatreUser space provides the vector as an unsigned int that is checked early for validity (vfio_set_irqs_validate_and_prepare()).
A later negative check of the provided vector is not necessary.
Remove the negative check and ensure the type used for the vector is consistent as an unsigned int.
--- drivers/vfio/pci/vfio_pci_intrs.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
drivers/vfio/pci/vfio_pci_intrs.c +6 -5
@@ -317,14 +317,14 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi }static int vfio_msi_set_vector_signal (struct vfio_pci_core_device *vdev, -int vector,int fd,bool msix) +unsigned int vector,int fd,bool msix) {struct pci_dev *pdev = vdev->pdev;struct eventfd_ctx *trigger;int irq, ret;u16 cmd; -if (vector <0 || vector >= vdev->num_ctx) +if (vector >= vdev->num_ctx)return -EINVAL ; irq =pci_irq_vector (pdev, vector);
@@ -399,7 +399,8 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev,static int vfio_msi_set_block (struct vfio_pci_core_device *vdev,unsigned start,unsigned count,int32_t *fds,bool msix) { -int i, j, ret =0 ; +int i, ret =0 ; +unsigned int j;if (start >= vdev->num_ctx || start + count > vdev->num_ctx)return -EINVAL ;
@@ -420,7 +421,7 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start,static void vfio_msi_disable (struct vfio_pci_core_device *vdev,bool msix) {struct pci_dev *pdev = vdev->pdev; -int i; +unsigned int i;u16 cmd;for (i =0 ; i < vdev->num_ctx; i++) {
@@ -542,7 +543,7 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { - int i; + unsigned int i;bool msix = (index ==VFIO_PCI_MSIX_IRQ_INDEX ) ? true : false;if (irq_is (vdev, index) && !count && (flags &VFIO_IRQ_SET_DATA_NONE )) {
Signature · 1 line
2.34.1
RFC PATCH 3/8 vfio/pci: Prepare for dynamic interrupt context storage
In reply to Reinette ChatreInterrupt context storage is statically allocated at the time interrupts are enabled. Following allocation, the interrupt context is managed by directly accessing the elements of the array using the vector as index.
It is possible to allocate additional MSI-X vectors after MSI-X has been enabled. Dynamic storage of interrupt context is needed to support adding new MSI-X vectors after initial enabling.
Replace direct access of array elements with pointers to the array elements. Doing so reduces impact of moving to a new data structure. Move interactions with the array to helpers to mostly contain changes needed to transition to a dynamic data structure.
No functional change intended.
--- drivers/vfio/pci/vfio_pci_intrs.c | 206 ++++++++++++++++++++---------- 1 file changed, 140 insertions(+), 66 deletions(-)
drivers/vfio/pci/vfio_pci_intrs.c +140 -66
@@ -48,6 +48,31 @@ static bool is_irq_none(struct vfio_pci_core_device *vdev) vdev->irq_type == VFIO_PCI_MSIX_IRQ_INDEX); } +static +struct vfio_pci_irq_ctx *vfio_irq_ctx_get (struct vfio_pci_core_device *vdev, +unsigned long index) +{ +if (index >= vdev->num_ctx) +return NULL ; +return &vdev->ctx[index]; +} + +static void vfio_irq_ctx_free_all (struct vfio_pci_core_device *vdev) +{ +kfree (vdev->ctx); +} + +static int vfio_irq_ctx_alloc_num (struct vfio_pci_core_device *vdev, +unsigned long num) +{ + vdev->ctx =kcalloc (num,sizeof (struct vfio_pci_irq_ctx ), +GFP_KERNEL_ACCOUNT ); +if (!vdev->ctx) +return -ENOMEM ; + +return 0 ; +} +/* * INTx */
@@ -55,17 +80,28 @@ static void vfio_send_intx_eventfd(void *opaque, void *unused) {struct vfio_pci_core_device *vdev = opaque; -if (likely (is_intx (vdev) && !vdev->virq_disabled)) -eventfd_signal (vdev->ctx[0 ].trigger,1 ); +if (likely (is_intx (vdev) && !vdev->virq_disabled)) { +struct vfio_pci_irq_ctx *ctx; + + ctx =vfio_irq_ctx_get (vdev,0 ); +if (!ctx) +return ; +eventfd_signal (ctx->trigger,1 ); + } }/* Returns true if the INTx vfio_pci_irq_ctx.masked value is changed. */ bool vfio_pci_intx_mask (struct vfio_pci_core_device *vdev) {struct pci_dev *pdev = vdev->pdev; +struct vfio_pci_irq_ctx *ctx;unsigned long flags;bool masked_changed = false; + ctx =vfio_irq_ctx_get (vdev,0 ); +if (!ctx) +return masked_changed; +spin_lock_irqsave (&vdev->irqlock, flags);/*
@@ -77,7 +113,7 @@ bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev)if (unlikely (!is_intx (vdev))) {if (vdev->pci_2_3)pci_intx (pdev,0 ); - }else if (!vdev->ctx[0 ].masked) { + }else if (!ctx->masked) {/* * Can't use check_and_mask here because we always want to * mask, not just when something is pending.
@@ -87,7 +123,7 @@ bool vfio_pci_intx_mask(struct vfio_pci_core_device *vdev)else disable_irq_nosync (pdev->irq); - vdev->ctx[0 ].masked = true; + ctx->masked = true; masked_changed = true; }
@@ -105,9 +141,14 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *unused) {struct vfio_pci_core_device *vdev = opaque;struct pci_dev *pdev = vdev->pdev; +struct vfio_pci_irq_ctx *ctx;unsigned long flags;int ret =0 ; + ctx =vfio_irq_ctx_get (vdev,0 ); +if (!ctx) +return ret; +spin_lock_irqsave (&vdev->irqlock, flags);/*
@@ -117,7 +158,7 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *unused)if (unlikely (!is_intx (vdev))) {if (vdev->pci_2_3)pci_intx (pdev,1 ); - }else if (vdev->ctx[0 ].masked && !vdev->virq_disabled) { + }else if (ctx->masked && !vdev->virq_disabled) {/* * A pending interrupt here would immediately trigger, * but we can avoid that overhead by just re-sending
@@ -129,7 +170,7 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *unused) }else enable_irq (pdev->irq); - vdev->ctx[0 ].masked = (ret >0 ); + ctx->masked = (ret >0 ); }spin_unlock_irqrestore (&vdev->irqlock, flags);
@@ -146,18 +187,23 @@ void vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev)static irqreturn_t vfio_intx_handler (int irq,void *dev_id) {struct vfio_pci_core_device *vdev = dev_id; +struct vfio_pci_irq_ctx *ctx;unsigned long flags;int ret =IRQ_NONE ; + ctx =vfio_irq_ctx_get (vdev,0 ); +if (!ctx) +return ret; +spin_lock_irqsave (&vdev->irqlock, flags);if (!vdev->pci_2_3) {disable_irq_nosync (vdev->pdev->irq); - vdev->ctx[0 ].masked = true; + ctx->masked = true; ret =IRQ_HANDLED ; - }else if (!vdev->ctx[0 ].masked &&/* may be shared */ + }else if (!ctx->masked &&/* may be shared */ pci_check_and_mask_intx (vdev->pdev)) { - vdev->ctx[0 ].masked = true; + ctx->masked = true; ret =IRQ_HANDLED ; }
@@ -171,15 +217,24 @@ static irqreturn_t vfio_intx_handler(int irq, void *dev_id)static int vfio_intx_enable (struct vfio_pci_core_device *vdev) { +struct vfio_pci_irq_ctx *ctx; +int ret; +if (!is_irq_none (vdev))return -EINVAL ;if (!vdev->pdev->irq)return -ENODEV ; - vdev->ctx =kzalloc (sizeof (struct vfio_pci_irq_ctx ),GFP_KERNEL_ACCOUNT ); -if (!vdev->ctx) -return -ENOMEM ; + ret =vfio_irq_ctx_alloc_num (vdev,1 ); +if (ret) +return ret; + + ctx =vfio_irq_ctx_get (vdev,0 ); +if (!ctx) { +vfio_irq_ctx_free_all (vdev); +return -EINVAL ; + } vdev->num_ctx =1 ;
@@ -189,9 +244,9 @@ static int vfio_intx_enable(struct vfio_pci_core_device *vdev)* here, non-PCI-2.3 devices will have to wait until the * interrupt is enabled. */ - vdev->ctx[0 ].masked = vdev->virq_disabled; + ctx->masked = vdev->virq_disabled;if (vdev->pci_2_3) -pci_intx (vdev->pdev, !vdev->ctx[0 ].masked); +pci_intx (vdev->pdev, !ctx->masked); vdev->irq_type =VFIO_PCI_INTX_IRQ_INDEX ;
@@ -202,41 +257,46 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, int fd) {struct pci_dev *pdev = vdev->pdev;unsigned long irqflags =IRQF_SHARED ; +struct vfio_pci_irq_ctx *ctx;struct eventfd_ctx *trigger;unsigned long flags;int ret; -if (vdev->ctx[0 ].trigger) { + ctx =vfio_irq_ctx_get (vdev,0 ); +if (!ctx) +return -EINVAL ; + +if (ctx->trigger) {free_irq (pdev->irq, vdev); -kfree (vdev->ctx[0 ].name); -eventfd_ctx_put (vdev->ctx[0 ].trigger); - vdev->ctx[0 ].trigger =NULL ; +kfree (ctx->name); +eventfd_ctx_put (ctx->trigger); + ctx->trigger =NULL ; }if (fd <0 )/* Disable only */ return 0 ; - vdev->ctx[0 ].name =kasprintf (GFP_KERNEL_ACCOUNT ,"vfio-intx(%s)" , -pci_name (pdev)); -if (!vdev->ctx[0 ].name) + ctx->name =kasprintf (GFP_KERNEL_ACCOUNT ,"vfio-intx(%s)" , +pci_name (pdev)); +if (!ctx->name)return -ENOMEM ; trigger =eventfd_ctx_fdget (fd);if (IS_ERR (trigger)) { -kfree (vdev->ctx[0 ].name); +kfree (ctx->name);return PTR_ERR (trigger); } - vdev->ctx[0 ].trigger = trigger; + ctx->trigger = trigger;if (!vdev->pci_2_3) irqflags =0 ; ret =request_irq (pdev->irq, vfio_intx_handler, - irqflags, vdev->ctx[0 ].name, vdev); + irqflags, ctx->name, vdev);if (ret) { - vdev->ctx[0 ].trigger =NULL ; -kfree (vdev->ctx[0 ].name); + ctx->trigger =NULL ; +kfree (ctx->name);eventfd_ctx_put (trigger);return ret; }
@@ -246,7 +306,7 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, int fd)* disable_irq won't. */ spin_lock_irqsave (&vdev->irqlock, flags); -if (!vdev->pci_2_3 && vdev->ctx[0 ].masked) +if (!vdev->pci_2_3 && ctx->masked)disable_irq_nosync (pdev->irq);spin_unlock_irqrestore (&vdev->irqlock, flags);
@@ -255,12 +315,17 @@ static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, int fd)static void vfio_intx_disable (struct vfio_pci_core_device *vdev) { -vfio_virqfd_disable (&vdev->ctx[0 ].unmask); -vfio_virqfd_disable (&vdev->ctx[0 ].mask); +struct vfio_pci_irq_ctx *ctx; + + ctx =vfio_irq_ctx_get (vdev,0 ); +if (ctx) { +vfio_virqfd_disable (&ctx->unmask); +vfio_virqfd_disable (&ctx->mask); + }vfio_intx_set_signal (vdev,-1 ); vdev->irq_type =VFIO_PCI_NUM_IRQS ; vdev->num_ctx =0 ; -kfree (vdev->ctx); +vfio_irq_ctx_free_all (vdev); }/*
@@ -284,10 +349,9 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msiif (!is_irq_none (vdev))return -EINVAL ; - vdev->ctx =kcalloc (nvec,sizeof (struct vfio_pci_irq_ctx ), -GFP_KERNEL_ACCOUNT ); -if (!vdev->ctx) -return -ENOMEM ; + ret =vfio_irq_ctx_alloc_num (vdev, nvec); +if (ret) +return ret;/* return the number of supported vectors if we can't get all: */ cmd =vfio_pci_memory_lock_and_enable (vdev);
@@ -296,7 +360,7 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msiif (ret >0 )pci_free_irq_vectors (pdev);vfio_pci_memory_unlock_and_restore (vdev, cmd); -kfree (vdev->ctx); +vfio_irq_ctx_free_all (vdev);return ret; }vfio_pci_memory_unlock_and_restore (vdev, cmd);
@@ -320,6 +384,7 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, unsigned int vector, int fd, bool msix) { struct pci_dev *pdev = vdev->pdev; +struct vfio_pci_irq_ctx *ctx;struct eventfd_ctx *trigger;int irq, ret;u16 cmd;
@@ -327,33 +392,33 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev,if (vector >= vdev->num_ctx)return -EINVAL ; + ctx =vfio_irq_ctx_get (vdev, vector); +if (!ctx) +return -EINVAL ; irq =pci_irq_vector (pdev, vector); -if (vdev->ctx[vector].trigger) { -irq_bypass_unregister_producer (&vdev->ctx[vector].producer); +if (ctx->trigger) { +irq_bypass_unregister_producer (&ctx->producer); cmd =vfio_pci_memory_lock_and_enable (vdev); -free_irq (irq, vdev->ctx[vector].trigger); +free_irq (irq, ctx->trigger);vfio_pci_memory_unlock_and_restore (vdev, cmd); - -kfree (vdev->ctx[vector].name); -eventfd_ctx_put (vdev->ctx[vector].trigger); - vdev->ctx[vector].trigger =NULL ; +kfree (ctx->name); +eventfd_ctx_put (ctx->trigger); + ctx->trigger =NULL ; }if (fd <0 )return 0 ; - vdev->ctx[vector].name =kasprintf (GFP_KERNEL_ACCOUNT , -"vfio-msi%s[%d](%s)" , - msix ?"x" :"" , vector, -pci_name (pdev)); -if (!vdev->ctx[vector].name) + ctx->name =kasprintf (GFP_KERNEL_ACCOUNT ,"vfio-msi%s[%d](%s)" , + msix ?"x" :"" , vector,pci_name (pdev)); +if (!ctx->name)return -ENOMEM ; trigger =eventfd_ctx_fdget (fd);if (IS_ERR (trigger)) { -kfree (vdev->ctx[vector].name); +kfree (ctx->name);return PTR_ERR (trigger); }
@@ -372,26 +437,25 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev,pci_write_msi_msg (irq, &msg); } - ret =request_irq (irq, vfio_msihandler,0 , - vdev->ctx[vector].name, trigger); + ret =request_irq (irq, vfio_msihandler,0 , ctx->name, trigger);vfio_pci_memory_unlock_and_restore (vdev, cmd);if (ret) { -kfree (vdev->ctx[vector].name); +kfree (ctx->name);eventfd_ctx_put (trigger);return ret; } - vdev->ctx[vector].producer.token = trigger; - vdev->ctx[vector].producer.irq = irq; - ret =irq_bypass_register_producer (&vdev->ctx[vector].producer); + ctx->producer.token = trigger; + ctx->producer.irq = irq; + ret =irq_bypass_register_producer (&ctx->producer);if (unlikely (ret)) {dev_info (&pdev->dev,"irq bypass producer (token %p) registration fails: %d\n" , - vdev->ctx[vector].producer.token, ret); + ctx->producer.token, ret); - vdev->ctx[vector].producer.token =NULL ; + ctx->producer.token =NULL ; } - vdev->ctx[vector].trigger = trigger; + ctx->trigger = trigger;return 0 ; }
@@ -421,13 +485,17 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start,static void vfio_msi_disable (struct vfio_pci_core_device *vdev,bool msix) {struct pci_dev *pdev = vdev->pdev; +struct vfio_pci_irq_ctx *ctx;unsigned int i;u16 cmd;for (i =0 ; i < vdev->num_ctx; i++) { -vfio_virqfd_disable (&vdev->ctx[i].unmask); -vfio_virqfd_disable (&vdev->ctx[i].mask); -vfio_msi_set_vector_signal (vdev, i,-1 , msix); + ctx =vfio_irq_ctx_get (vdev, i); +if (ctx) { +vfio_virqfd_disable (&ctx->unmask); +vfio_virqfd_disable (&ctx->mask); +vfio_msi_set_vector_signal (vdev, i,-1 , msix); + } } cmd =vfio_pci_memory_lock_and_enable (vdev);
@@ -443,7 +511,7 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) vdev->irq_type =VFIO_PCI_NUM_IRQS ; vdev->num_ctx =0 ; -kfree (vdev->ctx); +vfio_irq_ctx_free_all (vdev); }/*
@@ -463,14 +531,18 @@ static int vfio_pci_set_intx_unmask(struct vfio_pci_core_device *vdev,if (unmask)vfio_pci_intx_unmask (vdev); }else if (flags &VFIO_IRQ_SET_DATA_EVENTFD ) { +struct vfio_pci_irq_ctx *ctx =vfio_irq_ctx_get (vdev,0 );int32_t fd = *(int32_t *)data; + +if (!ctx) +return -EINVAL ;if (fd >=0 )return vfio_virqfd_enable ((void *) vdev, vfio_pci_intx_unmask_handler, vfio_send_intx_eventfd,NULL , - &vdev->ctx[0 ].unmask, fd); + &ctx->unmask, fd); -vfio_virqfd_disable (&vdev->ctx[0 ].unmask); +vfio_virqfd_disable (&ctx->unmask); }return 0 ;
@@ -543,6 +615,7 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { + struct vfio_pci_irq_ctx *ctx;unsigned int i;bool msix = (index ==VFIO_PCI_MSIX_IRQ_INDEX ) ? true : false;
@@ -577,14 +650,15 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev,return -EINVAL ;for (i = start; i < start + count; i++) { -if (!vdev->ctx[i].trigger) + ctx =vfio_irq_ctx_get (vdev, i); +if (!ctx || !ctx->trigger)continue ;if (flags &VFIO_IRQ_SET_DATA_NONE ) { -eventfd_signal (vdev->ctx[i].trigger,1 ); +eventfd_signal (ctx->trigger,1 ); }else if (flags &VFIO_IRQ_SET_DATA_BOOL ) {uint8_t *bools = data;if (bools[i - start]) -eventfd_signal (vdev->ctx[i].trigger,1 ); +eventfd_signal (ctx->trigger,1 ); } }return 0 ;
Signature · 1 line
2.34.1
RFC PATCH 4/8 vfio/pci: Use xarray for interrupt context storage
In reply to Reinette ChatreInterrupt context is statically allocated at the time interrupts are enabled. Following allocation, the context is managed by directly accessing the elements of the array using the vector as index. The storage is released when interrupts are disabled.
It is possible to dynamically allocate a single MSI-X index after MSI-X has been enabled. A dynamic storage for interrupt context is needed to support this. Replace the interrupt context array with an xarray (similar to what the core uses as store for MSI descriptors) that can support the dynamic expansion while maintaining the custom that uses the vector as index.
Use the new data storage to allocate all elements once and free all elements together. Dynamic usage follows.
Create helpers with understanding that it is only possible to (after initial MSI-X enabling) allocate a single MSI-X index at a time. The only time multiple MSI-X are allocated is during initial MSI-X enabling where failure results in no allocations.
--- drivers/vfio/pci/vfio_pci_core.c | 1 + drivers/vfio/pci/vfio_pci_intrs.c | 63 +++++++++++++++++++++++-------- include/linux/vfio_pci_core.h | 2 +- 3 files changed, 49 insertions(+), 17 deletions(-)
drivers/vfio/pci/vfio_pci_core.c +1 -0
@@ -2102,6 +2102,7 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)INIT_LIST_HEAD (&vdev->vma_list);INIT_LIST_HEAD (&vdev->sriov_pfs_item);init_rwsem (&vdev->memory_lock); +xa_init (&vdev->ctx);return 0 ; }
drivers/vfio/pci/vfio_pci_intrs.c +47 -16
88 lines not on this page · raw message
include/linux/vfio_pci_core.h +1 -1
@@ -59,7 +59,7 @@ struct vfio_pci_core_device {struct perm_bits *msi_perm;spinlock_t irqlock;struct mutex igate; -struct vfio_pci_irq_ctx *ctx; +struct xarray ctx;int num_ctx;int irq_type;int num_regions;
Signature · 1 line
2.34.1
RFC PATCH 5/8 vfio/pci: Remove interrupt context counter
In reply to Reinette Chatrestruct vfio_pci_core_device::num_ctx counts how many interrupt contexts have been allocated. When all interrupt contexts are allocated simultaneously num_ctx provides the upper bound of all vectors that can be used as indices into the interrupt context array.
With the upcoming support for dynamic MSI-X the number of interrupt contexts does not necessarily span the range of allocated interrupts. Consequently, num_ctx is no longer a trusted upper bound for valid indices.
Stop using num_ctx to determine if a provided vector is valid, use the existence of interrupt context directly.
Introduce a helper that ensures a provided interrupt range is allocated before any user requested action is taken. This maintains existing behavior (early exit without modifications) when user space attempts to modify a range of vectors that includes unallocated interrupts.
The checks that ensure that user space provides a range of vectors that is valid for the device are untouched.
---
Existing behavior on error paths is not maintained for MSI-X when adding support for dynamic MSI-X. Please see maintainer comments associated with "vfio/pci: Support dynamic MSI-x".
drivers/vfio/pci/vfio_pci_intrs.c | 30 ++++++++++++++++++++---------- include/linux/vfio_pci_core.h | 1 - 2 files changed, 20 insertions(+), 11 deletions(-)
drivers/vfio/pci/vfio_pci_intrs.c +20 -10
@@ -107,6 +107,21 @@ static int vfio_irq_ctx_alloc_num(struct vfio_pci_core_device *vdev,return ret; } +static bool vfio_irq_ctx_range_allocated (struct vfio_pci_core_device *vdev, +unsigned int start,unsigned int count) +{ +struct vfio_pci_irq_ctx *ctx; +unsigned int i; + +for (i = start; i < start + count; i++) { + ctx =xa_load (&vdev->ctx, i); +if (!ctx) +return false; + } + +return true; +} +/* * INTx */
@@ -270,8 +285,6 @@ static int vfio_intx_enable(struct vfio_pci_core_device *vdev)return -EINVAL ; } - vdev->num_ctx =1 ; -/* * If the virtual interrupt is masked, restore it. Devices * supporting DisINTx can be masked at the hardware level
@@ -358,7 +371,6 @@ static void vfio_intx_disable(struct vfio_pci_core_device *vdev) }vfio_intx_set_signal (vdev,-1 ); vdev->irq_type =VFIO_PCI_NUM_IRQS ; - vdev->num_ctx =0 ;vfio_irq_ctx_free_all (vdev); }
@@ -399,7 +411,6 @@ static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msi }vfio_pci_memory_unlock_and_restore (vdev, cmd); - vdev->num_ctx = nvec; vdev->irq_type = msix ?VFIO_PCI_MSIX_IRQ_INDEX :VFIO_PCI_MSI_IRQ_INDEX ;
@@ -423,9 +434,6 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev,int irq, ret;u16 cmd; -if (vector >= vdev->num_ctx) -return -EINVAL ; - ctx =vfio_irq_ctx_get (vdev, vector);if (!ctx)return -EINVAL ;
@@ -500,7 +508,7 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start,int i, ret =0 ;unsigned int j; -if (start >= vdev->num_ctx || start + count > vdev->num_ctx) +if (!vfio_irq_ctx_range_allocated (vdev, start, count))return -EINVAL ;for (i =0 , j = start; i < count && !ret; i++, j++) {
@@ -541,7 +549,6 @@ static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix)pci_intx (pdev,0 ); vdev->irq_type =VFIO_PCI_NUM_IRQS ; - vdev->num_ctx =0 ;vfio_irq_ctx_free_all (vdev); }
@@ -677,7 +684,10 @@ static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev,return ret; } -if (!irq_is (vdev, index) || start + count > vdev->num_ctx) +if (!irq_is (vdev, index)) +return -EINVAL ; + +if (!vfio_irq_ctx_range_allocated (vdev, start, count))return -EINVAL ;for (i = start; i < start + count; i++) {
include/linux/vfio_pci_core.h +0 -1
@@ -60,7 +60,6 @@ struct vfio_pci_core_device {spinlock_t irqlock;struct mutex igate;struct xarray ctx; -int num_ctx;int irq_type;int num_regions;struct vfio_pci_region *region;
Signature · 1 line
2.34.1
RFC PATCH 6/8 vfio/pci: Move to single error path
In reply to Reinette ChatreThe creation and release of interrupt context involves several steps that can fail. Cleanup after failure is done when the error is encountered, resulting in some repetitive code.
Support for dynamic MSI-X will introduce more steps during interrupt context creation and release.
Transition to centralized exit path in preparation for dynamic MSI-X to eliminate duplicate error handling code. Ensure no remaining state refers to freed memory.
--- drivers/vfio/pci/vfio_pci_intrs.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
drivers/vfio/pci/vfio_pci_intrs.c +11 -7
@@ -460,8 +460,8 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, trigger =eventfd_ctx_fdget (fd);if (IS_ERR (trigger)) { -kfree (ctx->name); -return PTR_ERR (trigger); + ret =PTR_ERR (trigger); + goto out_free_name; }/*
@@ -481,11 +481,8 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, ret =request_irq (irq, vfio_msihandler,0 , ctx->name, trigger);vfio_pci_memory_unlock_and_restore (vdev, cmd); -if (ret) { -kfree (ctx->name); -eventfd_ctx_put (trigger); -return ret; - } +if (ret) + goto out_put_eventfd_ctx; ctx->producer.token = trigger; ctx->producer.irq = irq;
@@ -500,6 +497,13 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, ctx->trigger = trigger;return 0 ; + +out_put_eventfd_ctx: +eventfd_ctx_put (trigger); +out_free_name: +kfree (ctx->name); + ctx->name =NULL ; +return ret; } static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start,
Signature · 1 line
2.34.1
RFC PATCH 7/8 vfio/pci: Support dynamic MSI-x
In reply to Reinette ChatreRecently introduced pci_msix_alloc_irq_at() and pci_msix_free_irq() enables an individual MSI-X index to be allocated and freed after MSI-X enabling.
Support dynamic MSI-X by keeping the association between allocated interrupt and vfio interrupt context. Allocate new context together with the new interrupt if no interrupt context exist for an MSI-X interrupt. Similarly, release an interrupt with its context.
---
Guidance is appreciated on expectations regarding maintaining existing error behavior. Earlier patch introduced the vfio_irq_ctx_range_allocated() helper to maintain existing error behavior. Now, this helper needs to be disabled for MSI-X. User space not wanting to dynamically allocate MSI-X interrupts, but providing invalid range when providing a new ACTION will now obtain new interrupts or new failures (potentially including freeing of existing interrupts) if the allocation of the new interrupts fail.
drivers/vfio/pci/vfio_pci_intrs.c | 101 ++++++++++++++++++++++++------ 1 file changed, 83 insertions(+), 18 deletions(-)
drivers/vfio/pci/vfio_pci_intrs.c +83 -18
@@ -55,6 +55,18 @@ struct vfio_pci_irq_ctx *vfio_irq_ctx_get(struct vfio_pci_core_device *vdev,return xa_load (&vdev->ctx, index); } +static void vfio_irq_ctx_free (struct vfio_pci_core_device *vdev, +unsigned long index) +{ +struct vfio_pci_irq_ctx *ctx; + + ctx =xa_load (&vdev->ctx, index); +if (ctx) { +xa_erase (&vdev->ctx, index); +kfree (ctx); + } +} +static void vfio_irq_ctx_free_all (struct vfio_pci_core_device *vdev) {struct vfio_pci_irq_ctx *ctx;
@@ -430,33 +442,63 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, {struct pci_dev *pdev = vdev->pdev;struct vfio_pci_irq_ctx *ctx; +struct msi_map msix_map = {};struct eventfd_ctx *trigger; +bool new_ctx;int irq, ret;u16 cmd; ctx =vfio_irq_ctx_get (vdev, vector); -if (!ctx) +/* Only MSI-X allows dynamic allocation. */ +if (!msix && !ctx)return -EINVAL ; + irq =pci_irq_vector (pdev, vector); +/* Context and interrupt are always allocated together. */ +WARN_ON ((ctx && irq == -EINVAL ) || (!ctx && irq != -EINVAL )); -if (ctx->trigger) { +if (ctx && ctx->trigger) {irq_bypass_unregister_producer (&ctx->producer); cmd =vfio_pci_memory_lock_and_enable (vdev);free_irq (irq, ctx->trigger); +if (msix) { + msix_map.index = vector; + msix_map.virq = irq; +pci_msix_free_irq (pdev, msix_map); + irq = -EINVAL ; + }vfio_pci_memory_unlock_and_restore (vdev, cmd);kfree (ctx->name);eventfd_ctx_put (ctx->trigger); ctx->trigger =NULL ; +if (msix) { +vfio_irq_ctx_free (vdev, vector); + ctx =NULL ; + } }if (fd <0 )return 0 ; +if (!ctx) { + ret =vfio_irq_ctx_alloc_single (vdev, vector); +if (ret) +return ret; + ctx =vfio_irq_ctx_get (vdev, vector); +if (!ctx) { + ret = -EINVAL ; + goto out_free_ctx; + } + new_ctx = true; + } + ctx->name =kasprintf (GFP_KERNEL_ACCOUNT ,"vfio-msi%s[%d](%s)" , msix ?"x" :"" , vector,pci_name (pdev)); -if (!ctx->name) -return -ENOMEM ; +if (!ctx->name) { + ret = -ENOMEM ; + goto out_free_ctx; + } trigger =eventfd_ctx_fdget (fd);if (IS_ERR (trigger)) {
@@ -464,25 +506,38 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, goto out_free_name; } -/* -* The MSIx vector table resides in device memory which may be cleared -* via backdoor resets. We don't allow direct access to the vector -* table so even if a userspace driver attempts to save/restore around -* such a reset it would be unsuccessful. To avoid this, restore the -* cached value of the message prior to enabling. -*/ cmd =vfio_pci_memory_lock_and_enable (vdev);if (msix) { -struct msi_msg msg; - -get_cached_msi_msg (irq, &msg); -pci_write_msi_msg (irq, &msg); +if (irq == -EINVAL ) { + msix_map =pci_msix_alloc_irq_at (pdev, vector,NULL ); +if (msix_map.index <0 ) { +vfio_pci_memory_unlock_and_restore (vdev, cmd); + ret = msix_map.index; + goto out_put_eventfd_ctx; + } + irq = msix_map.virq; + }else { +/* +* The MSIx vector table resides in device memory which +* may be cleared via backdoor resets. We don't allow +* direct access to the vector table so even if a +* userspace driver attempts to save/restore around +* such a reset it would be unsuccessful. To avoid +* this, restore the cached value of the message prior +* to enabling. +*/ +struct msi_msg msg; + +get_cached_msi_msg (irq, &msg); +pci_write_msi_msg (irq, &msg); + } } ret =request_irq (irq, vfio_msihandler,0 , ctx->name, trigger); -vfio_pci_memory_unlock_and_restore (vdev, cmd);if (ret) - goto out_put_eventfd_ctx; + goto out_free_irq_locked; + +vfio_pci_memory_unlock_and_restore (vdev, cmd); ctx->producer.token = trigger; ctx->producer.irq = irq;
@@ -498,11 +553,21 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev,return 0 ; +out_free_irq_locked: +if (msix && new_ctx) { + msix_map.index = vector; + msix_map.virq = irq; +pci_msix_free_irq (pdev, msix_map); + } +vfio_pci_memory_unlock_and_restore (vdev, cmd); out_put_eventfd_ctx:eventfd_ctx_put (trigger); out_free_name:kfree (ctx->name); ctx->name =NULL ; +out_free_ctx: +if (msix && new_ctx) +vfio_irq_ctx_free (vdev, vector);return ret; }
@@ -512,7 +577,7 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start,int i, ret =0 ;unsigned int j; -if (!vfio_irq_ctx_range_allocated (vdev, start, count)) +if (!msix && !vfio_irq_ctx_range_allocated (vdev, start, count))return -EINVAL ;for (i =0 , j = start; i < count && !ret; i++, j++) {
Signature · 1 line
2.34.1
Re: [RFC PATCH 7/8] vfio/pci: Support dynamic MSI-x
Quoted 33 lines · On Wed, 15 Mar 2023 13:59:27 -0700 Reinette Chatre <[email protected]>…
Recently introduced
pci_msix_alloc_irq_at()andpci_msix_free_irq()enables an individual MSI-X index to be allocated and freed after MSI-X enabling.Support dynamic MSI-X by keeping the association between allocated interrupt and vfio interrupt context. Allocate new context together with the new interrupt if no interrupt context exist for an MSI-X interrupt. Similarly, release an interrupt with its context.
Signed-off-by Reinette Chatre <[email protected]>---
Guidance is appreciated on expectations regarding maintaining existing error behavior. Earlier patch introduced the
vfio_irq_ctx_range_allocated()helper to maintain existing error behavior. Now, this helper needs to be disabled for MSI-X. User space not wanting to dynamically allocate MSI-X interrupts, but providing invalid range when providing a new ACTION will now obtain new interrupts or new failures (potentially including freeing of existing interrupts) if the allocation of the new interrupts fail.drivers/vfio/pci/vfio_pci_intrs.c | 101 ++++++++++++++++++++++++------ 1 file changed, 83 insertions(+), 18 deletions(-)drivers/vfio/pci/vfio_pci_intrs.c +1
-0@@ -55,6 +55,18 @@ struct vfio_pci_irq_ctx *vfio_irq_ctx_get(struct vfio_pci_core_device *vdev,return xa_load (&vdev->ctx, index); } +static void vfio_irq_ctx_free(struct vfio_pci_core_device *vdev,
drivers/vfio/pci/vfio_pci_intrs.c struct vfio_pci_irq_ctx *vfio_irq_ctx_get(struct vfio_pci_core_device *vdev, · line 68
Quoted 10 lines
drivers/vfio/pci/vfio_pci_intrs.c +10
-0+ unsigned long index) +{ +struct vfio_pci_irq_ctx *ctx; + + ctx =xa_load (&vdev->ctx, index); +if (ctx) { +xa_erase (&vdev->ctx, index); +kfree (ctx); + } +}
The only places calling this have a known valid ctx, so it seems redundant that we xa_load it again. Should ctx be a function arg to reduce this to simply xa_erase() + kfree()?
Quoted 44 lines
drivers/vfio/pci/vfio_pci_intrs.c +19
-2+static void vfio_irq_ctx_free_all (struct vfio_pci_core_device *vdev) {struct vfio_pci_irq_ctx *ctx;@@ -430,33 +442,63 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, {struct pci_dev *pdev = vdev->pdev;struct vfio_pci_irq_ctx *ctx; +struct msi_map msix_map = {};struct eventfd_ctx *trigger; +bool new_ctx;int irq, ret;u16 cmd; ctx =vfio_irq_ctx_get (vdev, vector); -if (!ctx) +/* Only MSI-X allows dynamic allocation. */ +if (!msix && !ctx)return -EINVAL ; + irq =pci_irq_vector (pdev, vector); +/* Context and interrupt are always allocated together. */ +WARN_ON ((ctx && irq == -EINVAL ) || (!ctx && irq != -EINVAL )); -if (ctx->trigger) { +if (ctx && ctx->trigger) {irq_bypass_unregister_producer (&ctx->producer); cmd =vfio_pci_memory_lock_and_enable (vdev);free_irq (irq, ctx->trigger); +if (msix) { + msix_map.index = vector; + msix_map.virq = irq; +pci_msix_free_irq (pdev, msix_map); + irq = -EINVAL ; + }vfio_pci_memory_unlock_and_restore (vdev, cmd);kfree (ctx->name);eventfd_ctx_put (ctx->trigger); ctx->trigger =NULL ; +if (msix) { +vfio_irq_ctx_free (vdev, vector); + ctx =NULL ; + }
drivers/vfio/pci/vfio_pci_intrs.c static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, · line 488
Quoted 10 lines
drivers/vfio/pci/vfio_pci_intrs.c +5
-0}if (fd <0 )return 0 ; +if (!ctx) { + ret =vfio_irq_ctx_alloc_single (vdev, vector); +if (ret) +return ret; + ctx =vfio_irq_ctx_get (vdev, vector);
This suggests vfio_irq_ctx_alloc_single() should return ctx.
Quoted 27 lines
drivers/vfio/pci/vfio_pci_intrs.c +11
-7+if (!ctx) { + ret = -EINVAL ; + goto out_free_ctx; + } + new_ctx = true; + } + ctx->name =kasprintf (GFP_KERNEL_ACCOUNT ,"vfio-msi%s[%d](%s)" , msix ?"x" :"" , vector,pci_name (pdev)); -if (!ctx->name) -return -ENOMEM ; +if (!ctx->name) { + ret = -ENOMEM ; + goto out_free_ctx; + } trigger =eventfd_ctx_fdget (fd);if (IS_ERR (trigger)) {@@ -464,25 +506,38 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, goto out_free_name; } -/* -* The MSIx vector table resides in device memory which may be cleared -* via backdoor resets. We don't allow direct access to the vector -* table so even if a userspace driver attempts to save/restore around -* such a reset it would be unsuccessful. To avoid this, restore the
drivers/vfio/pci/vfio_pci_intrs.c static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, · line 512
Quoted 10 lines
drivers/vfio/pci/vfio_pci_intrs.c +2
-6-* cached value of the message prior to enabling. -*/ cmd =vfio_pci_memory_lock_and_enable (vdev);if (msix) { -struct msi_msg msg; - -get_cached_msi_msg (irq, &msg); -pci_write_msi_msg (irq, &msg); +if (irq == -EINVAL ) { + msix_map =pci_msix_alloc_irq_at (pdev, vector,NULL );
It looks to me like we need to support MSI-X with both NORESIZE behavior and dynamic allocation based on pci_msix_can_alloc_dyn(). It's not entirely clear to me where this is and isn't supported, but the existence of the test helper suggests we can't assume support.
Quoted 11 lines
drivers/vfio/pci/vfio_pci_intrs.c +11
-0+if (msix_map.index <0 ) { +vfio_pci_memory_unlock_and_restore (vdev, cmd); + ret = msix_map.index; + goto out_put_eventfd_ctx; + } + irq = msix_map.virq; + }else { +/* +* The MSIx vector table resides in device memory which +* may be cleared via backdoor resets. We don't allow +* direct access to the vector table so even if a
drivers/vfio/pci/vfio_pci_intrs.c static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, · line 533
Quoted 10 lines
drivers/vfio/pci/vfio_pci_intrs.c +10
-0+* userspace driver attempts to save/restore around +* such a reset it would be unsuccessful. To avoid +* this, restore the cached value of the message prior +* to enabling. +*/ +struct msi_msg msg; + +get_cached_msi_msg (irq, &msg); +pci_write_msi_msg (irq, &msg); + }
I don't follow when this latter branch is ever taken in the new flow. It's stated earlier that ctx and irq are coupled, and I believe so is trigger. So if we had a previous ctx and irq (and trigger), we removed it and irq is now always -EINVAL here. Thanks,
Alex
Quoted 43 lines
drivers/vfio/pci/vfio_pci_intrs.c +14
-3} ret =request_irq (irq, vfio_msihandler,0 , ctx->name, trigger); -vfio_pci_memory_unlock_and_restore (vdev, cmd);if (ret) - goto out_put_eventfd_ctx; + goto out_free_irq_locked; + +vfio_pci_memory_unlock_and_restore (vdev, cmd); ctx->producer.token = trigger; ctx->producer.irq = irq;@@ -498,11 +553,21 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev,return 0 ; +out_free_irq_locked: +if (msix && new_ctx) { + msix_map.index = vector; + msix_map.virq = irq; +pci_msix_free_irq (pdev, msix_map); + } +vfio_pci_memory_unlock_and_restore (vdev, cmd); out_put_eventfd_ctx:eventfd_ctx_put (trigger); out_free_name:kfree (ctx->name); ctx->name =NULL ; +out_free_ctx: +if (msix && new_ctx) +vfio_irq_ctx_free (vdev, vector);return ret; }@@ -512,7 +577,7 @@ static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start,int i, ret =0 ;unsigned int j; -if (!vfio_irq_ctx_range_allocated (vdev, start, count)) +if (!msix && !vfio_irq_ctx_range_allocated (vdev, start, count))return -EINVAL ;for (i =0 , j = start; i < count && !ret; i++, j++) {
Re: [RFC PATCH 7/8] vfio/pci: Support dynamic MSI-x
Hi Alex,
Quoted 3 lines · Alex Williamson
On Wed, 15 Mar 2023 13:59:27 -0700 Reinette Chatre <[email protected]> wrote:
...
Quoted 7 lines
Quoted 7 lines
drivers/vfio/pci/vfio_pci_intrs.c +7
-0+static void vfio_irq_ctx_free (struct vfio_pci_core_device *vdev, +unsigned long index) +{ +struct vfio_pci_irq_ctx *ctx; + + ctx =xa_load (&vdev->ctx, index); +if (ctx) {
Quoted 8 lines
drivers/vfio/pci/vfio_pci_intrs.c struct vfio_pci_irq_ctx *vfio_irq_ctx_get(struct vfio_pci_core_device *vdev, · line 68
Quoted 4 lines
drivers/vfio/pci/vfio_pci_intrs.c +4
-0+xa_erase (&vdev->ctx, index); +kfree (ctx); + } +}The only places calling this have a known valid ctx, so it seems redundant that we
xa_loadit again. Should ctx be a function arg to reduce this to simplyxa_erase()+kfree()?
Good point. Will do.
...
Quoted 8 lines
drivers/vfio/pci/vfio_pci_intrs.c static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, · line 488
Quoted 5 lines
drivers/vfio/pci/vfio_pci_intrs.c +5
-0+if (!ctx) { + ret =vfio_irq_ctx_alloc_single (vdev, vector); +if (ret) +return ret; + ctx =vfio_irq_ctx_get (vdev, vector);This suggests
vfio_irq_ctx_alloc_single()should return ctx.
Thank you. Yes, will do.
Quoted 16 lines
Quoted 16 lines
drivers/vfio/pci/vfio_pci_intrs.c +0
-10@@ -464,25 +506,38 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, goto out_free_name; } -/* -* The MSIx vector table resides in device memory which may be cleared -* via backdoor resets. We don't allow direct access to the vector -* table so even if a userspace driver attempts to save/restore around -* such a reset it would be unsuccessful. To avoid this, restore the -* cached value of the message prior to enabling. -*/ cmd =vfio_pci_memory_lock_and_enable (vdev);if (msix) { -struct msi_msg msg; - -get_cached_msi_msg (irq, &msg);
Quoted 8 lines
drivers/vfio/pci/vfio_pci_intrs.c static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, · line 512
Quoted 3 lines
drivers/vfio/pci/vfio_pci_intrs.c +2
-1-pci_write_msi_msg (irq, &msg); +if (irq == -EINVAL ) { + msix_map =pci_msix_alloc_irq_at (pdev, vector,NULL );It looks to me like we need to support MSI-X with both NORESIZE behavior and dynamic allocation based on
pci_msix_can_alloc_dyn(). It's not entirely clear to me where this is and isn't supported, but the existence of the test helper suggests we can't assume support.
As I mentioned in my other response ([1]) I cannot see how pci_msix_can_alloc_dyn() can return false. Even so, yes, I can rework this series to support both the original and dynamic MSI-x allocation mechanisms.
Quoted 18 lines
Quoted 18 lines
drivers/vfio/pci/vfio_pci_intrs.c +18
-0+if (msix_map.index <0 ) { +vfio_pci_memory_unlock_and_restore (vdev, cmd); + ret = msix_map.index; + goto out_put_eventfd_ctx; + } + irq = msix_map.virq; + }else { +/* +* The MSIx vector table resides in device memory which +* may be cleared via backdoor resets. We don't allow +* direct access to the vector table so even if a +* userspace driver attempts to save/restore around +* such a reset it would be unsuccessful. To avoid +* this, restore the cached value of the message prior +* to enabling. +*/ +struct msi_msg msg; +
Quoted 8 lines
drivers/vfio/pci/vfio_pci_intrs.c static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, · line 533
Quoted 3 lines
drivers/vfio/pci/vfio_pci_intrs.c +3
-0+get_cached_msi_msg (irq, &msg); +pci_write_msi_msg (irq, &msg); + }I don't follow when this latter branch is ever taken in the new flow. It's stated earlier that ctx and irq are coupled, and I believe so is trigger. So if we had a previous ctx and irq (and trigger), we removed it and irq is now always
-EINVALhere. Thanks,
From what I understand MSI-X can be enabled without providing any triggers. That will result in the ctx and irq existing, but not trigger. When a trigger is assigned later, it will run the latter branch.
Thank you very much
Reinette
Re: [RFC PATCH 7/8] vfio/pci: Support dynamic MSI-x
In reply to Reinette ChatreHi Reinette,
url: https://github.com/intel-lab-lkp/linux/commits/Reinette-Chatre/vfio-pci-Consolidate-irq-cleanup-on-MSI-MSI-X-disable/20230316-050150 base: eeac8ede17557680855031c6f305ece2378af326 patch link: https://lore.kernel.org/r/591ce11f4a33e022fc9242324ebdc077202bedaf.1678911529.git.reinette.chatre%40intel.com patch subject: [RFC PATCH 7/8] vfio/pci: Support dynamic MSI-x config: ia64-randconfig-m031-20230312 (https://download.01.org/0day-ci/archive/20230316/[email protected]/config) compiler: ia64-linux-gcc (GCC) 12.1.0
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]> | Reported-by: Dan Carpenter <[email protected]> | Link: https://lore.kernel.org/r/[email protected]/
New smatch warnings:drivers/vfio/pci/vfio_pci_intrs.c:557 vfio_msi_set_vector_signal() error: uninitialized symbol 'new_ctx'.
Old smatch warnings:drivers/vfio/pci/vfio_pci_intrs.c:569 vfio_msi_set_vector_signal() error: uninitialized symbol 'new_ctx'.
vim +/new_ctx +557 drivers/vfio/pci/vfio_pci_intrs.c
536475109c8284 Max Gurtovoy 2021-08-26 440 static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev,
45d99835bd7f50 Reinette Chatre 2023-03-15 441 unsigned int vector, int fd, bool msix)
89e1f7d4c66d85 Alex Williamson 2012-07-31 442 {
89e1f7d4c66d85 Alex Williamson 2012-07-31 443 struct pci_dev *pdev = vdev->pdev;
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 444 struct vfio_pci_irq_ctx *ctx;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 445 struct msi_map msix_map = {};
89e1f7d4c66d85 Alex Williamson 2012-07-31 446 struct eventfd_ctx *trigger;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 447 bool new_ctx;Was never intiliazed to false.
b95d9305e8cb8d Alex Williamson 2016-02-22 448 int irq, ret;
abafbc551fdded Alex Williamson 2020-04-22 449 u16 cmd;
89e1f7d4c66d85 Alex Williamson 2012-07-31 450
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 451 ctx = vfio_irq_ctx_get(vdev, vector);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 452 /* Only MSI-X allows dynamic allocation. */
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 453 if (!msix && !ctx)
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 454 return -EINVAL;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 455
61771468e0a567 Christoph Hellwig 2016-09-11 456 irq = pci_irq_vector(pdev, vector);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 457 /* Context and interrupt are always allocated together. */
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 458 WARN_ON((ctx && irq == -EINVAL) || (!ctx && irq != -EINVAL));
b95d9305e8cb8d Alex Williamson 2016-02-22 459
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 460 if (ctx && ctx->trigger) {
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 461 irq_bypass_unregister_producer(&ctx->producer);
abafbc551fdded Alex Williamson 2020-04-22 462
abafbc551fdded Alex Williamson 2020-04-22 463 cmd = vfio_pci_memory_lock_and_enable(vdev);
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 464 free_irq(irq, ctx->trigger);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 465 if (msix) {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 466 msix_map.index = vector;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 467 msix_map.virq = irq;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 468 pci_msix_free_irq(pdev, msix_map);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 469 irq = -EINVAL;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 470 }
abafbc551fdded Alex Williamson 2020-04-22 471 vfio_pci_memory_unlock_and_restore(vdev, cmd);
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 472 kfree(ctx->name);
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 473 eventfd_ctx_put(ctx->trigger);
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 474 ctx->trigger = NULL;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 475 if (msix) {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 476 vfio_irq_ctx_free(vdev, vector);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 477 ctx = NULL;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 478 }
89e1f7d4c66d85 Alex Williamson 2012-07-31 479 }
89e1f7d4c66d85 Alex Williamson 2012-07-31 480
89e1f7d4c66d85 Alex Williamson 2012-07-31 481 if (fd < 0)
89e1f7d4c66d85 Alex Williamson 2012-07-31 482 return 0;
89e1f7d4c66d85 Alex Williamson 2012-07-31 483
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 484 if (!ctx) {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 485 ret = vfio_irq_ctx_alloc_single(vdev, vector);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 486 if (ret)
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 487 return ret;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 488 ctx = vfio_irq_ctx_get(vdev, vector);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 489 if (!ctx) {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 490 ret = -EINVAL;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 491 goto out_free_ctx;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 492 }
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 493 new_ctx = true;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 494 }
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 495
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 496 ctx->name = kasprintf(GFP_KERNEL_ACCOUNT, "vfio-msi%s[%d](%s)",
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 497 msix ? "x" : "", vector, pci_name(pdev));
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 498 if (!ctx->name) {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 499 ret = -ENOMEM;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 500 goto out_free_ctx;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 501 }
89e1f7d4c66d85 Alex Williamson 2012-07-31 502
89e1f7d4c66d85 Alex Williamson 2012-07-31 503 trigger = eventfd_ctx_fdget(fd);
89e1f7d4c66d85 Alex Williamson 2012-07-31 504 if (IS_ERR(trigger)) {
224474be3986e1 Reinette Chatre 2023-03-15 505 ret = PTR_ERR(trigger);
224474be3986e1 Reinette Chatre 2023-03-15 506 goto out_free_name;
89e1f7d4c66d85 Alex Williamson 2012-07-31 507 }
89e1f7d4c66d85 Alex Williamson 2012-07-31 508
abafbc551fdded Alex Williamson 2020-04-22 509 cmd = vfio_pci_memory_lock_and_enable(vdev);
b8f02af096b1fc Gavin Shan 2014-09-29 510 if (msix) {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 511 if (irq == -EINVAL) {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 512 msix_map = pci_msix_alloc_irq_at(pdev, vector, NULL);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 513 if (msix_map.index < 0) {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 514 vfio_pci_memory_unlock_and_restore(vdev, cmd);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 515 ret = msix_map.index;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 516 goto out_put_eventfd_ctx;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 517 }
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 518 irq = msix_map.virq;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 519 } else {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 520 /*
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 521 * The MSIx vector table resides in device memory which
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 522 * may be cleared via backdoor resets. We don't allow
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 523 * direct access to the vector table so even if a
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 524 * userspace driver attempts to save/restore around
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 525 * such a reset it would be unsuccessful. To avoid
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 526 * this, restore the cached value of the message prior
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 527 * to enabling.
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 528 */
b8f02af096b1fc Gavin Shan 2014-09-29 529 struct msi_msg msg;
b8f02af096b1fc Gavin Shan 2014-09-29 530
b8f02af096b1fc Gavin Shan 2014-09-29 531 get_cached_msi_msg(irq, &msg);
83a18912b0e8d2 Jiang Liu 2014-11-09 532 pci_write_msi_msg(irq, &msg);
b8f02af096b1fc Gavin Shan 2014-09-29 533 }
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 534 }
b8f02af096b1fc Gavin Shan 2014-09-29 535
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 536 ret = request_irq(irq, vfio_msihandler, 0, ctx->name, trigger);
224474be3986e1 Reinette Chatre 2023-03-15 537 if (ret)
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 538 goto out_free_irq_locked;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 539
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 540 vfio_pci_memory_unlock_and_restore(vdev, cmd);
89e1f7d4c66d85 Alex Williamson 2012-07-31 541
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 542 ctx->producer.token = trigger;
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 543 ctx->producer.irq = irq;
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 544 ret = irq_bypass_register_producer(&ctx->producer);
852b1beecb6ff9 Alex Williamson 2020-10-19 545 if (unlikely(ret)) {
6d7425f109d262 Feng Wu 2015-09-18 546 dev_info(&pdev->dev,
6d7425f109d262 Feng Wu 2015-09-18 547 "irq bypass producer (token %p) registration fails: %d\n",
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 548 ctx->producer.token, ret);
6d7425f109d262 Feng Wu 2015-09-18 549
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 550 ctx->producer.token = NULL;
852b1beecb6ff9 Alex Williamson 2020-10-19 551 }
d6ddf3a1f66d02 Reinette Chatre 2023-03-15 552 ctx->trigger = trigger;
89e1f7d4c66d85 Alex Williamson 2012-07-31 553
89e1f7d4c66d85 Alex Williamson 2012-07-31 554 return 0;
224474be3986e1 Reinette Chatre 2023-03-15 555
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 556 out_free_irq_locked:
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 @557 if (msix && new_ctx) {
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 558 msix_map.index = vector;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 559 msix_map.virq = irq;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 560 pci_msix_free_irq(pdev, msix_map);
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 561 }
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 562 vfio_pci_memory_unlock_and_restore(vdev, cmd);
224474be3986e1 Reinette Chatre 2023-03-15 563 out_put_eventfd_ctx:
224474be3986e1 Reinette Chatre 2023-03-15 564 eventfd_ctx_put(trigger);
224474be3986e1 Reinette Chatre 2023-03-15 565 out_free_name:
224474be3986e1 Reinette Chatre 2023-03-15 566 kfree(ctx->name);
224474be3986e1 Reinette Chatre 2023-03-15 567 ctx->name = NULL;
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 568 out_free_ctx:
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 569 if (msix && new_ctx)
db5c3d2cbe1a75 Reinette Chatre 2023-03-15 570 vfio_irq_ctx_free(vdev, vector);
224474be3986e1 Reinette Chatre 2023-03-15 571 return ret;
89e1f7d4c66d85 Alex Williamson 2012-07-31 572 }Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
Re: [RFC PATCH 7/8] vfio/pci: Support dynamic MSI-x
Quoted 14 lines · Dan Carpenter
Hi Reinette,
url: https://github.com/intel-lab-lkp/linux/commits/Reinette-Chatre/vfio-pci-Consolidate-irq-cleanup-on-MSI-MSI-X-disable/20230316-050150 base: eeac8ede17557680855031c6f305ece2378af326 patch link: https://lore.kernel.org/r/591ce11f4a33e022fc9242324ebdc077202bedaf.1678911529.git.reinette.chatre%40intel.com patch subject: [RFC PATCH 7/8] vfio/pci: Support dynamic MSI-x config: ia64-randconfig-m031-20230312 (https://download.01.org/0day-ci/archive/20230316/[email protected]/config) compiler: ia64-linux-gcc (GCC) 12.1.0If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]> | Reported-by: Dan Carpenter <[email protected]> | Link: https://lore.kernel.org/r/[email protected]/
Quoted 5 lines
New smatch warnings:
drivers/vfio/pci/vfio_pci_intrs.c:557vfio_msi_set_vector_signal()error: uninitialized symbol 'new_ctx'.Old smatch warnings:
drivers/vfio/pci/vfio_pci_intrs.c:569vfio_msi_set_vector_signal()error: uninitialized symbol 'new_ctx'.
Thank you very much Dan.
Reinette
RFC PATCH 8/8 vfio/pci: Clear VFIO_IRQ_INFO_NORESIZE for MSI-X
In reply to Reinette ChatreDynamic MSI-X is supported. Clear VFIO_IRQ_INFO_NORESIZE to provide guidance to user space.
--- drivers/vfio/pci/vfio_pci_core.c | 2 +- include/uapi/linux/vfio.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-)
drivers/vfio/pci/vfio_pci_core.c +1 -1
@@ -1111,7 +1111,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev,if (info.index ==VFIO_PCI_INTX_IRQ_INDEX ) info.flags |= (VFIO_IRQ_INFO_MASKABLE |VFIO_IRQ_INFO_AUTOMASKED ); -else +else if (info.index !=VFIO_PCI_MSIX_IRQ_INDEX ) info.flags |=VFIO_IRQ_INFO_NORESIZE ;return copy_to_user (arg, &info, minsz) ? -EFAULT :0 ;
include/uapi/linux/vfio.h +3 -0
@@ -511,6 +511,9 @@ struct vfio_region_info_cap_nvlink2_lnkspd {* then add and unmask vectors, it's up to userspace to make the decision * whether to allocate the maximum supported number of vectors or tear * down setup and incrementally increase the vectors as each is enabled. +* Absence of the NORESIZE flag indicates that vectors can be enabled +* and disabled dynamically without impacting other vectors within the +* index. */ struct vfio_irq_info {__u32 argsz;
Signature · 1 line
2.34.1
Re: [RFC PATCH 8/8] vfio/pci: Clear VFIO_IRQ_INFO_NORESIZE for MSI-X
Quoted 14 lines · On Wed, 15 Mar 2023 13:59:28 -0700 Reinette Chatre <[email protected]>…
Dynamic MSI-X is supported. Clear
VFIO_IRQ_INFO_NORESIZEto provide guidance to user space.Signed-off-by Reinette Chatre <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 2 +- include/uapi/linux/vfio.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-)drivers/vfio/pci/vfio_pci_core.c +0
-0
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev, · line 1116
Quoted 7 lines
drivers/vfio/pci/vfio_pci_core.c +1
-1@@ -1111,7 +1111,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev,if (info.index ==VFIO_PCI_INTX_IRQ_INDEX ) info.flags |= (VFIO_IRQ_INFO_MASKABLE |VFIO_IRQ_INFO_AUTOMASKED ); -else +else if (info.index !=VFIO_PCI_MSIX_IRQ_INDEX ) info.flags |=VFIO_IRQ_INFO_NORESIZE ;
I think we need to check pci_msix_can_alloc_dyn(), right? Thanks,
Alex
Quoted 15 lines
drivers/vfio/pci/vfio_pci_core.c +0
-0return copy_to_user (arg, &info, minsz) ? -EFAULT :0 ;include/uapi/linux/vfio.h +3
-0@@ -511,6 +511,9 @@ struct vfio_region_info_cap_nvlink2_lnkspd {* then add and unmask vectors, it's up to userspace to make the decision * whether to allocate the maximum supported number of vectors or tear * down setup and incrementally increase the vectors as each is enabled. +* Absence of the NORESIZE flag indicates that vectors can be enabled +* and disabled dynamically without impacting other vectors within the +* index. */ struct vfio_irq_info {__u32 argsz;
Re: [RFC PATCH 8/8] vfio/pci: Clear VFIO_IRQ_INFO_NORESIZE for MSI-X
Hi Alex,
Quoted 12 lines · Alex Williamson
Quoted 9 lines · On Wed, 15 Mar 2023 13:59:28 -0700 Reinette Chatre <[email protected]>…
Dynamic MSI-X is supported. Clear
VFIO_IRQ_INFO_NORESIZEto provide guidance to user space.Signed-off-by Reinette Chatre <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 2 +- include/uapi/linux/vfio.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-)
Quoted 14 lines
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev, · line 1116
Quoted 12 lines
drivers/vfio/pci/vfio_pci_core.c +1
-1@@ -1111,7 +1111,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev,if (info.index ==VFIO_PCI_INTX_IRQ_INDEX ) info.flags |= (VFIO_IRQ_INFO_MASKABLE |VFIO_IRQ_INFO_AUTOMASKED ); -else +else if (info.index !=VFIO_PCI_MSIX_IRQ_INDEX ) info.flags |=VFIO_IRQ_INFO_NORESIZE ;I think we need to check
pci_msix_can_alloc_dyn(), right? Thanks,
Can pci_msix_can_alloc_dyn() ever return false?
I cannot see how pci_msix_can_alloc_dyn() can return false when considering the VFIO PCI MSI-X flow:
vfio_msi_enable(..., ..., msix == true)
pci_alloc_irq_vectors(..., ..., ..., flag == PCI_IRQ_MSIX)
pci_alloc_irq_vectors_affinity()
__pci_enable_msix_range()
pci_setup_msix_device_domain()
pci_create_device_domain(..., &pci_msix_template, ...)The template used above, pci_msix_template, has MSI_FLAG_PCI_MSIX_ALLOC_DYN hardcoded. This is the flag that pci_msix_can_alloc_dyn() tests for.
If indeed pci_msix_can_alloc_dyn() can return false in the VFIO PCI MSI-X usage then this series needs to be reworked to continue supporting existing allocation mechanism as well as dynamic MSI-X allocation. Which allocation mechanism to use would be depend on pci_msix_can_alloc_dyn().
Alternatively, if you agree that VFIO PCI MSI-X can expect pci_msix_can_alloc_dyn() to always return true then I should perhaps add a test in vfio_msi_enable() that confirms this is the case. This would cause vfio_msi_enable() to fail if dynamic MSI-X is not possible, perhaps even complain loudly with a WARN.
Reinette
Re: [RFC PATCH 8/8] vfio/pci: Clear VFIO_IRQ_INFO_NORESIZE for MSI-X
Quoted 50 lines · On Fri, 17 Mar 2023 15:21:09 -0700 Reinette Chatre <[email protected]>…
Hi Alex,
Quoted 12 lines · Alex Williamson
Quoted 9 lines · On Wed, 15 Mar 2023 13:59:28 -0700 Reinette Chatre <[email protected]>…
Dynamic MSI-X is supported. Clear
VFIO_IRQ_INFO_NORESIZEto provide guidance to user space.Signed-off-by Reinette Chatre <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 2 +- include/uapi/linux/vfio.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-)Quoted 14 lines
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev, · line 1116
Quoted 12 lines
drivers/vfio/pci/vfio_pci_core.c +1
-1@@ -1111,7 +1111,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev,if (info.index ==VFIO_PCI_INTX_IRQ_INDEX ) info.flags |= (VFIO_IRQ_INFO_MASKABLE |VFIO_IRQ_INFO_AUTOMASKED ); -else +else if (info.index !=VFIO_PCI_MSIX_IRQ_INDEX ) info.flags |=VFIO_IRQ_INFO_NORESIZE ;I think we need to check
pci_msix_can_alloc_dyn(), right? Thanks,Can
pci_msix_can_alloc_dyn()ever return false?I cannot see how
pci_msix_can_alloc_dyn()can return false when considering the VFIO PCI MSI-X flow:
vfio_msi_enable(..., ..., msix == true)
pci_alloc_irq_vectors(..., ..., ..., flag == PCI_IRQ_MSIX)pci_alloc_irq_vectors_affinity() __pci_enable_msix_range() pci_setup_msix_device_domain() pci_create_device_domain(..., &pci_msix_template, ...)The template used above,
pci_msix_template, hasMSI_FLAG_PCI_MSIX_ALLOC_DYNhardcoded. This is the flag thatpci_msix_can_alloc_dyn()tests for.If indeed
pci_msix_can_alloc_dyn()can return false in the VFIO PCI MSI-X usage then this series needs to be reworked to continue supporting existing allocation mechanism as well as dynamic MSI-X allocation. Which allocation mechanism to use would be depend onpci_msix_can_alloc_dyn().
Quoted 5 lines
Alternatively, if you agree that VFIO PCI MSI-X can expect
pci_msix_can_alloc_dyn()to always return true then I should perhaps add a test invfio_msi_enable()that confirms this is the case. This would causevfio_msi_enable()to fail if dynamic MSI-X is not possible, perhaps even complain loudly with a WARN.
pci_setup_msix_device_domain() says it returns true if:
* True when:
* - The device does not have a MSI parent irq domain associated,
* which keeps the legacy architecture specific and the global
* PCI/MSI domain models working
* - The MSI-X domain exists already
* - The MSI-X domain was successfully allocated
That first one seems concerning, dynamic allocation only works on irq domain configurations. What does that exclude and do we care about any of them for vfio-pci? Minimally this suggests a dependency on IRQ_DOMAIN, which we don't currently have, but I'm not sure if supporting irq domains is the same as having irq domains. Thanks,
Alex
Re: [RFC PATCH 8/8] vfio/pci: Clear VFIO_IRQ_INFO_NORESIZE for MSI-X
Hi Alex,
Quoted 68 lines · Alex Williamson
Quoted 50 lines · On Fri, 17 Mar 2023 15:21:09 -0700 Reinette Chatre <[email protected]>…
Hi Alex,
Quoted 12 lines · Alex Williamson
Quoted 9 lines · On Wed, 15 Mar 2023 13:59:28 -0700 Reinette Chatre <[email protected]>…
Dynamic MSI-X is supported. Clear
VFIO_IRQ_INFO_NORESIZEto provide guidance to user space.Signed-off-by Reinette Chatre <[email protected]>--- drivers/vfio/pci/vfio_pci_core.c | 2 +- include/uapi/linux/vfio.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-)Quoted 14 lines
drivers/vfio/pci/vfio_pci_core.c static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev, · line 1116
Quoted 12 lines
drivers/vfio/pci/vfio_pci_core.c +1
-1@@ -1111,7 +1111,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pci_core_device *vdev,if (info.index ==VFIO_PCI_INTX_IRQ_INDEX ) info.flags |= (VFIO_IRQ_INFO_MASKABLE |VFIO_IRQ_INFO_AUTOMASKED ); -else +else if (info.index !=VFIO_PCI_MSIX_IRQ_INDEX ) info.flags |=VFIO_IRQ_INFO_NORESIZE ;I think we need to check
pci_msix_can_alloc_dyn(), right? Thanks,Can
pci_msix_can_alloc_dyn()ever return false?I cannot see how
pci_msix_can_alloc_dyn()can return false when considering the VFIO PCI MSI-X flow:
vfio_msi_enable(..., ..., msix == true)
pci_alloc_irq_vectors(..., ..., ..., flag == PCI_IRQ_MSIX)pci_alloc_irq_vectors_affinity() __pci_enable_msix_range() pci_setup_msix_device_domain() pci_create_device_domain(..., &pci_msix_template, ...)The template used above,
pci_msix_template, hasMSI_FLAG_PCI_MSIX_ALLOC_DYNhardcoded. This is the flag thatpci_msix_can_alloc_dyn()tests for.If indeed
pci_msix_can_alloc_dyn()can return false in the VFIO PCI MSI-X usage then this series needs to be reworked to continue supporting existing allocation mechanism as well as dynamic MSI-X allocation. Which allocation mechanism to use would be depend onpci_msix_can_alloc_dyn().Quoted 5 lines
Alternatively, if you agree that VFIO PCI MSI-X can expect
pci_msix_can_alloc_dyn()to always return true then I should perhaps add a test invfio_msi_enable()that confirms this is the case. This would causevfio_msi_enable()to fail if dynamic MSI-X is not possible, perhaps even complain loudly with a WARN.
pci_setup_msix_device_domain()says it returns true if:* True when:
* - The device does not have a MSI parent irq domain associated,
* which keeps the legacy architecture specific and the global
* PCI/MSI domain models working
* - The MSI-X domain exists already
* - The MSI-X domain was successfully allocated
Quoted 5 lines
That first one seems concerning, dynamic allocation only works on irq domain configurations. What does that exclude and do we care about any of them for vfio-pci? Minimally this suggests a dependency on
IRQ_DOMAIN, which we don't currently have, but I'm not sure if supporting irq domains is the same as having irq domains. Thanks,
Just to confirm, as I mentioned in [1] I do plan to rework this solution to support both allocation mechanisms, using pci_msix_can_alloc_dyn() to pick which one to use. Thank you very much for pointing out this gap to me.
Reinette
Re: [RFC PATCH 8/8] vfio/pci: Clear VFIO_IRQ_INFO_NORESIZE for MSI-X
In reply to Alex WilliamsonQuoted 9 lines · Alex Williamson
pci_setup_msix_device_domain()says it returns true if:* True when:
* - The device does not have a MSI parent irq domain associated,
* which keeps the legacy architecture specific and the global
* PCI/MSI domain models working
* - The MSI-X domain exists already
* - The MSI-X domain was successfully allocated
Quoted 3 lines
That first one seems concerning, dynamic allocation only works on irq domain configurations. What does that exclude and do we care about any of them for vfio-pci?
Several archs and other weird things override the MSI-X programming. Eg by turning it into a hypervisor call or something. These were not converted to dynamic mode.
So at the VFIO level you can get end up with MSI support but done through legacy paths that don't support dynamic allocation. Eg on POWER, xen, etc.
Jason
Quoted 47 lines · On Wed, 15 Mar 2023 13:59:20 -0700 Reinette Chatre <[email protected]>…
== Cover letter ==
Qemu allocates interrupts incrementally at the time the guest unmasks an interrupt, for example each time a Linux guest runs
request_irq().Dynamic allocation of MSI-X interrupts was not possible until v6.2 [1]. This prompted Qemu to, when allocating a new interrupt, first release all previously allocated interrupts (including disable of MSI-X) followed by re-allocation of all interrupts that includes the new interrupt. Please see [2] for a detailed discussion about this issue.
Releasing and re-allocating interrupts may be acceptable if all interrupts are unmasked during device initialization. If unmasking of interrupts occur during runtime this may result in lost interrupts. For example, consider an accelerator device with multiple work queues, each work queue having a dedicated interrupt. A work queue can be enabled at any time with its associated interrupt unmasked while other work queues are already active. Having all interrupts released and MSI-X disabled to enable the new work queue will impact active work queues.
This series builds on the recent interrupt sub-system core changes that added support for dynamic MSI-X allocation after initial MSI-X enabling.
Add support for dynamic MSI-X allocation to vfio-pci. A flag indicating lack of support for dynamic allocation already exist:
VFIO_IRQ_INFO_NORESIZEand has always been set for MSI and MSI-X. With support for dynamic MSI-X the flag is cleared for MSI-X, enabling Qemu to modify its behavior.== Why is this an RFC ? ==
vfio support for dynamic MSI-X needs to work with existing user space as well as upcoming user space that takes advantage of this feature. I would appreciate guidance on the expectations and requirements surrounding error handling when considering existing user space.
For example, consider the following scenario:
Start: Consider a passthrough device that supports 10 MSI-X interrupts
(0 to 9) and existing Qemu allocated interrupts 0 to 4.Scenario: Qemu (hypothetically) attempts to associate a new action to
interrupts 0 to 7. Early checking of this range invfio_set_irqs_validate_and_prepare()will pass since it is a valid range for the device. What happens after the early checking is considered next:
Quoted 4 lines
Current behavior (before this series): Since the provided range, 0 - 7,
exceeds the allocated range, no action will be taken on existing allocated interrupts 0 - 4 and the Qemu request will fail without making any state changes.
I must be missing something, it was described correctly earlier that QEMU will disable MSI-X and re-enable with a new vector count. Not only does QEMU not really have a way to fail this change, pretty much nothing would work if we did.
What happens in this case is that the QEMU vfio-pci driver gets a vector_use callback for one of these new vectors {5,6,7}, vfio_msix_vector_do_use() checks that's greater than we have enabled, disables MSI-X, and re-enables it for the new vector count. Worst case we'll do that 3 times if the vectors are presented in ascending order.
Quoted 6 lines
New behavior (with this series): Since vfio supports dynamic MSI-X new
interrupts will be allocated for vectors 5, 6, and 7. Please note that this changes the behavior encountered by unmodified Qemu: new interrupts are allocated instead of returning an error. Even more, since the range provided by Qemu spans 0 - 7, a failure during allocation of 5, 6, or 7 will result in release of entire range.
As above, QEMU doesn't currently return an error here, nor is there actually any means to return an error. MSI-X is not paravirtualized and the PCI spec definition of MSI-X does not define that a driver needs to check whether the device accepted unmasking a vector. All we can do in QEMU if the host fails to configure the device as directed is print an error message as a breadcrumb to help debug why the device stopped working. In practice, I don't think I've ever seen this.
So really the difference should be transparent to the guest. The risk of an error allocating vectors on the host is the same, the only significant difference should be the amount of disruption at the device that we can better approximate the request of the guest.
Quoted 4 lines
This series aims to maintain existing error behavior for MSI (please see "vfio/pci: Remove interrupt context counter") but I would appreciate your guidance on whether existing error behavior should be maintained for MSI-X and how to do so if it is a requirement.
Based on above, there really can never be an error if we expect the device to work, so I think there's a misread of the current status. Dynamic MSI-X support should simply reduce the disruption and chance of lost interrupts at the device, but the points where we risk that the host cannot provide the configuration we need are the same. Thanks,
Alex
Hi Alex,
Quoted 3 lines · Alex Williamson
On Wed, 15 Mar 2023 13:59:20 -0700 Reinette Chatre <[email protected]> wrote:
...
Quoted 23 lines
Quoted 18 lines
== Why is this an RFC ? ==
vfio support for dynamic MSI-X needs to work with existing user space as well as upcoming user space that takes advantage of this feature. I would appreciate guidance on the expectations and requirements surrounding error handling when considering existing user space.
For example, consider the following scenario: Start: Consider a passthrough device that supports 10 MSI-X interrupts (0 to 9) and existing Qemu allocated interrupts 0 to 4.
Scenario: Qemu (hypothetically) attempts to associate a new action to interrupts 0 to 7. Early checking of this range in
vfio_set_irqs_validate_and_prepare()will pass since it is a valid range for the device. What happens after the early checking is considered next:Quoted 4 lines
Current behavior (before this series): Since the provided range, 0
- 7, exceeds the allocated range, no action will be taken on existing allocated interrupts 0 - 4 and the Qemu request will fail without making any state changes.
Quoted 4 lines
I must be missing something, it was described correctly earlier that QEMU will disable MSI-X and re-enable with a new vector count. Not only does QEMU not really have a way to fail this change, pretty much nothing would work if we did.
Thank you very much for confirming Qemu behavior.
One of my goals is to ensure that these kernel changes do not break existing user space in any way. The only area I could find where existing user space may encounter new behavior is if user space makes the (pre dynamic MSI-X) mistake of attempting to associate triggers to interrupts that have not been allocated. Thank you for confirming that this is not a valid scenario for Qemu.
Quoted 6 lines
What happens in this case is that the QEMU vfio-pci driver gets a
vector_usecallback for one of these new vectors {5,6,7},vfio_msix_vector_do_use()checks that's greater than we have enabled, disables MSI-X, and re-enables it for the new vector count. Worst case we'll do that 3 times if the vectors are presented in ascending order.
Indeed. While testing this work I modified Qemu's vfio_msix_vector_do_use() to consider VFIO_IRQ_INFO_NORESIZE. I saw the vector_use callback arriving for each new vector and having Qemu just associate a new action with the new vector (call vfio_set_irq_signaling()) instead of disabling MSI-X followed by enabling all vectors worked well with the changes I present here. A single interrupt was dynamically allocated and enabled without impacting others.
Quoted 5 lines
Based on above, there really can never be an error if we expect the device to work, so I think there's a misread of the current status. Dynamic MSI-X support should simply reduce the disruption and chance of lost interrupts at the device, but the points where we risk that the host cannot provide the configuration we need are the same.
Thank you very much Alex. In this case, please do consider this submission as a submission for inclusion. I'd be happy to resubmit without the "RFC" prefix if that is preferred.
Reinette
Quoted 9 lines
From: Chatre, Reinette <[email protected]>
Sent: Friday, March 17, 2023 7:38 AMQuoted 5 lines
Based on above, there really can never be an error if we expect the device to work, so I think there's a misread of the current status. Dynamic MSI-X support should simply reduce the disruption and chance of lost interrupts at the device, but the points where we risk that the host cannot provide the configuration we need are the same.
Quoted 4 lines
Thank you very much Alex. In this case, please do consider this submission as a submission for inclusion. I'd be happy to resubmit without the "RFC" prefix if that is preferred.
With that do we still want to keep the error behavior for MSI?
If no patch5 can be simplified e.g. no need of vfio_irq_ctx_range_allocated() and MSI/MSI-X error behaviors become consistent.
Hi Kevin,
Quoted 14 lines · Kevin
Quoted 9 lines
From: Chatre, Reinette <[email protected]>
Sent: Friday, March 17, 2023 7:38 AMQuoted 5 lines
Based on above, there really can never be an error if we expect the device to work, so I think there's a misread of the current status. Dynamic MSI-X support should simply reduce the disruption and chance of lost interrupts at the device, but the points where we risk that the host cannot provide the configuration we need are the same.
Quoted 4 lines
Thank you very much Alex. In this case, please do consider this submission as a submission for inclusion. I'd be happy to resubmit without the "RFC" prefix if that is preferred.
Quoted 4 lines
With that do we still want to keep the error behavior for MSI?
If no patch5 can be simplified e.g. no need of
vfio_irq_ctx_range_allocated()and MSI/MSI-X error behaviors become consistent.
Thank you. Yes, if I understand correctly MSI and MSI-X error handling can become consistent. I'll modify patch 5 to remove vfio_irq_ctx_range_allocated().
Reinette