Skip to contentSkip to conversation

vfio: Ignore -ENODEV when getting MSI cookie

RFC PATCH In review Andre Przywara 3 participants 4 messages Mar 12, 2020 – Mar 31, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 vfio: Ignore -ENODEV when getting MSI cookie

When we try to get an MSI cookie for a VFIO device, that can fail if CONFIG_IOMMU_DMA is not set. In this case iommu_get_msi_cookie() returns -ENODEV, and that should not be fatal.

Ignore that case and proceed with the initialisation.

This fixes VFIO with a platform device on the Calxeda Midway (no MSIs).

---
Hi,

not sure this is the right fix, or we should rather check if the platform doesn't support MSIs at all (which doesn't seem to be easy to do).
Or is this because arm-smmu.c always reserves an IOMMU_RESV_SW_MSI region?

Also this seems to be long broken, actually since Eric introduced MSI support in 4.10-rc3, but at least since the initialisation order was fixed with f6810c15cf9.

Grateful for any insight.

Cheers,
Andre

 drivers/vfio/vfio_iommu_type1.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/vfio/vfio_iommu_type1.c +1 -1
index a177bf2c6683..467e217ef09a 100644
@@ -1786,7 +1786,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,
 
 	if (resv_msi) {
 		ret = iommu_get_msi_cookie(domain->domain, resv_msi_base);
-		if (ret)
+		if (ret && ret != -ENODEV)
 			goto out_detach;
 	}
 
Signature · 1 line
2.17.1
Acked-by on v1 1/1
#1 lore raw
Quoted 9 lines · Andre Przywara

When we try to get an MSI cookie for a VFIO device, that can fail if CONFIG_IOMMU_DMA is not set. In this case iommu_get_msi_cookie() returns -ENODEV, and that should not be fatal.

Ignore that case and proceed with the initialisation.

This fixes VFIO with a platform device on the Calxeda Midway (no MSIs).

Quoted 8 lines

---
Hi,

not sure this is the right fix, or we should rather check if the platform doesn't support MSIs at all (which doesn't seem to be easy to do).
Or is this because arm-smmu.c always reserves an IOMMU_RESV_SW_MSI region?

Both, really - ideally VFIO should be able to skip all MSI_related setup if the system doesn't support MSIs, but equally the SMMU drivers would also ideally not expose a pointless SW_MSI region in the same situation.

In lieu of a 'nice' way of acheiving that, I think this patch seems reasonable - ENODEV doesn't clash with any real error that can occur when iommu-dma is present, and carrying on without a cookie should be fine since the MSI hooks that would otherwise dereference it will also be no-ops.

Perhaps it might be worth a comment to clarify that this is specifically to allow vfio-platform to work with iommu-dma disabled, but either way,

Quoted 3 lines

Also this seems to be long broken, actually since Eric introduced MSI support in 4.10-rc3, but at least since the initialisation order was fixed with f6810c15cf9.

I'm sure the entire Midway userbase have been up-in-arms the whole time... :P

Robin.

Quoted 23 lines

Grateful for any insight.

Cheers,
Andre

  drivers/vfio/vfio_iommu_type1.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/vfio/vfio_iommu_type1.c +1 -1
index a177bf2c6683..467e217ef09a 100644
@@ -1786,7 +1786,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,
 
  	if (resv_msi) {
  		ret = iommu_get_msi_cookie(domain->domain, resv_msi_base);
-		if (ret)
+		if (ret && ret != -ENODEV)
  			goto out_detach;
  	}
 
Reviewed-by
#2 lore raw

Hi Andre,

Quoted 23 lines · Robin Murphy
Quoted 9 lines · Andre Przywara

When we try to get an MSI cookie for a VFIO device, that can fail if CONFIG_IOMMU_DMA is not set. In this case iommu_get_msi_cookie() returns -ENODEV, and that should not be fatal.

Ignore that case and proceed with the initialisation.

This fixes VFIO with a platform device on the Calxeda Midway (no MSIs).

Quoted 8 lines

---
Hi,

not sure this is the right fix, or we should rather check if the platform doesn't support MSIs at all (which doesn't seem to be easy to do).
Or is this because arm-smmu.c always reserves an IOMMU_RESV_SW_MSI region?

Both, really - ideally VFIO should be able to skip all MSI_related setup if the system doesn't support MSIs, but equally the SMMU drivers would also ideally not expose a pointless SW_MSI region in the same situation.

Quoted 5 lines

In lieu of a 'nice' way of acheiving that, I think this patch seems reasonable - ENODEV doesn't clash with any real error that can occur when iommu-dma is present, and carrying on without a cookie should be fine since the MSI hooks that would otherwise dereference it will also be no-ops.

Looks OK to me as well.

About looking at whether MSI is in use I wonder whether we could do like irq_domain_check_msi_remap() in kernel/irq/irqdomain.c without checking IRQ_DOMAIN_FLAG_MSI_REMAP.

But if this simple patch fixes this marginal Midway vfio-platform use case, that should be good enough.

Thanks

Eric

Quoted 39 lines

Perhaps it might be worth a comment to clarify that this is specifically to allow vfio-platform to work with iommu-dma disabled, but either way,

Quoted 3 lines

Also this seems to be long broken, actually since Eric introduced MSI support in 4.10-rc3, but at least since the initialisation order was fixed with f6810c15cf9.

I'm sure the entire Midway userbase have been up-in-arms the whole time... :P

Robin.

Quoted 23 lines

Grateful for any insight.

Cheers,
Andre

  drivers/vfio/vfio_iommu_type1.c | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/vfio/vfio_iommu_type1.c +1 -1
index a177bf2c6683..467e217ef09a 100644
@@ -1786,7 +1786,7 @@ static int vfio_iommu_type1_attach_group(void
 *iommu_data,
         if (resv_msi) {
           ret = iommu_get_msi_cookie(domain->domain, resv_msi_base);
-        if (ret)
+        if (ret && ret != -ENODEV)
               goto out_detach;
       }

#3 lore raw
In reply to Andre Przywara

Hi Andre,

Quoted 9 lines · Andre Przywara

When we try to get an MSI cookie for a VFIO device, that can fail if CONFIG_IOMMU_DMA is not set. In this case iommu_get_msi_cookie() returns -ENODEV, and that should not be fatal.

Ignore that case and proceed with the initialisation.

This fixes VFIO with a platform device on the Calxeda Midway (no MSIs).

Would you mind resending this as non RFC (+ R-b tags) so that it gets a chance to be taken by Alex for 5.7

Thanks

Eric

Quoted 35 lines

---
Hi,

not sure this is the right fix, or we should rather check if the platform doesn't support MSIs at all (which doesn't seem to be easy to do).
Or is this because arm-smmu.c always reserves an IOMMU_RESV_SW_MSI region?

Also this seems to be long broken, actually since Eric introduced MSI support in 4.10-rc3, but at least since the initialisation order was fixed with f6810c15cf9.

Grateful for any insight.

Cheers,
Andre

 drivers/vfio/vfio_iommu_type1.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/vfio/vfio_iommu_type1.c +1 -1
index a177bf2c6683..467e217ef09a 100644
@@ -1786,7 +1786,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,
 
 	if (resv_msi) {
 		ret = iommu_get_msi_cookie(domain->domain, resv_msi_base);
-		if (ret)
+		if (ret && ret != -ENODEV)
 			goto out_detach;
 	}