Skip to contentSkip to conversation

iommu: Add support to change domain to blocked domain via sysfs

PATCH In review Vasant Hegde 2 participants 9 messages Jul 22, 2024 – Oct 16, 2024 IOMMU

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iommu: Add support to change domain to blocked domain via sysfs

Add support to change default domain to 'blocked' domain. This is useful for debugging.

iommu_group_show_type() has support to display 'blocked' domain type.

---
 drivers/iommu/iommu.c | 2 ++
 include/linux/iommu.h | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)
drivers/iommu/iommu.c +2 -0
index 9df7cc75c1bc..5fd6afd9090f 100644
@@ -3041,6 +3041,8 @@ static ssize_t iommu_group_store_type(struct iommu_group *group,
 		req_type = IOMMU_DOMAIN_DMA;
 	else if (sysfs_streq(buf, "DMA-FQ"))
 		req_type = IOMMU_DOMAIN_DMA_FQ;
+	else if (sysfs_streq(buf, "blocked"))
+		req_type = IOMMU_DOMAIN_BLOCKED;
 	else if (sysfs_streq(buf, "auto"))
 		req_type = 0;
 	else
include/linux/iommu.h +2 -1
index 17b3f36ad843..2741c35a6788 100644
@@ -169,6 +169,7 @@ struct iommu_domain_geometry {
 
 #define __IOMMU_DOMAIN_NESTED	(1U << 6)  /* User-managed address space nested
 					      on a stage-2 translation        */
+#define __IOMMU_DOMAIN_BLOCKED	(1U << 7)  /* Blocked domain */
 
 #define IOMMU_DOMAIN_ALLOC_FLAGS ~__IOMMU_DOMAIN_DMA_FQ
 /*
@@ -189,7 +190,7 @@ struct iommu_domain_geometry {
  *	IOMMU_DOMAIN_PLATFORM	- Legacy domain for drivers that do their own
  *				  dma_api stuff. Do not use in new drivers.
  */
-#define IOMMU_DOMAIN_BLOCKED	(0U)
+#define IOMMU_DOMAIN_BLOCKED	(__IOMMU_DOMAIN_BLOCKED)
 #define IOMMU_DOMAIN_IDENTITY	(__IOMMU_DOMAIN_PT)
 #define IOMMU_DOMAIN_UNMANAGED	(__IOMMU_DOMAIN_PAGING)
 #define IOMMU_DOMAIN_DMA	(__IOMMU_DOMAIN_PAGING |	\
Signature · 1 line
2.31.1
#1 lore raw
Quoted 2 lines · Vasant Hegde

Add support to change default domain to 'blocked' domain. This is useful for debugging.

But how does it work? If you set the default domain (aka the domain used by the DMA API) to blocked, then surely no drivers should be bindable too? Did that happen inadvertanly for some other reason?

Maybe this is just a neat hack for testing purposes?

Jason

#2 lore raw

Hi Jason,

Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Vasant Hegde

Add support to change default domain to 'blocked' domain. This is useful for debugging.

But how does it work?

1. unload the module
2. echo "blocked" > /sys/kernel/iommu_group/<groupid>/type

Quoted 3 lines

If you set the default domain (aka the domain
used by the DMA API) to blocked, then surely no drivers should be bindable too?

That's correct.

Quoted 3 lines

Did that happen inadvertanly for some other reason?

Maybe this is just a neat hack for testing purposes?

Correct. I wanted to test blocked domain implementation. I found this is the neat way of doing it. I used it for testing and then thought its better to put it in upstream. Hence posted it.

-Vasant

#3 lore raw
Quoted 10 lines · Vasant Hegde

Hi Jason,

Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Vasant Hegde

Add support to change default domain to 'blocked' domain. This is useful for debugging.

But how does it work?

Quoted 8 lines

1. unload the module
2. echo "blocked" > /sys/kernel/iommu_group/<groupid>/type

Quoted 3 lines

If you set the default domain (aka the domain
used by the DMA API) to blocked, then surely no drivers should be bindable too?

That's correct.

What prevents binding after being blocked though?

Jason

#4 lore raw

Jason,

Quoted 14 lines · Jason Gunthorpe
Quoted 13 lines · Vasant Hegde

Hi Jason,

Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Vasant Hegde

Add support to change default domain to 'blocked' domain. This is useful for debugging.

But how does it work?

1. unload the module
2. echo "blocked" > /sys/kernel/iommu_group/<groupid>/type

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

If you set the default domain (aka the domain
used by the DMA API) to blocked, then surely no drivers should be bindable too?

That's correct.

What prevents binding after being blocked though?

Device is in blocked domain. So any access from device will be blocked.

-Vasant

#5 lore raw
Quoted 21 lines · Vasant Hegde

Jason,

Quoted 18 lines · Jason Gunthorpe
Quoted 17 lines · Vasant Hegde

Hi Jason,

Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Vasant Hegde

Add support to change default domain to 'blocked' domain. This is useful for debugging.

But how does it work?

1. unload the module
2. echo "blocked" > /sys/kernel/iommu_group/<groupid>/type

Quoted 3 lines

If you set the default domain (aka the domain
used by the DMA API) to blocked, then surely no drivers should be bindable too?

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

That's correct.

What prevents binding after being blocked though?

Device is in blocked domain. So any access from device will be blocked.

My point is that is just going to make any driver that binds to it malfunction. The DMA API does not comprehend a blocked translation to fail all DMAs. It is great for testing because you call all the DMA failures logged, but not useful otherwise.

Jason

#6 lore raw

Jason,

Quoted 24 lines · Jason Gunthorpe
Quoted 23 lines · Vasant Hegde

Jason,

Quoted 20 lines · Jason Gunthorpe
Quoted 18 lines · Vasant Hegde

Hi Jason,

Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Vasant Hegde

Add support to change default domain to 'blocked' domain. This is useful for debugging.

But how does it work?

1. unload the module
2. echo "blocked" > /sys/kernel/iommu_group/<groupid>/type

Quoted 3 lines

If you set the default domain (aka the domain
used by the DMA API) to blocked, then surely no drivers should be bindable too?

That's correct.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

What prevents binding after being blocked though?

Device is in blocked domain. So any access from device will be blocked.

My point is that is just going to make any driver that binds to it malfunction. The DMA API does not comprehend a blocked translation to fail all DMAs. It is great for testing because you call all the DMA failures logged, but not useful otherwise.

That's right. Purpose is to test blocked domain.

I'd expect someone changing domain via sysfs knows what he is doing. Hence I thought its fine to add blocked domain support.

-Vasant

#7 lore raw

Hi Jason, Joerg,

Quoted 37 lines · Vasant Hegde

Jason,

Quoted 24 lines · Jason Gunthorpe
Quoted 23 lines · Vasant Hegde

Jason,

Quoted 20 lines · Jason Gunthorpe
Quoted 18 lines · Vasant Hegde

Hi Jason,

Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Vasant Hegde

Add support to change default domain to 'blocked' domain. This is useful for debugging.

But how does it work?

1. unload the module
2. echo "blocked" > /sys/kernel/iommu_group/<groupid>/type

Quoted 3 lines

If you set the default domain (aka the domain
used by the DMA API) to blocked, then surely no drivers should be bindable too?

That's correct.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

What prevents binding after being blocked though?

Device is in blocked domain. So any access from device will be blocked.

My point is that is just going to make any driver that binds to it malfunction. The DMA API does not comprehend a blocked translation to fail all DMAs. It is great for testing because you call all the DMA failures logged, but not useful otherwise.

Quoted 4 lines

That's right. Purpose is to test blocked domain.

I'd expect someone changing domain via sysfs knows what he is doing. Hence I thought its fine to add blocked domain support.

Can we get this merged?

Currently other than this patch we don't have any way to force devices to attached to blocked domain. So testing BLOCKED domain and confirming we didn't break anything becomes difficult.

I have been carrying this as out of tree patch. I apply this patch and run my tests . But its nice if we can get this patch in upstream.

-Vasant

#8 lore raw
Quoted 8 lines · Vasant Hegde
Quoted 2 lines

I'd expect someone changing domain via sysfs knows what he is doing. Hence I thought its fine to add blocked domain support.

Can we get this merged?

Currently other than this patch we don't have any way to force devices to attached to blocked domain. So testing BLOCKED domain and confirming we didn't break anything becomes difficult.

You can open vfio and get a blocked domain?

I not sure we should merge this, it is confusing to end users to put the device into a non-functional state..

Quoted 2 lines

I have been carrying this as out of tree patch. I apply this patch and run my tests . But its nice if we can get this patch in upstream.

Write your test with VFIO instead?

Joel has a NVMe VFIO driver, and rdma-core has a mlx4 vfio driver, I think it would be easy enough to make those generate blocked transfers??

Jason