Skip to contentSkip to conversation

iommu/intel: SNP bit is not dependent on iommu domain coherency

PATCH In review Alex Williamson 2 participants 4 messages Oct 29, 2013 – Jan 7, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iommu/intel: SNP bit is not dependent on iommu domain coherency

The setting of the SNP bit in the intel-iommu page tables should not be dependent on the current capability of the iommu domain. The current VT-d spec (2.2) indicates the SNP bit is "treated as reserved[0] by hardware implementations not supporting Snoop Control". Furthermore, section 3.7.3 indicates:

If the Snoop Control (SC) field in extended capability Register is reported as 0, snoop behavior for access to the page mapped through second-level translation is determined by the no-snoop attribute in the request.

This all seems to indicate that hardware incapable of Snoop Control will handle the SNP bit as zero regardless of the value stored in the PTE.

The trouble with the current implementation is that mapping flags depend on the state of the iommu domain at the time of the mapping, yet no attempt is made to update existing mappings when the iommu domain composition changes. This leaves the iommu domain in a state where some mappings may enforce coherency, others do not, and the user of the IOMMU API has no ability to later enable the desired flags atomically with respect to DMA.

If we always honor the IOMMU_CACHE flag then an IOMMU API user who specifies IOMMU_CACHE for all mappings can assume that the coherency of the mappings within a domain follow the coherency capability of the domain itself.

---
 drivers/iommu/intel-iommu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +1 -1
index 15e9b57..c46c6a6 100644
@@ -4084,7 +4084,7 @@ static int intel_iommu_map(struct iommu_domain *domain,
 		prot |= DMA_PTE_READ;
 	if (iommu_prot & IOMMU_WRITE)
 		prot |= DMA_PTE_WRITE;
-	if ((iommu_prot & IOMMU_CACHE) && dmar_domain->iommu_snooping)
+	if (iommu_prot & IOMMU_CACHE)
 		prot |= DMA_PTE_SNP;
 
 	max_addr = iova + size;
#1 lore raw

David,

Any comments on this patch? Thanks,

Alex

Quoted 47 lines · Alex Williamson

The setting of the SNP bit in the intel-iommu page tables should not be dependent on the current capability of the iommu domain. The current VT-d spec (2.2) indicates the SNP bit is "treated as reserved[0] by hardware implementations not supporting Snoop Control". Furthermore, section 3.7.3 indicates:

If the Snoop Control (SC) field in extended capability Register is reported as 0, snoop behavior for access to the page mapped through second-level translation is determined by the no-snoop attribute in the request.

This all seems to indicate that hardware incapable of Snoop Control will handle the SNP bit as zero regardless of the value stored in the PTE.

The trouble with the current implementation is that mapping flags depend on the state of the iommu domain at the time of the mapping, yet no attempt is made to update existing mappings when the iommu domain composition changes. This leaves the iommu domain in a state where some mappings may enforce coherency, others do not, and the user of the IOMMU API has no ability to later enable the desired flags atomically with respect to DMA.

If we always honor the IOMMU_CACHE flag then an IOMMU API user who specifies IOMMU_CACHE for all mappings can assume that the coherency of the mappings within a domain follow the coherency capability of the domain itself.

---
 drivers/iommu/intel-iommu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +1 -1
index 15e9b57..c46c6a6 100644
@@ -4084,7 +4084,7 @@ static int intel_iommu_map(struct iommu_domain *domain,
 		prot |= DMA_PTE_READ;
 	if (iommu_prot & IOMMU_WRITE)
 		prot |= DMA_PTE_WRITE;
-	if ((iommu_prot & IOMMU_CACHE) && dmar_domain->iommu_snooping)
+	if (iommu_prot & IOMMU_CACHE)
 		prot |= DMA_PTE_SNP;
 
 	max_addr = iova + size;
#2 lore raw

Alex Williamson wrote on 2013-12-24:

Quoted 4 lines

David,

Any comments on this patch? Thanks,

Hi Alex,

There do have some IOMMUs will treat SNP bit in the PTE as reserved (0) and will cause a reserved field violation fault if it is set but hardware not support snoop-control(bit 7 in ECAP_REG is 0). So your patch seems wrong to me.

Quoted 40 lines

Alex

drivers/iommu/intel-iommu.c static int intel_iommu_map(struct iommu_domain

Quoted 36 lines · Alex Williamson

The setting of the SNP bit in the intel-iommu page tables should not be dependent on the current capability of the iommu domain. The current VT-d spec (2.2) indicates the SNP bit is "treated as reserved[0] by hardware implementations not supporting Snoop Control". Furthermore, section 3.7.3 indicates:

If the Snoop Control (SC) field in extended capability Register is reported as 0, snoop behavior for access to the page mapped through second-level translation is determined by the no-snoop attribute in the request.
This all seems to indicate that hardware incapable of Snoop Control will handle the SNP bit as zero regardless of the value stored in the PTE.

The trouble with the current implementation is that mapping flags depend on the state of the iommu domain at the time of the mapping, yet no attempt is made to update existing mappings when the iommu domain composition changes. This leaves the iommu domain in a state where some mappings may enforce coherency, others do not, and the user of the IOMMU API has no ability to later enable the desired flags atomically with respect to DMA.

If we always honor the IOMMU_CACHE flag then an IOMMU API user who specifies IOMMU_CACHE for all mappings can assume that the coherency of the mappings within a domain follow the coherency capability of the domain itself.

---
 drivers/iommu/intel-iommu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index 15e9b57..c46c6a6 100644

drivers/iommu/intel-iommu.c +0 -0
@@ -4084,7 +4084,7 @@ static int intel_iommu_map(struct iommu_domain

*domain,

Quoted 10 lines

drivers/iommu/intel-iommu.c static int intel_iommu_map(struct iommu_domain *domain, · line 4090

Quoted 8 lines
drivers/iommu/intel-iommu.c +1 -1
 		prot |= DMA_PTE_READ;
 	if (iommu_prot & IOMMU_WRITE)
 		prot |= DMA_PTE_WRITE;
-	if ((iommu_prot & IOMMU_CACHE) && dmar_domain->iommu_snooping)
+	if (iommu_prot & IOMMU_CACHE)
 		prot |= DMA_PTE_SNP;
 
 	max_addr = iova + size;

Best regards,
Yang

#3 lore raw
Quoted 6 lines · Yang Z

Alex Williamson wrote on 2013-12-24:

Quoted 4 lines

David,

Any comments on this patch? Thanks,

Quoted 6 lines

Hi Alex,

There do have some IOMMUs will treat SNP bit in the PTE as reserved
(0) and will cause a reserved field violation fault if it is set but hardware not support snoop-control(bit 7 in ECAP_REG is 0). So your patch seems wrong to me.

Thanks for the reply Yang. So effectively IOMMU_CACHE (SNP) is unusable on intel-iommu if we want a domain that can manage any device in the system, right? We have no way to atomically change the SNP bit across a domain so the only chance for enabling it is if all the DRHD units support snoop-control, but then a hot-added DRHD makes that impossible to predict. Is there any way we can make intel-iommu "do the right thing"? These are rather low-level implementation details of intel-iommu that I'd rather not know about as a user of the IOMMU API. Thanks,

Alex

Quoted 54 lines
Quoted 38 lines · Alex Williamson

drivers/iommu/intel-iommu.c static int intel_iommu_map(struct iommu_domain

Quoted 36 lines

The setting of the SNP bit in the intel-iommu page tables should not be dependent on the current capability of the iommu domain. The current VT-d spec (2.2) indicates the SNP bit is "treated as reserved[0] by hardware implementations not supporting Snoop Control". Furthermore, section 3.7.3 indicates:

If the Snoop Control (SC) field in extended capability Register is reported as 0, snoop behavior for access to the page mapped through second-level translation is determined by the no-snoop attribute in the request.
This all seems to indicate that hardware incapable of Snoop Control will handle the SNP bit as zero regardless of the value stored in the PTE.

The trouble with the current implementation is that mapping flags depend on the state of the iommu domain at the time of the mapping, yet no attempt is made to update existing mappings when the iommu domain composition changes. This leaves the iommu domain in a state where some mappings may enforce coherency, others do not, and the user of the IOMMU API has no ability to later enable the desired flags atomically with respect to DMA.

If we always honor the IOMMU_CACHE flag then an IOMMU API user who specifies IOMMU_CACHE for all mappings can assume that the coherency of the mappings within a domain follow the coherency capability of the domain itself.

---
 drivers/iommu/intel-iommu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index 15e9b57..c46c6a6 100644

drivers/iommu/intel-iommu.c +0 -0
@@ -4084,7 +4084,7 @@ static int intel_iommu_map(struct iommu_domain

*domain,

Quoted 10 lines

drivers/iommu/intel-iommu.c static int intel_iommu_map(struct iommu_domain *domain, · line 4090

Quoted 8 lines
drivers/iommu/intel-iommu.c +1 -1
 		prot |= DMA_PTE_READ;
 	if (iommu_prot & IOMMU_WRITE)
 		prot |= DMA_PTE_WRITE;
-	if ((iommu_prot & IOMMU_CACHE) && dmar_domain->iommu_snooping)
+	if (iommu_prot & IOMMU_CACHE)
 		prot |= DMA_PTE_SNP;
 
 	max_addr = iova + size;

Best regards,
Yang