Skip to contentSkip to conversation

intel-iommu: Default to non-coherent for domains unattached to iommus

PATCH Applied Alex Williamson 3 participants 3 messages Nov 12, 2011 – Sep 18, 2012

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 intel-iommu: Default to non-coherent for domains unattached to iommus

domain_update_iommu_coherency() currently defaults to setting domains as coherent when the domain is not attached to any iommus. This allows for a window in domain_context_mapping_one() where such a domain can update context entries non-coherently, and only after update the domain capability to clear iommu_coherency.

This can be seen using KVM device assignment on VT-d systems that do not support coherency in the ecap register. When a device is added to a guest, a domain is created (iommu_coherency = 0), the device is attached, and ranges are mapped. If we then hot unplug the device, the coherency is updated and set to the default (1) since no iommus are attached to the domain. A subsequent attach of a device makes use of the same dmar domain (now marked coherent) updates context entries with coherency enabled, and only disables coherency as the last step in the process.

To fix this, switch domain_update_iommu_coherency() to use the safer, non-coherent default for domains not attached to iommus.

---

v2: Fix spelling in commit log :-\

 drivers/iommu/intel-iommu.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
drivers/iommu/intel-iommu.c +3 -1
index c0c7820..6b9d8c1 100644
@@ -560,7 +560,9 @@ static void domain_update_iommu_coherency(struct dmar_domain *domain)
 {
 	int i;
 
-	domain->iommu_coherency = 1;
+	i = find_first_bit(&domain->iommu_bmp, g_num_of_iommus);
+
+	domain->iommu_coherency = i < g_num_of_iommus ? 1 : 0;
 
 	for_each_set_bit(i, &domain->iommu_bmp, g_num_of_iommus) {
 		if (!ecap_coherent(g_iommus[i]->ecap)) {
#1 lore raw

Review static void domain_update_iommu_coherency(struct

Quoted 17 lines · Alex Williamson <alex.williamson <at> redhat.com> writes:

domain_update_iommu_coherency() currently defaults to setting domains as coherent when the domain is not attached to any iommus. This allows for a window in domain_context_mapping_one() where such a domain can update context entries non-coherently, and only after update the domain capability to clear iommu_coherency.

This can be seen using KVM device assignment on VT-d systems that do not support coherency in the ecap register. When a device is added to a guest, a domain is created (iommu_coherency = 0), the device is attached, and ranges are mapped. If we then hot unplug the device, the coherency is updated and set to the default (1) since no iommus are attached to the domain. A subsequent attach of a device makes use of the same dmar domain (now marked coherent) updates context entries with coherency enabled, and only disables coherency as the last step in the process.

(quoted diff) +0 -0
@@ -560,7 +560,9 @@ static void domain_update_iommu_coherency(struct 

dmar_domain *domain)

drivers/iommu/intel-iommu.c static void domain_update_iommu_coherency(struct dmar_domain *domain) · line 568

Quoted 12 lines
drivers/iommu/intel-iommu.c +3 -1
 {
 	int i;
 
-	domain->iommu_coherency = 1;
+	i = find_first_bit(&domain->iommu_bmp, g_num_of_iommus);
+
+	domain->iommu_coherency = i < g_num_of_iommus ? 1 : 0;
 
 	for_each_set_bit(i, &domain->iommu_bmp, g_num_of_iommus) {
 		if (!ecap_coherent(g_iommus[i]->ecap)) {

I'm running into this same problem on two HP systems while testing pass through mode. Both systems are running the iommu in non-coherent mode and it does not appear there is a way to change it in BIOS. I applied the above code change to my 3.5 kernel and the original problem has gone away but now I'm getting a fault 6 on a PTE entry.

DMAR:[DMA Read] Request device [00:1e.0] fault addr 1000 DMAR:[fault reason 06] PTE Read access is not set

Is anyone working on a fix for this problem?

-Craig

Applied
#2 lore raw
In reply to Alex Williamson
Quoted 16 lines · Alex Williamson

domain_update_iommu_coherency() currently defaults to setting domains as coherent when the domain is not attached to any iommus. This allows for a window in domain_context_mapping_one() where such a domain can update context entries non-coherently, and only after update the domain capability to clear iommu_coherency.

This can be seen using KVM device assignment on VT-d systems that do not support coherency in the ecap register. When a device is added to a guest, a domain is created (iommu_coherency = 0), the device is attached, and ranges are mapped. If we then hot unplug the device, the coherency is updated and set to the default (1) since no iommus are attached to the domain. A subsequent attach of a device makes use of the same dmar domain (now marked coherent) updates context entries with coherency enabled, and only disables coherency as the last step in the process.

Quoted 7 lines

To fix this, switch domain_update_iommu_coherency() to use the safer, non-coherent default for domains not attached to iommus.

Applied, thanks.

Signature · 5 lines
AMD Operating System Research Center

Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach
General Managers: Alberto Bozzo
Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632