intel-iommu: Default to non-coherent for domains unattached to iommus
PATCH v2 1/1 intel-iommu: Default to non-coherent for domains unattached to iommus
domain_update_iommu_coherency() currently defaults to setting domains as coherent when the domain is not attached to any iommus. This allows for a window in domain_context_mapping_one() where such a domain can update context entries non-coherently, and only after update the domain capability to clear iommu_coherency.
This can be seen using KVM device assignment on VT-d systems that do not support coherency in the ecap register. When a device is added to a guest, a domain is created (iommu_coherency = 0), the device is attached, and ranges are mapped. If we then hot unplug the device, the coherency is updated and set to the default (1) since no iommus are attached to the domain. A subsequent attach of a device makes use of the same dmar domain (now marked coherent) updates context entries with coherency enabled, and only disables coherency as the last step in the process.
To fix this, switch domain_update_iommu_coherency() to use the safer, non-coherent default for domains not attached to iommus.
---
v2: Fix spelling in commit log :-\
drivers/iommu/intel-iommu.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
drivers/iommu/intel-iommu.c +3 -1
@@ -560,7 +560,9 @@ static void domain_update_iommu_coherency(struct dmar_domain *domain) {int i; - domain->iommu_coherency =1 ; + i =find_first_bit (&domain->iommu_bmp, g_num_of_iommus); + + domain->iommu_coherency = i < g_num_of_iommus ?1 :0 ;for_each_set_bit (i, &domain->iommu_bmp, g_num_of_iommus) {if (!ecap_coherent (g_iommus[i]->ecap)) {
Review static void domain_update_iommu_coherency(struct
Quoted 17 lines · Alex Williamson <alex.williamson <at> redhat.com> writes:
domain_update_iommu_coherency()currently defaults to setting domains as coherent when the domain is not attached to any iommus. This allows for a window indomain_context_mapping_one()where such a domain can update context entries non-coherently, and only after update the domain capability to cleariommu_coherency.This can be seen using KVM device assignment on VT-d systems that do not support coherency in the ecap register. When a device is added to a guest, a domain is created (
iommu_coherency= 0), the device is attached, and ranges are mapped. If we then hot unplug the device, the coherency is updated and set to the default (1) since no iommus are attached to the domain. A subsequent attach of a device makes use of the same dmar domain (now marked coherent) updates context entries with coherency enabled, and only disables coherency as the last step in the process.(quoted diff) +0
-0@@ -560,7 +560,9 @@ static void domain_update_iommu_coherency(struct
dmar_domain *domain)
drivers/iommu/intel-iommu.c static void domain_update_iommu_coherency(struct dmar_domain *domain) · line 568
Quoted 12 lines
drivers/iommu/intel-iommu.c +3
-1{int i; - domain->iommu_coherency =1 ; + i =find_first_bit (&domain->iommu_bmp, g_num_of_iommus); + + domain->iommu_coherency = i < g_num_of_iommus ?1 :0 ;for_each_set_bit (i, &domain->iommu_bmp, g_num_of_iommus) {if (!ecap_coherent (g_iommus[i]->ecap)) {
I'm running into this same problem on two HP systems while testing pass through mode. Both systems are running the iommu in non-coherent mode and it does not appear there is a way to change it in BIOS. I applied the above code change to my 3.5 kernel and the original problem has gone away but now I'm getting a fault 6 on a PTE entry.
DMAR:[DMA Read] Request device [00:1e.0] fault addr 1000 DMAR:[fault reason 06] PTE Read access is not set
Is anyone working on a fix for this problem?
-Craig
Joerg Roedel
AppliedQuoted 16 lines · Alex Williamson
domain_update_iommu_coherency()currently defaults to setting domains as coherent when the domain is not attached to any iommus. This allows for a window indomain_context_mapping_one()where such a domain can update context entries non-coherently, and only after update the domain capability to cleariommu_coherency.This can be seen using KVM device assignment on VT-d systems that do not support coherency in the ecap register. When a device is added to a guest, a domain is created (
iommu_coherency= 0), the device is attached, and ranges are mapped. If we then hot unplug the device, the coherency is updated and set to the default (1) since no iommus are attached to the domain. A subsequent attach of a device makes use of the same dmar domain (now marked coherent) updates context entries with coherency enabled, and only disables coherency as the last step in the process.
Quoted 7 lines
To fix this, switch
domain_update_iommu_coherency()to use the safer, non-coherent default for domains not attached to iommus.Signed-off-by Alex Williamson <[email protected]>Tested-by Donald Dutile <[email protected]>Acked-by Donald Dutile <[email protected]>Acked-by Chris Wright <[email protected]>
Applied, thanks.
Signature · 5 lines
AMD Operating System Research Center Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach General Managers: Alberto Bozzo Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632