iommu/vt-d: Fix PASID directory pointer coherency
Subsystems (3)
PATCH v3 1/1 iommu/vt-d: Fix PASID directory pointer coherency
On platforms that do not support IOMMU Extended capability bit 0 Page-walk Coherency, CPU caches are not snooped when IOMMU is accessing any translation structures. IOMMU access goes only directly to memory. Intel IOMMU code was missing a flush for the PASID table directory that resulted in the unrecoverable fault as shown below.
This patch adds clflush calls whenever allocating and updating a PASID table directory to ensure cache coherency.
On the reverse direction, there's no need to clflush the PASID directory pointer when we deactivate a context entry in that IOMMU hardware will not see the old PASID directory pointer after we clear the context entry. PASID directory entries are also never freed once allocated.
[ 0.555386] DMAR: DRHD: handling fault status reg 3 [ 0.555805] DMAR: [DMA Read NO_PASID] Request device [00:0d.2] fault addr 0x1026a4000 [fault reason 0x51] SM: Present bit in Directory Entry is clear [ 0.556348] DMAR: Dump dmar1 table entries for IOVA 0x1026a4000 [ 0.556348] DMAR: scalable mode root entry: hi 0x0000000102448001, low 0x0000000101b3e001 [ 0.556348] DMAR: context entry: hi 0x0000000000000000, low 0x0000000101b4d401 [ 0.556348] DMAR: pasid dir entry: 0x0000000101b4e001 [ 0.556348] DMAR: pasid table entry[0]: 0x0000000000000109 [ 0.556348] DMAR: pasid table entry[1]: 0x0000000000000001 [ 0.556348] DMAR: pasid table entry[2]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[3]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[4]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[5]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[6]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[7]: 0x0000000000000000 [ 0.556348] DMAR: PTE not present at level 4
---
v3: Add clflush after PASID directory allocation to prevent malicious device attack with unauthorized PASIDs. Also flush all the PASID entries after directory updates. (Baolu)
v2: Add clflush to PASID directory update case (Baolu, Kevin review)
--- drivers/iommu/intel/iommu.c | 2 ++ drivers/iommu/intel/pasid.c | 7 +++++++ 2 files changed, 9 insertions(+)
drivers/iommu/intel/iommu.c +2 -0
@@ -1976,6 +1976,8 @@ static int domain_context_mapping_one(struct dmar_domain *domain, pds =context_get_sm_pds (table); context->lo = (u64 )virt_to_phys (table->table) |context_pdts (pds); +if (!ecap_coherent (iommu->ecap)) +clflush_cache_range (table->table,sizeof (u64));/* Setup the RID_PASID field: */ context_set_sm_rid2pasid (context,PASID_RID2PASID );
drivers/iommu/intel/pasid.c +7 -0
@@ -128,6 +128,9 @@ int intel_pasid_alloc_table(struct device *dev) pasid_table->max_pasid =1 << (order +PAGE_SHIFT +3 ); info->pasid_table = pasid_table; +if (!ecap_coherent (info->iommu->ecap)) +clflush_cache_range (pasid_table->table, size); +return 0 ; }
@@ -215,6 +218,10 @@ static struct pasid_entry *intel_pasid_get_entry(struct device *dev, u32 pasid)free_pgtable_page (entries); goto retry; } +if (!ecap_coherent (info->iommu->ecap)) { +clflush_cache_range (entries,VTD_PAGE_SIZE ); +clflush_cache_range (&dir[dir_index].val,sizeof (*dir)); + } }return &entries[index];
Signature · 1 line
2.25.1
Quoted 65 lines
From: Jacob Pan <[email protected]>
Sent: Thursday, February 9, 2023 1:19 AM
To: LKML <[email protected]>; [email protected]; Lu Baolu <[email protected]>; Joerg Roedel <[email protected]>
Cc: David Woodhouse <[email protected]>; Raj, Ashok <[email protected]>; Tian, Kevin <[email protected]>; Liu, Yi L <[email protected]>; Jacob Pan <[email protected]>; [email protected]; Ghorai, Sukumar <[email protected]>
Subject: [PATCH v3] iommu/vt-d: Fix PASID directory pointer coherencyOn platforms that do not support IOMMU Extended capability bit 0 Page-walk Coherency, CPU caches are not snooped when IOMMU is accessing
any translation structures. IOMMU access goes only directly to memory. Intel IOMMU code was missing a flush for the PASID table directory that resulted in the unrecoverable fault as shown below.This patch adds clflush calls whenever allocating and updating a PASID table directory to ensure cache coherency.
On the reverse direction, there's no need to clflush the PASID directory pointer when we deactivate a context entry in that IOMMU hardware will not see the old PASID directory pointer after we clear the context entry. PASID directory entries are also never freed once allocated.
[ 0.555386] DMAR: DRHD: handling fault status reg 3 [ 0.555805] DMAR: [DMA Read NO_PASID] Request device [00:0d.2] fault addr 0x1026a4000 [fault reason 0x51] SM: Present bit in Directory Entry is clear [ 0.556348] DMAR: Dump dmar1 table entries for IOVA 0x1026a4000 [ 0.556348] DMAR: scalable mode root entry: hi 0x0000000102448001, low 0x0000000101b3e001 [ 0.556348] DMAR: context entry: hi 0x0000000000000000, low 0x0000000101b4d401 [ 0.556348] DMAR: pasid dir entry: 0x0000000101b4e001 [ 0.556348] DMAR: pasid table entry[0]: 0x0000000000000109 [ 0.556348] DMAR: pasid table entry[1]: 0x0000000000000001 [ 0.556348] DMAR: pasid table entry[2]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[3]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[4]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[5]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[6]: 0x0000000000000000 [ 0.556348] DMAR: pasid table entry[7]: 0x0000000000000000 [ 0.556348] DMAR: PTE not present at level 4Cc <[email protected]>Fixes 0bbeb01a4faf ("iommu/vt-d: Manage scalalble mode PASID tables")Reported-by Sukumar Ghorai <[email protected]>Signed-off-by Ashok Raj <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v3: Add clflush after PASID directory allocation to prevent malicious device attack with unauthorized PASIDs. Also flush all the PASID entries after directory updates. (Baolu)
v2: Add clflush to PASID directory update case (Baolu, Kevin review)--- drivers/iommu/intel/iommu.c | 2 ++ drivers/iommu/intel/pasid.c | 7 +++++++ 2 files changed, 9 insertions(+)drivers/iommu/intel/iommu.c +0
-0
drivers/iommu/intel/iommu.c static int domain_context_mapping_one(struct · line 1981
Quoted 6 lines
drivers/iommu/intel/iommu.c +2
-0@@ -1976,6 +1976,8 @@ static int domain_context_mapping_one(structdmar_domain *domain, pds =context_get_sm_pds (table); context->lo = (u64 )virt_to_phys (table->table) |context_pdts (pds); +if (!ecap_coherent (iommu->ecap)) +clflush_cache_range (table->table,sizeof (u64));
this is not required if cache is already flushed when the table is allocated.
Quoted 31 lines
drivers/iommu/intel/iommu.c +0
-0/* Setup the RID_PASID field: */ context_set_sm_rid2pasid (context,PASID_RID2PASID );drivers/iommu/intel/pasid.c +7
-0@@ -128,6 +128,9 @@ int intel_pasid_alloc_table(struct device *dev) pasid_table->max_pasid =1 << (order +PAGE_SHIFT +3 ); info->pasid_table = pasid_table; +if (!ecap_coherent (info->iommu->ecap)) +clflush_cache_range (pasid_table->table, size); +return 0 ; }@@ -215,6 +218,10 @@ static struct pasid_entry *intel_pasid_get_entry(struct device *dev, u32 pasid)free_pgtable_page (entries); goto retry; } +if (!ecap_coherent (info->iommu->ecap)) { +clflush_cache_range (entries,VTD_PAGE_SIZE ); +clflush_cache_range (&dir[dir_index].val,sizeof (*dir)); + } }return &entries[index];
--
2.25.1
Hi Kevin,
Quoted 11 lines · Kevin
drivers/iommu/intel/iommu.c static int domain_context_mapping_one(struct · line 1982
Quoted 8 lines
drivers/iommu/intel/iommu.c +2
-0@@ -1976,6 +1976,8 @@ static int domain_context_mapping_one(structdmar_domain *domain, pds =context_get_sm_pds (table); context->lo = (u64 )virt_to_phys (table->table) |context_pdts (pds); +if (!ecap_coherent (iommu->ecap)) +clflush_cache_range (table->table,sizeof (u64));this is not required if cache is already flushed when the table is allocated.
yes you're right.
Thanks,
Jacob