Respect VASIZE for address limits
Subsystems (2)
v3 cover Respect VASIZE for address limits
The AMD IOMMU driver was only using EFR[HATS] to derive the VA size, ignoring the IVINFO[VASIZE] limit. This could create domains that exceed the bounds advertised by hypervisor.
Patch 1 removes PT_FEAT_DYNAMIC_TOP from PT_FORCE_ENABLED_FEATURES to fix a spurious domain allocation failure when the VA size is small enough that dynamic top growth is unnecessary.
Patch 2 makes the driver respect IVINFO[VASIZE] when determining the addressable VA range.
Changes:
v3: - Remove specific value checking for vasize.
- Add patch for PT_FORCE_ENABLED_FEATURES.
v2: - Mark ivinfo_vasize_bits() as __init.
Ankit Soni (2):
iommu/amd: Remove PT_FEAT_DYNAMIC_TOP from PT_FORCE_ENABLED_FEATURES iommu/amd: Adhere to IVINFO[VASIZE] for address limits
drivers/iommu/amd/amd_iommu.h | 2 +- drivers/iommu/amd/amd_iommu_types.h | 1 + drivers/iommu/amd/init.c | 13 +++++++++---- drivers/iommu/amd/iommu.c | 3 +-- drivers/iommu/generic_pt/fmt/iommu_amdv1.c | 2 +- 5 files changed, 13 insertions(+), 8 deletions(-)
Signature · 1 line
2.43.0
PATCH v3 1/2 iommu/amd: Remove PT_FEAT_DYNAMIC_TOP from PT_FORCE_ENABLED_FEATURES
Dynamic top is used to grow the page table levels. However, if the VA size is small and the initial page table level already covers the entire address space, table growth is not required. In that case, the generic page table framework clears PT_FEAT_DYNAMIC_TOP, which in turn causes error -EOPNOTSUPP and leads to domain initialization failure.
Remove PT_FEAT_DYNAMIC_TOP from PT_FORCE_ENABLED_FEATURES, since during domain allocation AMD IOMMU driver explicitly sets PT_FEAT_DYNAMIC_TOP via cfg.common.features, this keeps normal runtime behavior unchanged.
--- drivers/iommu/generic_pt/fmt/iommu_amdv1.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/generic_pt/fmt/iommu_amdv1.c +1 -1
@@ -9,7 +9,7 @@ BIT(PT_FEAT_AMDV1_ENCRYPT_TABLES) | \ BIT(PT_FEAT_AMDV1_FORCE_COHERENCE))#define PT_FORCE_ENABLED_FEATURES \ - (BIT(PT_FEAT_DYNAMIC_TOP) | BIT(PT_FEAT_AMDV1_ENCRYPT_TABLES) | \ + (BIT(PT_FEAT_AMDV1_ENCRYPT_TABLES) | \ BIT(PT_FEAT_AMDV1_FORCE_COHERENCE))#include "iommu_template.h"
Signature · 1 line
2.43.0
PATCH v3 2/2 iommu/amd: Adhere to IVINFO[VASIZE] for address limits
In reply to Ankit SoniACPI IVRS IVHD’s IVINFO field reports the maximum virtual address size (VASIZE) supported by the IOMMU. The AMD IOMMU driver currently caps this with pagetable level reported by EFR[HATS] when configuring paging domains (hw_max_vasz_lg2). On systems where firmware or VM advertises smaller or different limits, the driver may over-advertise capabilities and create domains outside the hardware’s actual bounds.
--- drivers/iommu/amd/amd_iommu.h | 2 +- drivers/iommu/amd/amd_iommu_types.h | 1 + drivers/iommu/amd/init.c | 13 +++++++++---- drivers/iommu/amd/iommu.c | 3 +-- 4 files changed, 12 insertions(+), 7 deletions(-)
drivers/iommu/amd/amd_iommu.h +1 -1
@@ -41,7 +41,7 @@ int amd_iommu_enable_faulting(unsigned int cpu);extern int amd_iommu_guest_ir;extern enum protection_domain_mode amd_iommu_pgtable;extern int amd_iommu_gpt_level; -extern u8 amd_iommu_hpt_level; +extern u8 amd_iommu_hpt_vasize;extern unsigned long amd_iommu_pgsize_bitmap;extern bool amd_iommu_hatdis;
drivers/iommu/amd/amd_iommu_types.h +1 -0
@@ -395,6 +395,7 @@#define IOMMU_IVINFO_OFFSET 36#define IOMMU_IVINFO_EFRSUP BIT(0)#define IOMMU_IVINFO_DMA_REMAP BIT(1) +#define IOMMU_IVINFO_VASIZE GENMASK_ULL(21, 15)/* IOMMU Feature Reporting Field (for IVHD type 10h */ #define IOMMU_FEAT_GASUP_SHIFT 6
drivers/iommu/amd/init.c +9 -4
@@ -152,8 +152,8 @@ bool amd_iommu_dump;bool amd_iommu_irq_remap__read_mostly ;enum protection_domain_mode amd_iommu_pgtable =PD_MODE_V1 ; -/* Host page table level */ -u8 amd_iommu_hpt_level; +/* Virtual address size */ +u8 amd_iommu_hpt_vasize;/* Guest page table level */ int amd_iommu_gpt_level =PAGE_MODE_4_LEVEL ;
@@ -3188,7 +3188,7 @@ static int __init early_amd_iommu_init(void)struct acpi_table_header *ivrs_base;int ret;acpi_status status; -u8 efr_hats; +u8 efr_hats, max_vasize;if (!amd_iommu_detected)return -ENODEV ;
@@ -3218,6 +3218,10 @@ static int __init early_amd_iommu_init(void)ivinfo_init (ivrs_base); + max_vasize =FIELD_GET (IOMMU_IVINFO_VASIZE , amd_iommu_ivinfo); +if (!max_vasize) + max_vasize =64 ; + amd_iommu_target_ivhd_type =get_highest_supported_ivhd_type (ivrs_base);DUMP_printk ("Using IVHD type %#x\n" , amd_iommu_target_ivhd_type);
@@ -3240,7 +3244,8 @@ static int __init early_amd_iommu_init(void)* efr[HATS] bits specify the maximum host translation level * supported, with LEVEL 4 being initial max level. */ - amd_iommu_hpt_level = efr_hats +PAGE_MODE_4_LEVEL ; + amd_iommu_hpt_vasize = min_t(unsigned int, max_vasize, + (efr_hats +PAGE_MODE_4_LEVEL -1 ) *9 +21 ); }else {pr_warn_once (FW_BUG "Disable host address translation due to invalid translation level (%#x).\n" , efr_hats);
drivers/iommu/amd/iommu.c +1 -2
@@ -2712,8 +2712,7 @@ static struct iommu_domain *amd_iommu_domain_alloc_paging_v1(struct device *dev,else cfg.common.features |=BIT (PT_FEAT_FLUSH_RANGE ); - cfg.common.hw_max_vasz_lg2 = -min (64 , (amd_iommu_hpt_level -1 ) *9 +21 ); + cfg.common.hw_max_vasz_lg2 = amd_iommu_hpt_vasize; cfg.common.hw_max_oasz_lg2 =52 ; cfg.starting_level =2 ; domain->domain.ops = &amdv1_ops;
Signature · 1 line
2.43.0
Re: [PATCH v3 1/2] iommu/amd: Remove PT_FEAT_DYNAMIC_TOP from PT_FORCE_ENABLED_FEATURES
In reply to Ankit Soni+ Jason
Quoted 10 lines · Ankit Soni
Dynamic top is used to grow the page table levels. However, if the VA size is small and the initial page table level already covers the entire address space, table growth is not required. In that case, the generic page table framework clears
PT_FEAT_DYNAMIC_TOP, which in turn causes error-EOPNOTSUPPand leads to domain initialization failure.Remove
PT_FEAT_DYNAMIC_TOPfromPT_FORCE_ENABLED_FEATURES, since during domain allocation AMD IOMMU driver explicitly setsPT_FEAT_DYNAMIC_TOPvia cfg.common.features, this keeps normal runtime behavior unchanged.
Quoted 1 line
Signed-off-by Ankit Soni <[email protected]>
-Vasant
Quoted 17 lines
--- drivers/iommu/generic_pt/fmt/iommu_amdv1.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/iommu/generic_pt/fmt/iommu_amdv1.c +1
-1@@ -9,7 +9,7 @@ BIT(PT_FEAT_AMDV1_ENCRYPT_TABLES) | \ BIT(PT_FEAT_AMDV1_FORCE_COHERENCE))#define PT_FORCE_ENABLED_FEATURES \ - (BIT(PT_FEAT_DYNAMIC_TOP) | BIT(PT_FEAT_AMDV1_ENCRYPT_TABLES) | \ + (BIT(PT_FEAT_AMDV1_ENCRYPT_TABLES) | \ BIT(PT_FEAT_AMDV1_FORCE_COHERENCE))#include "iommu_template.h"
Re: [PATCH v3 2/2] iommu/amd: Adhere to IVINFO[VASIZE] for address limits
In reply to Ankit SoniQuoted 10 lines · Ankit Soni
ACPI IVRS IVHD’s IVINFO field reports the maximum virtual address size (VASIZE) supported by the IOMMU. The AMD IOMMU driver currently caps this with pagetable level reported by EFR[HATS] when configuring paging domains (
hw_max_vasz_lg2). On systems where firmware or VM advertises smaller or different limits, the driver may over-advertise capabilities and create domains outside the hardware’s actual bounds.Signed-off-by Ankit Soni <[email protected]>
-Vasant
Quoted 91 lines
--- drivers/iommu/amd/amd_iommu.h | 2 +- drivers/iommu/amd/amd_iommu_types.h | 1 + drivers/iommu/amd/init.c | 13 +++++++++---- drivers/iommu/amd/iommu.c | 3 +-- 4 files changed, 12 insertions(+), 7 deletions(-)drivers/iommu/amd/amd_iommu.h +1
-1@@ -41,7 +41,7 @@ int amd_iommu_enable_faulting(unsigned int cpu);extern int amd_iommu_guest_ir;extern enum protection_domain_mode amd_iommu_pgtable;extern int amd_iommu_gpt_level; -extern u8 amd_iommu_hpt_level; +extern u8 amd_iommu_hpt_vasize;extern unsigned long amd_iommu_pgsize_bitmap;extern bool amd_iommu_hatdis;drivers/iommu/amd/amd_iommu_types.h +1
-0@@ -395,6 +395,7 @@#define IOMMU_IVINFO_OFFSET 36#define IOMMU_IVINFO_EFRSUP BIT(0)#define IOMMU_IVINFO_DMA_REMAP BIT(1) +#define IOMMU_IVINFO_VASIZE GENMASK_ULL(21, 15)/* IOMMU Feature Reporting Field (for IVHD type 10h */ #define IOMMU_FEAT_GASUP_SHIFT 6drivers/iommu/amd/init.c +9
-4@@ -152,8 +152,8 @@ bool amd_iommu_dump;bool amd_iommu_irq_remap__read_mostly ;enum protection_domain_mode amd_iommu_pgtable =PD_MODE_V1 ; -/* Host page table level */ -u8 amd_iommu_hpt_level; +/* Virtual address size */ +u8 amd_iommu_hpt_vasize;/* Guest page table level */ int amd_iommu_gpt_level =PAGE_MODE_4_LEVEL ;@@ -3188,7 +3188,7 @@ static int __init early_amd_iommu_init(void)struct acpi_table_header *ivrs_base;int ret;acpi_status status; -u8 efr_hats; +u8 efr_hats, max_vasize;if (!amd_iommu_detected)return -ENODEV ;@@ -3218,6 +3218,10 @@ static int __init early_amd_iommu_init(void)ivinfo_init (ivrs_base); + max_vasize =FIELD_GET (IOMMU_IVINFO_VASIZE , amd_iommu_ivinfo); +if (!max_vasize) + max_vasize =64 ; + amd_iommu_target_ivhd_type =get_highest_supported_ivhd_type (ivrs_base);DUMP_printk ("Using IVHD type %#x\n" , amd_iommu_target_ivhd_type);@@ -3240,7 +3244,8 @@ static int __init early_amd_iommu_init(void)* efr[HATS] bits specify the maximum host translation level * supported, with LEVEL 4 being initial max level. */ - amd_iommu_hpt_level = efr_hats +PAGE_MODE_4_LEVEL ; + amd_iommu_hpt_vasize = min_t(unsigned int, max_vasize, + (efr_hats +PAGE_MODE_4_LEVEL -1 ) *9 +21 ); }else {pr_warn_once (FW_BUG "Disable host address translation due to invalid translation level (%#x).\n" , efr_hats);drivers/iommu/amd/iommu.c +1
-2@@ -2712,8 +2712,7 @@ static struct iommu_domain *amd_iommu_domain_alloc_paging_v1(struct device *dev,else cfg.common.features |=BIT (PT_FEAT_FLUSH_RANGE ); - cfg.common.hw_max_vasz_lg2 = -min (64 , (amd_iommu_hpt_level -1 ) *9 +21 ); + cfg.common.hw_max_vasz_lg2 = amd_iommu_hpt_vasize; cfg.common.hw_max_oasz_lg2 =52 ; cfg.starting_level =2 ; domain->domain.ops = &amdv1_ops;
Re: [PATCH v3 1/2] iommu/amd: Remove PT_FEAT_DYNAMIC_TOP from PT_FORCE_ENABLED_FEATURES
In reply to Vasant HegdeQuoted 10 lines · Vasant Hegde
+ Jason
Quoted 6 lines · Ankit Soni
Dynamic top is used to grow the page table levels. However, if the VA size is small and the initial page table level already covers the entire address space, table growth is not required. In that case, the generic page table framework clears
PT_FEAT_DYNAMIC_TOP, which in turn causes error-EOPNOTSUPPand leads to domain initialization failure.
Quoted 6 lines
Quoted 4 lines
Remove
PT_FEAT_DYNAMIC_TOPfromPT_FORCE_ENABLED_FEATURES, since during domain allocation AMD IOMMU driver explicitly setsPT_FEAT_DYNAMIC_TOPvia cfg.common.features, this keeps normal runtime behavior unchanged.Fixes 879ced2bab1 ("iommupt: Add the AMD IOMMU v1 page table format")
No.. There is no bug here.
I don't really understand why you'd want to do this, the driver still sets PT_FEAT_DYNAMIC_TOP, so what is the issue?
It shouldn't be removed from PT_FORCE_ENABLED_FEATURES, something else is wrong.
Jason
Re: [PATCH v3 1/2] iommu/amd: Remove PT_FEAT_DYNAMIC_TOP from PT_FORCE_ENABLED_FEATURES
Quoted 20 lines · Jason Gunthorpe
Quoted 10 lines · Vasant Hegde
+ Jason
Quoted 6 lines · Ankit Soni
Dynamic top is used to grow the page table levels. However, if the VA size is small and the initial page table level already covers the entire address space, table growth is not required. In that case, the generic page table framework clears
PT_FEAT_DYNAMIC_TOP, which in turn causes error-EOPNOTSUPPand leads to domain initialization failure.Quoted 6 lines
Quoted 4 lines
Remove
PT_FEAT_DYNAMIC_TOPfromPT_FORCE_ENABLED_FEATURES, since during domain allocation AMD IOMMU driver explicitly setsPT_FEAT_DYNAMIC_TOPvia cfg.common.features, this keeps normal runtime behavior unchanged.Fixes 879ced2bab1 ("iommupt: Add the AMD IOMMU v1 page table format")No.. There is no bug here.
Quoted 7 lines
I don't really understand why you'd want to do this, the driver still sets
PT_FEAT_DYNAMIC_TOP, so what is the issue?It shouldn't be removed from
PT_FORCE_ENABLED_FEATURES, something else is wrong.Jason
When hw_max_vasz_lg2 = 32 and starting_level = 2, top_range.max_vasz_lg2 comes back as 32 (clamped by min(32, 39)), so it matches common->max_vasz_lg2. The 'pt_init_common()' clears dynamic top because "the table can't grow beyond its current level." with next condition in this function, domain init will fail with "-EOPNOTSUPP".
Ankit
Re: [PATCH v3 1/2] iommu/amd: Remove PT_FEAT_DYNAMIC_TOP from PT_FORCE_ENABLED_FEATURES
Quoted 27 lines · Ankit Soni
Quoted 26 lines · Jason Gunthorpe
Quoted 10 lines · Vasant Hegde
+ Jason
Quoted 6 lines · Ankit Soni
Dynamic top is used to grow the page table levels. However, if the VA size is small and the initial page table level already covers the entire address space, table growth is not required. In that case, the generic page table framework clears
PT_FEAT_DYNAMIC_TOP, which in turn causes error-EOPNOTSUPPand leads to domain initialization failure.Quoted 6 lines
Quoted 4 lines
Remove
PT_FEAT_DYNAMIC_TOPfromPT_FORCE_ENABLED_FEATURES, since during domain allocation AMD IOMMU driver explicitly setsPT_FEAT_DYNAMIC_TOPvia cfg.common.features, this keeps normal runtime behavior unchanged.Fixes 879ced2bab1 ("iommupt: Add the AMD IOMMU v1 page table format")No.. There is no bug here.
I don't really understand why you'd want to do this, the driver still sets
PT_FEAT_DYNAMIC_TOP, so what is the issue?It shouldn't be removed from
PT_FORCE_ENABLED_FEATURES, something else is wrong.
Quoted 7 lines
Quoted 1 line
Jason
When
hw_max_vasz_lg2= 32 andstarting_level= 2, top_range.max_vasz_lg2 comes back as 32 (clamped bymin(32, 39)), so it matchescommon->max_vasz_lg2. The 'pt_init_common()' clears dynamic top because "the table can't grow beyond its current level." with next condition in this function, domain init will fail with "-EOPNOTSUPP".
Ok, that is much clearer, but this is not the way to fix it.
First it needs test coverage added
And probably we should remove that if but it should be checked closely.
Jason
Re: [PATCH v3 2/2] iommu/amd: Adhere to IVINFO[VASIZE] for address limits
In reply to Ankit SoniQuoted 7 lines · Ankit Soni
ACPI IVRS IVHD’s IVINFO field reports the maximum virtual address size (VASIZE) supported by the IOMMU. The AMD IOMMU driver currently caps this with pagetable level reported by EFR[HATS] when configuring paging domains (
hw_max_vasz_lg2). On systems where firmware or VM advertises smaller or different limits, the driver may over-advertise capabilities and create domains outside the hardware’s actual bounds.
Quoted 7 lines
Signed-off-by Ankit Soni <[email protected]>--- drivers/iommu/amd/amd_iommu.h | 2 +- drivers/iommu/amd/amd_iommu_types.h | 1 + drivers/iommu/amd/init.c | 13 +++++++++---- drivers/iommu/amd/iommu.c | 3 +-- 4 files changed, 12 insertions(+), 7 deletions(-)
Jason
Re: [PATCH v3 1/2] iommu/amd: Remove PT_FEAT_DYNAMIC_TOP from PT_FORCE_ENABLED_FEATURES
In reply to Jason GunthorpeQuoted 36 lines · Jason Gunthorpe
Quoted 27 lines · Ankit Soni
Quoted 26 lines · Jason Gunthorpe
Quoted 10 lines · Vasant Hegde
+ Jason
Quoted 6 lines · Ankit Soni
Dynamic top is used to grow the page table levels. However, if the VA size is small and the initial page table level already covers the entire address space, table growth is not required. In that case, the generic page table framework clears
PT_FEAT_DYNAMIC_TOP, which in turn causes error-EOPNOTSUPPand leads to domain initialization failure.Quoted 6 lines
Quoted 4 lines
Remove
PT_FEAT_DYNAMIC_TOPfromPT_FORCE_ENABLED_FEATURES, since during domain allocation AMD IOMMU driver explicitly setsPT_FEAT_DYNAMIC_TOPvia cfg.common.features, this keeps normal runtime behavior unchanged.Fixes 879ced2bab1 ("iommupt: Add the AMD IOMMU v1 page table format")No.. There is no bug here.
I don't really understand why you'd want to do this, the driver still sets
PT_FEAT_DYNAMIC_TOP, so what is the issue?It shouldn't be removed from
PT_FORCE_ENABLED_FEATURES, something else is wrong.Quoted 7 lines
Quoted 1 line
Jason
When
hw_max_vasz_lg2= 32 andstarting_level= 2, top_range.max_vasz_lg2 comes back as 32 (clamped bymin(32, 39)), so it matchescommon->max_vasz_lg2. The 'pt_init_common()' clears dynamic top because "the table can't grow beyond its current level." with next condition in this function, domain init will fail with "-EOPNOTSUPP".
Quoted 7 lines
Ok, that is much clearer, but this is not the way to fix it.
First it needs test coverage added
And probably we should remove that if but it should be checked closely.
Jason
Thanks for the heads‑up. I’ll add the test coverage and the other changes separately.
-Ankit