Skip to contentSkip to conversation

amd/iommu: remove hidden AMD inclusive mappings

PATCH In review Roger Pau Monne 5 participants 7 messages Sep 21, 2018 – Oct 2, 2018

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 amd/iommu: remove hidden AMD inclusive mappings

And just rely on arch_iommu_hwdom_init to setup the correct inclusive mappings as it's done for Intel.

AMD has code in amd_iommu_hwdom_init to setup inclusive mappings up to max_pdx, remove this since it's now a duplication of arch_iommu_hwdom_init. Note that AMD mapped every page with a valid mfn up to max_pdx, arch_iommu_hwdom_init will only do so for memory below 4GB, so this is a functional change for AMD.

Move the default setting of iommu_hwdom_{inclusive/reserved} to arch_iommu_hwdom_init since the defaults are now the same for both Intel and AMD.

---
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Brian Woods <[email protected]>
Cc: Kevin Tian <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Paul Durrant <[email protected]>

---
 xen/drivers/passthrough/amd/pci_amd_iommu.c | 39 ---------------------
 xen/drivers/passthrough/vtd/iommu.c         |  7 ----
 xen/drivers/passthrough/x86/iommu.c         |  8 ++++-
 3 files changed, 7 insertions(+), 47 deletions(-)
xen/drivers/passthrough/amd/pci_amd_iommu.c +0 -39
index 4a633ca940..821fe03df5 100644
@@ -250,50 +250,11 @@ static int amd_iommu_add_device(u8 devfn, struct pci_dev *pdev);
 
 static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)
 {
-    unsigned long i; 
     const struct amd_iommu *iommu;
 
-    /* Inclusive IOMMU mappings are disabled by default on AMD hardware. */
-    if ( iommu_hwdom_inclusive == -1 )
-        iommu_hwdom_inclusive = 0;
-    /* Reserved IOMMU mappings are disabled by default on AMD hardware. */
-    if ( iommu_hwdom_reserved == -1 )
-        iommu_hwdom_reserved = 0;
-
     if ( allocate_domain_resources(dom_iommu(d)) )
         BUG();
 
-    if ( !iommu_hwdom_passthrough && !need_iommu(d) )
-    {
-        int rc = 0;
-
-        /* Set up 1:1 page table for dom0 */
-        for ( i = 0; i < max_pdx; i++ )
-        {
-            unsigned long pfn = pdx_to_pfn(i);
-
-            /*
-             * XXX Should we really map all non-RAM (above 4G)? Minimally
-             * a pfn_valid() check would seem desirable here.
-             */
-            if ( mfn_valid(_mfn(pfn)) )
-            {
-                int ret = amd_iommu_map_page(d, pfn, pfn,
-                                             IOMMUF_readable|IOMMUF_writable);
-
-                if ( !rc )
-                    rc = ret;
-            }
-
-            if ( !(i & 0xfffff) )
-                process_pending_softirqs();
-        }
-
-        if ( rc )
-            AMD_IOMMU_DEBUG("d%d: IOMMU mapping failed: %d\n",
-                            d->domain_id, rc);
-    }
-
     for_each_amd_iommu ( iommu )
         if ( iomem_deny_access(d, PFN_DOWN(iommu->mmio_base_phys),
                                PFN_DOWN(iommu->mmio_base_phys +
xen/drivers/passthrough/vtd/iommu.c +0 -7
index bb422ec58c..cf8a80d7a1 100644
@@ -1304,13 +1304,6 @@ static void __hwdom_init intel_iommu_hwdom_init(struct domain *d)
 {
     struct acpi_drhd_unit *drhd;
 
-    /* Inclusive mappings are enabled by default on Intel hardware for PV. */
-    if ( iommu_hwdom_inclusive == -1 )
-        iommu_hwdom_inclusive = is_pv_domain(d);
-    /* Reserved IOMMU mappings are enabled by default on Intel hardware. */
-    if ( iommu_hwdom_reserved == -1 )
-        iommu_hwdom_reserved = 1;
-
     setup_hwdom_pci_devices(d, setup_hwdom_device);
     setup_hwdom_rmrr(d);
     /* Make sure workarounds are applied before enabling the IOMMU(s). */
xen/drivers/passthrough/x86/iommu.c +7 -1
index b7c8b5be41..2de8822c59 100644
@@ -210,7 +210,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
 
     BUG_ON(!is_hardware_domain(d));
 
-    ASSERT(iommu_hwdom_inclusive != -1 && iommu_hwdom_inclusive != -1);
+    /* Inclusive mappings are enabled by default for PV. */
+    if ( iommu_hwdom_inclusive == -1 )
+        iommu_hwdom_inclusive = is_pv_domain(d);
+    /* Reserved IOMMU mappings are enabled by default. */
+    if ( iommu_hwdom_reserved == -1 )
+        iommu_hwdom_reserved = 1;
+
     if ( iommu_hwdom_inclusive && !is_pv_domain(d) )
     {
         printk(XENLOG_WARNING
Signature · 7 lines
2.19.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Reviewed-by on v1 1/1
#1 lore raw
Quoted 19 lines

-----Original Message-----
From: Roger Pau Monne [mailto:[email protected]]
Sent: 21 September 2018 16:21
To: [email protected]
Cc: Roger Pau Monne <[email protected]>; Suravee Suthikulpanit <[email protected]>; Brian Woods <[email protected]>; Kevin Tian <[email protected]>; Jan Beulich <[email protected]>; Paul Durrant <[email protected]>
Subject: [PATCH] amd/iommu: remove hidden AMD inclusive mappings

And just rely on arch_iommu_hwdom_init to setup the correct inclusive mappings as it's done for Intel.

AMD has code in amd_iommu_hwdom_init to setup inclusive mappings up to max_pdx, remove this since it's now a duplication of arch_iommu_hwdom_init. Note that AMD mapped every page with a valid mfn up to max_pdx, arch_iommu_hwdom_init will only do so for memory below 4GB, so this is a functional change for AMD.

Quoted 4 lines

Move the default setting of iommu_hwdom_{inclusive/reserved} to arch_iommu_hwdom_init since the defaults are now the same for both Intel and AMD.

IMO the functional change and consequent alignment of the defaults is a good thing.

Quoted 2 lines
Quoted 114 lines

---
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Brian Woods <[email protected]>
Cc: Kevin Tian <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Paul Durrant <[email protected]>

---
 xen/drivers/passthrough/amd/pci_amd_iommu.c | 39 ---------------------
 xen/drivers/passthrough/vtd/iommu.c         |  7 ----
 xen/drivers/passthrough/x86/iommu.c         |  8 ++++-
 3 files changed, 7 insertions(+), 47 deletions(-)
xen/drivers/passthrough/amd/pci_amd_iommu.c +0 -25
index 4a633ca940..821fe03df5 100644
@@ -250,50 +250,11 @@ static int amd_iommu_add_device(u8 devfn, struct
 pci_dev *pdev);
 
 static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)
 {
-    unsigned long i;
     const struct amd_iommu *iommu;
 
-    /* Inclusive IOMMU mappings are disabled by default on AMD hardware.
 */
-    if ( iommu_hwdom_inclusive == -1 )
-        iommu_hwdom_inclusive = 0;
-    /* Reserved IOMMU mappings are disabled by default on AMD hardware.
 */
-    if ( iommu_hwdom_reserved == -1 )
-        iommu_hwdom_reserved = 0;
-
     if ( allocate_domain_resources(dom_iommu(d)) )
         BUG();
 
-    if ( !iommu_hwdom_passthrough && !need_iommu(d) )
-    {
-        int rc = 0;
-
-        /* Set up 1:1 page table for dom0 */
-        for ( i = 0; i < max_pdx; i++ )
-        {
-            unsigned long pfn = pdx_to_pfn(i);
-
-            /*
-             * XXX Should we really map all non-RAM (above 4G)? Minimally
-             * a pfn_valid() check would seem desirable here.
-             */
-            if ( mfn_valid(_mfn(pfn)) )
-            {
-                int ret = amd_iommu_map_page(d, pfn, pfn,
-

IOMMUF_readable|IOMMUF_writable);

xen/drivers/passthrough/amd/pci_amd_iommu.c +0 -14
-
-                if ( !rc )
-                    rc = ret;
-            }
-
-            if ( !(i & 0xfffff) )
-                process_pending_softirqs();
-        }
-
-        if ( rc )
-            AMD_IOMMU_DEBUG("d%d: IOMMU mapping failed: %d\n",
-                            d->domain_id, rc);
-    }
-
     for_each_amd_iommu ( iommu )
         if ( iomem_deny_access(d, PFN_DOWN(iommu->mmio_base_phys),
                                PFN_DOWN(iommu->mmio_base_phys +
xen/drivers/passthrough/vtd/iommu.c +0 -1
index bb422ec58c..cf8a80d7a1 100644
@@ -1304,13 +1304,6 @@ static void __hwdom_init
 intel_iommu_hwdom_init(struct domain *d)
 {
     struct acpi_drhd_unit *drhd;
 
-    /* Inclusive mappings are enabled by default on Intel hardware for

PV. */

xen/drivers/passthrough/vtd/iommu.c +0 -3
-    if ( iommu_hwdom_inclusive == -1 )
-        iommu_hwdom_inclusive = is_pv_domain(d);
-    /* Reserved IOMMU mappings are enabled by default on Intel hardware.

*/

xen/drivers/passthrough/vtd/iommu.c +0 -3
-    if ( iommu_hwdom_reserved == -1 )
-        iommu_hwdom_reserved = 1;
-
     setup_hwdom_pci_devices(d, setup_hwdom_device);
     setup_hwdom_rmrr(d);
     /* Make sure workarounds are applied before enabling the IOMMU(s). */
xen/drivers/passthrough/x86/iommu.c +7 -1
index b7c8b5be41..2de8822c59 100644
@@ -210,7 +210,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain
 *d)
 
     BUG_ON(!is_hardware_domain(d));
 
-    ASSERT(iommu_hwdom_inclusive != -1 && iommu_hwdom_inclusive != -1);
+    /* Inclusive mappings are enabled by default for PV. */
+    if ( iommu_hwdom_inclusive == -1 )
+        iommu_hwdom_inclusive = is_pv_domain(d);
+    /* Reserved IOMMU mappings are enabled by default. */
+    if ( iommu_hwdom_reserved == -1 )
+        iommu_hwdom_reserved = 1;
+
     if ( iommu_hwdom_inclusive && !is_pv_domain(d) )
     {
Quoted 3 lines

printk(XENLOG_WARNING
--
2.19.0

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Reviewed-by on v1 1/1
#2 lore raw
In reply to Roger Pau Monne
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 21.09.18 at 17:20, <[email protected]> wrote:

And just rely on arch_iommu_hwdom_init to setup the correct inclusive mappings as it's done for Intel.

AMD has code in amd_iommu_hwdom_init to setup inclusive mappings up to max_pdx, remove this since it's now a duplication of arch_iommu_hwdom_init. Note that AMD mapped every page with a valid mfn up to max_pdx, arch_iommu_hwdom_init will only do so for memory below 4GB, so this is a functional change for AMD.

Quoted 6 lines

Move the default setting of iommu_hwdom_{inclusive/reserved} to arch_iommu_hwdom_init since the defaults are now the same for both Intel and AMD.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Reviewed-by on v1 1/1
#3 lore raw
In reply to Roger Pau Monne
Quoted 13 lines

From: Roger Pau Monne [mailto:[email protected]]
Sent: Friday, September 21, 2018 11:21 PM

And just rely on arch_iommu_hwdom_init to setup the correct inclusive mappings as it's done for Intel.

AMD has code in amd_iommu_hwdom_init to setup inclusive mappings up to
max_pdx, remove this since it's now a duplication of arch_iommu_hwdom_init. Note that AMD mapped every page with a valid mfn up to max_pdx, arch_iommu_hwdom_init will only do so for memory below 4GB, so this is a functional change for AMD.

Quoted 6 lines

Move the default setting of iommu_hwdom_{inclusive/reserved} to arch_iommu_hwdom_init since the defaults are now the same for both Intel and AMD.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw

AMD Ping [PATCH] amd/iommu: remove hidden AMD inclusive mappings

In reply to Roger Pau Monne

Can I please get an Ack or otherwise from the AMD maintainers?

Thanks, Roger.

Quoted 122 lines · Roger Pau Monne

And just rely on arch_iommu_hwdom_init to setup the correct inclusive mappings as it's done for Intel.

AMD has code in amd_iommu_hwdom_init to setup inclusive mappings up to max_pdx, remove this since it's now a duplication of arch_iommu_hwdom_init. Note that AMD mapped every page with a valid mfn up to max_pdx, arch_iommu_hwdom_init will only do so for memory below 4GB, so this is a functional change for AMD.

Move the default setting of iommu_hwdom_{inclusive/reserved} to arch_iommu_hwdom_init since the defaults are now the same for both Intel and AMD.

---
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Brian Woods <[email protected]>
Cc: Kevin Tian <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Paul Durrant <[email protected]>

---
 xen/drivers/passthrough/amd/pci_amd_iommu.c | 39 ---------------------
 xen/drivers/passthrough/vtd/iommu.c         |  7 ----
 xen/drivers/passthrough/x86/iommu.c         |  8 ++++-
 3 files changed, 7 insertions(+), 47 deletions(-)
xen/drivers/passthrough/amd/pci_amd_iommu.c +0 -39
index 4a633ca940..821fe03df5 100644
@@ -250,50 +250,11 @@ static int amd_iommu_add_device(u8 devfn, struct pci_dev *pdev);
 
 static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)
 {
-    unsigned long i; 
     const struct amd_iommu *iommu;
 
-    /* Inclusive IOMMU mappings are disabled by default on AMD hardware. */
-    if ( iommu_hwdom_inclusive == -1 )
-        iommu_hwdom_inclusive = 0;
-    /* Reserved IOMMU mappings are disabled by default on AMD hardware. */
-    if ( iommu_hwdom_reserved == -1 )
-        iommu_hwdom_reserved = 0;
-
     if ( allocate_domain_resources(dom_iommu(d)) )
         BUG();
 
-    if ( !iommu_hwdom_passthrough && !need_iommu(d) )
-    {
-        int rc = 0;
-
-        /* Set up 1:1 page table for dom0 */
-        for ( i = 0; i < max_pdx; i++ )
-        {
-            unsigned long pfn = pdx_to_pfn(i);
-
-            /*
-             * XXX Should we really map all non-RAM (above 4G)? Minimally
-             * a pfn_valid() check would seem desirable here.
-             */
-            if ( mfn_valid(_mfn(pfn)) )
-            {
-                int ret = amd_iommu_map_page(d, pfn, pfn,
-                                             IOMMUF_readable|IOMMUF_writable);
-
-                if ( !rc )
-                    rc = ret;
-            }
-
-            if ( !(i & 0xfffff) )
-                process_pending_softirqs();
-        }
-
-        if ( rc )
-            AMD_IOMMU_DEBUG("d%d: IOMMU mapping failed: %d\n",
-                            d->domain_id, rc);
-    }
-
     for_each_amd_iommu ( iommu )
         if ( iomem_deny_access(d, PFN_DOWN(iommu->mmio_base_phys),
                                PFN_DOWN(iommu->mmio_base_phys +
xen/drivers/passthrough/vtd/iommu.c +0 -7
index bb422ec58c..cf8a80d7a1 100644
@@ -1304,13 +1304,6 @@ static void __hwdom_init intel_iommu_hwdom_init(struct domain *d)
 {
     struct acpi_drhd_unit *drhd;
 
-    /* Inclusive mappings are enabled by default on Intel hardware for PV. */
-    if ( iommu_hwdom_inclusive == -1 )
-        iommu_hwdom_inclusive = is_pv_domain(d);
-    /* Reserved IOMMU mappings are enabled by default on Intel hardware. */
-    if ( iommu_hwdom_reserved == -1 )
-        iommu_hwdom_reserved = 1;
-
     setup_hwdom_pci_devices(d, setup_hwdom_device);
     setup_hwdom_rmrr(d);
     /* Make sure workarounds are applied before enabling the IOMMU(s). */
xen/drivers/passthrough/x86/iommu.c +7 -1
index b7c8b5be41..2de8822c59 100644
@@ -210,7 +210,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
 
     BUG_ON(!is_hardware_domain(d));
 
-    ASSERT(iommu_hwdom_inclusive != -1 && iommu_hwdom_inclusive != -1);
+    /* Inclusive mappings are enabled by default for PV. */
+    if ( iommu_hwdom_inclusive == -1 )
+        iommu_hwdom_inclusive = is_pv_domain(d);
+    /* Reserved IOMMU mappings are enabled by default. */
+    if ( iommu_hwdom_reserved == -1 )
+        iommu_hwdom_reserved = 1;
+
     if ( iommu_hwdom_inclusive && !is_pv_domain(d) )
     {
         printk(XENLOG_WARNING
Signature · 1 line
2.19.0

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw
In reply to Roger Pau Monne

Roger,

I'm still catching up on the map-inclusive and map-reserved stuff. I have a couple questions below.

Quoted 8 lines · Roger Pau Monne

And just rely on arch_iommu_hwdom_init to setup the correct inclusive mappings as it's done for Intel.

AMD has code in amd_iommu_hwdom_init to setup inclusive mappings up to max_pdx, remove this since it's now a duplication of arch_iommu_hwdom_init. Note that AMD mapped every page with a valid mfn up to max_pdx, arch_iommu_hwdom_init will only do so for memory below 4GB, so this is a functional change for AMD.

Is there any reasons why limit to only below 4GB?

Quoted 25 lines

Move the default setting of iommu_hwdom_{inclusive/reserved} to arch_iommu_hwdom_init since the defaults are now the same for both Intel and AMD.

---
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Brian Woods <[email protected]>
Cc: Kevin Tian <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Paul Durrant <[email protected]>

---
  xen/drivers/passthrough/amd/pci_amd_iommu.c | 39 ---------------------
  xen/drivers/passthrough/vtd/iommu.c         |  7 ----
  xen/drivers/passthrough/x86/iommu.c         |  8 ++++-
  3 files changed, 7 insertions(+), 47 deletions(-)

...

xen/drivers/passthrough/x86/iommu.c +0 -0
index b7c8b5be41..2de8822c59 100644

xen/drivers/passthrough/x86/iommu.c void __hwdom_init arch_iommu_hwdom_init(struct domain *d) · line 213

Quoted 4 lines
xen/drivers/passthrough/x86/iommu.c +0 -1
@@ -210,7 +210,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
 
      BUG_ON(!is_hardware_domain(d));
 
-    ASSERT(iommu_hwdom_inclusive != -1 && iommu_hwdom_inclusive != -1);

Not sure if this was a typo. The logic looks strange. Anyhow, it is removed.

However, I notice in the xen/drivers/passthrough/iommu.c that
the parsing logic for the map-reserved option is setting the iommu_hwdom_inclusive instead of the iommu_hwdom_reserved. Is that intentional?

Also, what's the difference b/w the option map-inclusive parameter in drivers/passthrough/iommu.c:
parse_dom0_iommu_param() and the option iommu_inclusive_mapping in drivers/passthrough/vtd/x86/vtd.c?

xen/drivers/passthrough/x86/iommu.c void __hwdom_init arch_iommu_hwdom_init(struct domain *d) · line 222

Quoted 11 lines
xen/drivers/passthrough/x86/iommu.c +7 -0
+    /* Inclusive mappings are enabled by default for PV. */
+    if ( iommu_hwdom_inclusive == -1 )
+        iommu_hwdom_inclusive = is_pv_domain(d);
+    /* Reserved IOMMU mappings are enabled by default. */
+    if ( iommu_hwdom_reserved == -1 )
+        iommu_hwdom_reserved = 1;
+
      if ( iommu_hwdom_inclusive && !is_pv_domain(d) )
      {
          printk(XENLOG_WARNING

Thanks,
Suravee

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw
Quoted 9 lines · Suravee Suthikulpanit

Roger,

I'm still catching up on the map-inclusive and map-reserved stuff. I have a couple questions below.

Quoted 3 lines · Roger Pau Monne

And just rely on arch_iommu_hwdom_init to setup the correct inclusive mappings as it's done for Intel.

Quoted 7 lines
Quoted 5 lines

AMD has code in amd_iommu_hwdom_init to setup inclusive mappings up to max_pdx, remove this since it's now a duplication of arch_iommu_hwdom_init. Note that AMD mapped every page with a valid mfn up to max_pdx, arch_iommu_hwdom_init will only do so for memory below 4GB, so this is a functional change for AMD.

Is there any reasons why limit to only below 4GB?

This is what Intel IOMMU has always done, so this patch unifies the inclusive mappings code between Intel and AMD. Note that reserved regions above 4GB will still be mapped.

Quoted 20 lines
Quoted 20 lines

Move the default setting of iommu_hwdom_{inclusive/reserved} to arch_iommu_hwdom_init since the defaults are now the same for both Intel and AMD.

---
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Brian Woods <[email protected]>
Cc: Kevin Tian <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Paul Durrant <[email protected]>

---
  xen/drivers/passthrough/amd/pci_amd_iommu.c | 39 ---------------------
  xen/drivers/passthrough/vtd/iommu.c         |  7 ----
  xen/drivers/passthrough/x86/iommu.c         |  8 ++++-
  3 files changed, 7 insertions(+), 47 deletions(-)

...

Quoted 9 lines

xen/drivers/passthrough/x86/iommu.c void __hwdom_init arch_iommu_hwdom_init(struct domain *d) · line 211

Quoted 7 lines
xen/drivers/passthrough/x86/iommu.c +0 -1
index b7c8b5be41..2de8822c59 100644
@@ -210,7 +210,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
      BUG_ON(!is_hardware_domain(d));
-    ASSERT(iommu_hwdom_inclusive != -1 && iommu_hwdom_inclusive != -1);

Not sure if this was a typo. The logic looks strange. Anyhow, it is removed.

Oh right, this should check inclusive and reserved.

Quoted 3 lines

However, I notice in the xen/drivers/passthrough/iommu.c that
the parsing logic for the map-reserved option is setting the iommu_hwdom_inclusive instead of the iommu_hwdom_reserved. Is that intentional?

It's the same typo as above, will send a patch to fix it.

Quoted 2 lines

Also, what's the difference b/w the option map-inclusive parameter in drivers/passthrough/iommu.c:
parse_dom0_iommu_param() and the option iommu_inclusive_mapping in drivers/passthrough/vtd/x86/vtd.c?

The old iommu_inclusive_mapping option is keep for compatibility reasons, but the documentation has been updated to mark that option as deprecated.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel