Skip to contentSkip to conversation

paravirtual IOMMU pre-requisites and clean-up

PATCH Superseded Paul Durrant 2 participants 9 messages Sep 18, 2018

v10 of 14 Superseded Latest is v14 →

lore
Share & tools
lore
#0 lore raw

v10 cover paravirtual IOMMU pre-requisites and clean-up

This series contains pre-requisites and clean-up needed for paravirtual IOMMU support.

I have separated these patches to avoid further delaying their application whilst I re-work the implementation of paravirtual IOMMU after review of v6 of the series. Several of them already have all necessary acks.

Paul Durrant (7):
iommu: introduce the concept of DFN...
iommu: make use of type-safe DFN and MFN in exported functions
iommu: push use of type-safe DFN and MFN into iommu_ops
iommu: don't domain_crash() inside iommu_map/unmap_page()
memory: add check_get_page_from_gfn() as a wrapper...
vtd: add missing check for shared EPT...
vtd: add lookup_page method to iommu_ops

 xen/arch/arm/p2m.c                            |  7 ++-
 xen/arch/x86/hvm/emulate.c                    | 25 ++++----
 xen/arch/x86/hvm/hvm.c                        | 14 +----
 xen/arch/x86/mm.c                             | 13 ++--
 xen/arch/x86/mm/p2m-ept.c                     | 13 +++-
 xen/arch/x86/mm/p2m-pt.c                      | 48 ++++++++------
 xen/arch/x86/mm/p2m.c                         | 32 +++++++---
 xen/arch/x86/x86_64/mm.c                      |  5 +-
 xen/common/grant_table.c                      | 48 +++++++-------
 xen/common/memory.c                           | 56 +++++++++++++----
 xen/drivers/passthrough/amd/iommu_cmd.c       | 18 +++---
 xen/drivers/passthrough/amd/iommu_map.c       | 88 ++++++++++++++------------
 xen/drivers/passthrough/amd/pci_amd_iommu.c   |  4 +-
 xen/drivers/passthrough/arm/smmu.c            | 20 +++---
 xen/drivers/passthrough/iommu.c               | 53 ++++++++--------
 xen/drivers/passthrough/vtd/iommu.c           | 91 ++++++++++++++++++++-------
 xen/drivers/passthrough/vtd/iommu.h           |  3 +
 xen/drivers/passthrough/vtd/x86/vtd.c         |  1 -
 xen/drivers/passthrough/x86/iommu.c           |  9 ++-
 xen/include/asm-arm/p2m.h                     |  4 ++
 xen/include/asm-x86/hvm/svm/amd-iommu-proto.h |  8 +--
 xen/include/asm-x86/iommu.h                   | 12 ++++
 xen/include/asm-x86/p2m.h                     |  3 +
 xen/include/xen/iommu.h                       | 51 ++++++++++++---
 xen/include/xen/mm.h                          |  5 ++
 xen/include/xen/sched.h                       |  5 ++
 26 files changed, 414 insertions(+), 222 deletions(-)

---
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Jun Nakajima <[email protected]>
Cc: Kevin Tian <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: Wei Liu <[email protected]>

Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH v10 1/7 iommu: introduce the concept of DFN...

...meaning 'device DMA frame number' i.e. a frame number mapped in the IOMMU (rather than the MMU) and hence used for DMA address translation.

This patch is a largely cosmetic change that substitutes the terms 'gfn' and 'gaddr' for 'dfn' and 'daddr' in all the places where the frame number or address relate to a device rather than the CPU.

The parts that are not purely cosmetic are:

- the introduction of a type-safe declaration of dfn_t and definition of INVALID_DFN to make the substitution of gfn_x(INVALID_GFN) mechanical.
- the introduction of __dfn_to_daddr and __daddr_to_dfn (and type-safe variants without the leading __) with some use of the former.

Subsequent patches will convert code to make use of type-safe DFNs.

---
Cc: Wei Liu <[email protected]>
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Julien Grall <[email protected]>

v9:
- Re-word comment in mm.h.
- Move definitions relating to daddr into asm-x86/iommu.h since these are not used by any ARM IOMMU implementation.
- Fix __daddr_to_dfn() to properly parenthesize and remove cast.

v8:
- Correct definition of INVALID_DFN.
- Don't use _AC in definition of IOMMU_PAGE_SIZE.

v7:
- Re-name BFN -> DFN as requested by Jan.
- Dropped Wei's R-b because of name change.

v6:
- Dropped changes to 'mfn' section in xen/mm.h as suggested by Kevin.

v3:
- Get rid of intermediate 'frame' variables again.

v2:
- Addressed comments from Jan.

---
 xen/drivers/passthrough/amd/iommu_cmd.c     | 18 +++----
 xen/drivers/passthrough/amd/iommu_map.c     | 78 ++++++++++++++---------------
 xen/drivers/passthrough/amd/pci_amd_iommu.c |  2 +-
 xen/drivers/passthrough/arm/smmu.c          | 16 +++---
 xen/drivers/passthrough/iommu.c             | 28 +++++------
 xen/drivers/passthrough/vtd/iommu.c         | 30 +++++------
 xen/include/asm-x86/iommu.h                 | 12 +++++
 xen/include/xen/iommu.h                     | 26 +++++++---
 xen/include/xen/mm.h                        |  5 ++
 9 files changed, 123 insertions(+), 92 deletions(-)
xen/drivers/passthrough/amd/iommu_cmd.c +9 -9
index 08247fa354..d4d071e53e 100644
@@ -284,7 +284,7 @@ void invalidate_iommu_all(struct amd_iommu *iommu)
 }
 
 void amd_iommu_flush_iotlb(u8 devfn, const struct pci_dev *pdev,
-                           uint64_t gaddr, unsigned int order)
+                           daddr_t daddr, unsigned int order)
 {
     unsigned long flags;
     struct amd_iommu *iommu;
@@ -315,12 +315,12 @@ void amd_iommu_flush_iotlb(u8 devfn, const struct pci_dev *pdev,
 
     /* send INVALIDATE_IOTLB_PAGES command */
     spin_lock_irqsave(&iommu->lock, flags);
-    invalidate_iotlb_pages(iommu, maxpend, 0, queueid, gaddr, req_id, order);
+    invalidate_iotlb_pages(iommu, maxpend, 0, queueid, daddr, req_id, order);
     flush_command_buffer(iommu);
     spin_unlock_irqrestore(&iommu->lock, flags);
 }
 
-static void amd_iommu_flush_all_iotlbs(struct domain *d, uint64_t gaddr,
+static void amd_iommu_flush_all_iotlbs(struct domain *d, daddr_t daddr,
                                        unsigned int order)
 {
     struct pci_dev *pdev;
@@ -333,7 +333,7 @@ static void amd_iommu_flush_all_iotlbs(struct domain *d, uint64_t gaddr,
         u8 devfn = pdev->devfn;
 
         do {
-            amd_iommu_flush_iotlb(devfn, pdev, gaddr, order);
+            amd_iommu_flush_iotlb(devfn, pdev, daddr, order);
             devfn += pdev->phantom_stride;
         } while ( devfn != pdev->devfn &&
                   PCI_SLOT(devfn) == PCI_SLOT(pdev->devfn) );
@@ -342,7 +342,7 @@ static void amd_iommu_flush_all_iotlbs(struct domain *d, uint64_t gaddr,
 
 /* Flush iommu cache after p2m changes. */
 static void _amd_iommu_flush_pages(struct domain *d,
-                                   uint64_t gaddr, unsigned int order)
+                                   daddr_t daddr, unsigned int order)
 {
     unsigned long flags;
     struct amd_iommu *iommu;
@@ -352,13 +352,13 @@ static void _amd_iommu_flush_pages(struct domain *d,
     for_each_amd_iommu ( iommu )
     {
         spin_lock_irqsave(&iommu->lock, flags);
-        invalidate_iommu_pages(iommu, gaddr, dom_id, order);
+        invalidate_iommu_pages(iommu, daddr, dom_id, order);
         flush_command_buffer(iommu);
         spin_unlock_irqrestore(&iommu->lock, flags);
     }
 
     if ( ats_enabled )
-        amd_iommu_flush_all_iotlbs(d, gaddr, order);
+        amd_iommu_flush_all_iotlbs(d, daddr, order);
 }
 
 void amd_iommu_flush_all_pages(struct domain *d)
@@ -367,9 +367,9 @@ void amd_iommu_flush_all_pages(struct domain *d)
 }
 
 void amd_iommu_flush_pages(struct domain *d,
-                           unsigned long gfn, unsigned int order)
+                           unsigned long dfn, unsigned int order)
 {
-    _amd_iommu_flush_pages(d, (uint64_t) gfn << PAGE_SHIFT, order);
+    _amd_iommu_flush_pages(d, __dfn_to_daddr(dfn), order);
 }
 
 void amd_iommu_flush_device(struct amd_iommu *iommu, uint16_t bdf)
xen/drivers/passthrough/amd/iommu_map.c +39 -39

259 lines not on this page · raw message

xen/drivers/passthrough/amd/pci_amd_iommu.c +1 -1
index 330f9ce386..bcc47eb96c 100644
@@ -585,7 +585,7 @@ static void amd_dump_p2m_table_level(struct page_info* pg, int level,
                 maddr_to_page(next_table_maddr), next_level,
                 address, indent + 1);
         else
-            printk("%*sgfn: %08lx  mfn: %08lx\n",
+            printk("%*sdfn: %08lx  mfn: %08lx\n",
                    indent, "",
                    (unsigned long)PFN_DOWN(address),
                    (unsigned long)PFN_DOWN(next_table_maddr));
xen/drivers/passthrough/arm/smmu.c +8 -8
index 43ece42a50..1ff09bbb02 100644
@@ -2551,7 +2551,7 @@ static int __must_check arm_smmu_iotlb_flush_all(struct domain *d)
 }
 
 static int __must_check arm_smmu_iotlb_flush(struct domain *d,
-                                             unsigned long gfn,
+                                             unsigned long dfn,
                                              unsigned int page_count)
 {
 	/* ARM SMMU v1 doesn't have flush by VMA and VMID */
@@ -2746,7 +2746,7 @@ static void arm_smmu_iommu_domain_teardown(struct domain *d)
 	xfree(xen_domain);
 }
 
-static int __must_check arm_smmu_map_page(struct domain *d, unsigned long gfn,
+static int __must_check arm_smmu_map_page(struct domain *d, unsigned long dfn,
 			unsigned long mfn, unsigned int flags)
 {
 	p2m_type_t t;
@@ -2757,10 +2757,10 @@ static int __must_check arm_smmu_map_page(struct domain *d, unsigned long gfn,
 	 * protected by an IOMMU, Xen needs to add a 1:1 mapping in the domain
 	 * p2m to allow DMA request to work.
 	 * This is only valid when the domain is directed mapped. Hence this
-	 * function should only be used by gnttab code with gfn == mfn.
+	 * function should only be used by gnttab code with gfn == mfn == dfn.
 	 */
 	BUG_ON(!is_domain_direct_mapped(d));
-	BUG_ON(mfn != gfn);
+	BUG_ON(mfn != dfn);
 
 	/* We only support readable and writable flags */
 	if (!(flags & (IOMMUF_readable | IOMMUF_writable)))
@@ -2772,19 +2772,19 @@ static int __must_check arm_smmu_map_page(struct domain *d, unsigned long gfn,
 	 * The function guest_physmap_add_entry replaces the current mapping
 	 * if there is already one...
 	 */
-	return guest_physmap_add_entry(d, _gfn(gfn), _mfn(mfn), 0, t);
+	return guest_physmap_add_entry(d, _gfn(dfn), _mfn(dfn), 0, t);
 }
 
-static int __must_check arm_smmu_unmap_page(struct domain *d, unsigned long gfn)
+static int __must_check arm_smmu_unmap_page(struct domain *d, unsigned long dfn)
 {
 	/*
 	 * This function should only be used by gnttab code when the domain
-	 * is direct mapped
+	 * is direct mapped (i.e. gfn == mfn == dfn).
 	 */
 	if ( !is_domain_direct_mapped(d) )
 		return -EINVAL;
 
-	return guest_physmap_remove_page(d, _gfn(gfn), _mfn(gfn), 0);
+	return guest_physmap_remove_page(d, _gfn(dfn), _mfn(dfn), 0);
 }
 
 static const struct iommu_ops arm_smmu_iommu_ops = {
xen/drivers/passthrough/iommu.c +14 -14
index ee3f523fdf..ca8262be7c 100644
@@ -215,7 +215,7 @@ void __hwdom_init iommu_hwdom_init(struct domain *d)
         page_list_for_each ( page, &d->page_list )
         {
             unsigned long mfn = mfn_x(page_to_mfn(page));
-            unsigned long gfn = mfn_to_gmfn(d, mfn);
+            unsigned long dfn = mfn_to_gmfn(d, mfn);
             unsigned int mapping = IOMMUF_readable;
             int ret;
 
@@ -224,7 +224,7 @@ void __hwdom_init iommu_hwdom_init(struct domain *d)
                   == PGT_writable_page) )
                 mapping |= IOMMUF_writable;
 
-            ret = hd->platform_ops->map_page(d, gfn, mfn, mapping);
+            ret = hd->platform_ops->map_page(d, dfn, mfn, mapping);
             if ( !rc )
                 rc = ret;
 
@@ -295,7 +295,7 @@ void iommu_domain_destroy(struct domain *d)
     arch_iommu_domain_destroy(d);
 }
 
-int iommu_map_page(struct domain *d, unsigned long gfn, unsigned long mfn,
+int iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,
                    unsigned int flags)
 {
     const struct domain_iommu *hd = dom_iommu(d);
@@ -304,13 +304,13 @@ int iommu_map_page(struct domain *d, unsigned long gfn, unsigned long mfn,
     if ( !iommu_enabled || !hd->platform_ops )
         return 0;
 
-    rc = hd->platform_ops->map_page(d, gfn, mfn, flags);
+    rc = hd->platform_ops->map_page(d, dfn, mfn, flags);
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
             printk(XENLOG_ERR
-                   "d%d: IOMMU mapping gfn %#lx to mfn %#lx failed: %d\n",
-                   d->domain_id, gfn, mfn, rc);
+                   "d%d: IOMMU mapping dfn %#lx to mfn %#lx failed: %d\n",
+                   d->domain_id, dfn, mfn, rc);
 
         if ( !is_hardware_domain(d) )
             domain_crash(d);
@@ -319,7 +319,7 @@ int iommu_map_page(struct domain *d, unsigned long gfn, unsigned long mfn,
     return rc;
 }
 
-int iommu_unmap_page(struct domain *d, unsigned long gfn)
+int iommu_unmap_page(struct domain *d, unsigned long dfn)
 {
     const struct domain_iommu *hd = dom_iommu(d);
     int rc;
@@ -327,13 +327,13 @@ int iommu_unmap_page(struct domain *d, unsigned long gfn)
     if ( !iommu_enabled || !hd->platform_ops )
         return 0;
 
-    rc = hd->platform_ops->unmap_page(d, gfn);
+    rc = hd->platform_ops->unmap_page(d, dfn);
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
             printk(XENLOG_ERR
-                   "d%d: IOMMU unmapping gfn %#lx failed: %d\n",
-                   d->domain_id, gfn, rc);
+                   "d%d: IOMMU unmapping dfn %#lx failed: %d\n",
+                   d->domain_id, dfn, rc);
 
         if ( !is_hardware_domain(d) )
             domain_crash(d);
@@ -359,7 +359,7 @@ static void iommu_free_pagetables(unsigned long unused)
                             cpumask_cycle(smp_processor_id(), &cpu_online_map));
 }
 
-int iommu_iotlb_flush(struct domain *d, unsigned long gfn,
+int iommu_iotlb_flush(struct domain *d, unsigned long dfn,
                       unsigned int page_count)
 {
     const struct domain_iommu *hd = dom_iommu(d);
@@ -368,13 +368,13 @@ int iommu_iotlb_flush(struct domain *d, unsigned long gfn,
     if ( !iommu_enabled || !hd->platform_ops || !hd->platform_ops->iotlb_flush )
         return 0;
 
-    rc = hd->platform_ops->iotlb_flush(d, gfn, page_count);
+    rc = hd->platform_ops->iotlb_flush(d, dfn, page_count);
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
             printk(XENLOG_ERR
-                   "d%d: IOMMU IOTLB flush failed: %d, gfn %#lx, page count %u\n",
-                   d->domain_id, rc, gfn, page_count);
+                   "d%d: IOMMU IOTLB flush failed: %d, dfn %#lx, page count %u\n",
+                   d->domain_id, rc, dfn, page_count);
 
         if ( !is_hardware_domain(d) )
             domain_crash(d);
xen/drivers/passthrough/vtd/iommu.c +15 -15
index adc70f205a..88720d4406 100644
@@ -585,7 +585,7 @@ static int __must_check iommu_flush_all(void)
 }
 
 static int __must_check iommu_flush_iotlb(struct domain *d,
-                                          unsigned long gfn,
+                                          unsigned long dfn,
                                           bool_t dma_old_pte_present,
                                           unsigned int page_count)
 {
@@ -612,12 +612,12 @@ static int __must_check iommu_flush_iotlb(struct domain *d,
         if ( iommu_domid == -1 )
             continue;
 
-        if ( page_count != 1 || gfn == gfn_x(INVALID_GFN) )
+        if ( page_count != 1 || dfn == dfn_x(INVALID_DFN) )
             rc = iommu_flush_iotlb_dsi(iommu, iommu_domid,
                                        0, flush_dev_iotlb);
         else
             rc = iommu_flush_iotlb_psi(iommu, iommu_domid,
-                                       (paddr_t)gfn << PAGE_SHIFT_4K,
+                                       __dfn_to_daddr(dfn),
                                        PAGE_ORDER_4K,
                                        !dma_old_pte_present,
                                        flush_dev_iotlb);
@@ -633,15 +633,15 @@ static int __must_check iommu_flush_iotlb(struct domain *d,
 }
 
 static int __must_check iommu_flush_iotlb_pages(struct domain *d,
-                                                unsigned long gfn,
+                                                unsigned long dfn,
                                                 unsigned int page_count)
 {
-    return iommu_flush_iotlb(d, gfn, 1, page_count);
+    return iommu_flush_iotlb(d, dfn, 1, page_count);
 }
 
 static int __must_check iommu_flush_iotlb_all(struct domain *d)
 {
-    return iommu_flush_iotlb(d, gfn_x(INVALID_GFN), 0, 0);
+    return iommu_flush_iotlb(d, dfn_x(INVALID_DFN), 0, 0);
 }
 
 /* clear one page's page table */
@@ -1768,7 +1768,7 @@ static void iommu_domain_teardown(struct domain *d)
 }
 
 static int __must_check intel_iommu_map_page(struct domain *d,
-                                             unsigned long gfn,
+                                             unsigned long dfn,
                                              unsigned long mfn,
                                              unsigned int flags)
 {
@@ -1787,14 +1787,14 @@ static int __must_check intel_iommu_map_page(struct domain *d,
 
     spin_lock(&hd->arch.mapping_lock);
 
-    pg_maddr = addr_to_dma_page_maddr(d, (paddr_t)gfn << PAGE_SHIFT_4K, 1);
+    pg_maddr = addr_to_dma_page_maddr(d, __dfn_to_daddr(dfn), 1);
     if ( pg_maddr == 0 )
     {
         spin_unlock(&hd->arch.mapping_lock);
         return -ENOMEM;
     }
     page = (struct dma_pte *)map_vtd_domain_page(pg_maddr);
-    pte = page + (gfn & LEVEL_MASK);
+    pte = page + (dfn & LEVEL_MASK);
     old = *pte;
     dma_set_pte_addr(new, (paddr_t)mfn << PAGE_SHIFT_4K);
     dma_set_pte_prot(new,
@@ -1818,22 +1818,22 @@ static int __must_check intel_iommu_map_page(struct domain *d,
     unmap_vtd_domain_page(page);
 
     if ( !this_cpu(iommu_dont_flush_iotlb) )
-        rc = iommu_flush_iotlb(d, gfn, dma_pte_present(old), 1);
+        rc = iommu_flush_iotlb(d, dfn, dma_pte_present(old), 1);
 
     return rc;
 }
 
 static int __must_check intel_iommu_unmap_page(struct domain *d,
-                                               unsigned long gfn)
+                                               unsigned long dfn)
 {
     /* Do nothing if hardware domain and iommu supports pass thru. */
     if ( iommu_hwdom_passthrough && is_hardware_domain(d) )
         return 0;
 
-    return dma_pte_clear_one(d, (paddr_t)gfn << PAGE_SHIFT_4K);
+    return dma_pte_clear_one(d, __dfn_to_daddr(dfn));
 }
 
-int iommu_pte_flush(struct domain *d, u64 gfn, u64 *pte,
+int iommu_pte_flush(struct domain *d, uint64_t dfn, uint64_t *pte,
                     int order, int present)
 {
     struct acpi_drhd_unit *drhd;
@@ -1857,7 +1857,7 @@ int iommu_pte_flush(struct domain *d, u64 gfn, u64 *pte,
             continue;
 
         rc = iommu_flush_iotlb_psi(iommu, iommu_domid,
-                                   (paddr_t)gfn << PAGE_SHIFT_4K,
+                                   __dfn_to_daddr(dfn),
                                    order, !present, flush_dev_iotlb);
         if ( rc > 0 )
         {
@@ -2627,7 +2627,7 @@ static void vtd_dump_p2m_table_level(paddr_t pt_maddr, int level, paddr_t gpa,
             vtd_dump_p2m_table_level(dma_pte_addr(*pte), next_level, 
                                      address, indent + 1);
         else
-            printk("%*sgfn: %08lx mfn: %08lx\n",
+            printk("%*sdfn: %08lx mfn: %08lx\n",
                    indent, "",
                    (unsigned long)(address >> PAGE_SHIFT_4K),
                    (unsigned long)(dma_pte_addr(*pte) >> PAGE_SHIFT_4K));
xen/include/asm-x86/iommu.h +12 -0
index 14ad0489a6..0ed4a9e86d 100644
@@ -30,6 +30,18 @@ struct g2m_ioport {
     unsigned int np;
 };
 
+#define IOMMU_PAGE_SHIFT 12
+#define IOMMU_PAGE_SIZE  (1 << IOMMU_PAGE_SHIFT)
+#define IOMMU_PAGE_MASK  (~(IOMMU_PAGE_SIZE - 1))
+
+typedef uint64_t daddr_t;
+
+#define __dfn_to_daddr(dfn) ((daddr_t)(dfn) << IOMMU_PAGE_SHIFT)
+#define __daddr_to_dfn(daddr) ((daddr) >> IOMMU_PAGE_SHIFT)
+
+#define dfn_to_daddr(dfn) __dfn_to_daddr(dfn_x(dfn))
+#define daddr_to_dfn(daddr) _dfn(__daddr_to_dfn(daddr))
+
 struct arch_iommu
 {
     u64 pgd_maddr;                 /* io page directory machine address */
xen/include/xen/iommu.h +20 -6
index 57c4e81ec6..290e0aada6 100644
@@ -23,11 +23,25 @@
 #include <xen/page-defs.h>
 #include <xen/spinlock.h>
 #include <xen/pci.h>
+#include <xen/typesafe.h>
 #include <public/hvm/ioreq.h>
 #include <public/domctl.h>
 #include <asm/device.h>
 #include <asm/iommu.h>
 
+TYPE_SAFE(uint64_t, dfn);
+#define PRI_dfn     PRIx64
+#define INVALID_DFN _dfn(~0ULL)
+
+#ifndef dfn_t
+#define dfn_t /* Grep fodder: dfn_t, _dfn() and dfn_x() are defined above */
+#define _dfn
+#define dfn_x
+#undef dfn_t
+#undef _dfn
+#undef dfn_x
+#endif
+
 extern bool_t iommu_enable, iommu_enabled;
 extern bool_t force_iommu, iommu_verbose;
 extern bool_t iommu_workaround_bios_bug, iommu_igfx;
@@ -64,9 +78,9 @@ void iommu_teardown(struct domain *d);
 #define IOMMUF_readable  (1u<<_IOMMUF_readable)
 #define _IOMMUF_writable 1
 #define IOMMUF_writable  (1u<<_IOMMUF_writable)
-int __must_check iommu_map_page(struct domain *d, unsigned long gfn,
+int __must_check iommu_map_page(struct domain *d, unsigned long dfn,
                                 unsigned long mfn, unsigned int flags);
-int __must_check iommu_unmap_page(struct domain *d, unsigned long gfn);
+int __must_check iommu_unmap_page(struct domain *d, unsigned long dfn);
 
 enum iommu_feature
 {
@@ -154,9 +168,9 @@ struct iommu_ops {
 #endif /* HAS_PCI */
 
     void (*teardown)(struct domain *d);
-    int __must_check (*map_page)(struct domain *d, unsigned long gfn,
+    int __must_check (*map_page)(struct domain *d, unsigned long dfn,
                                  unsigned long mfn, unsigned int flags);
-    int __must_check (*unmap_page)(struct domain *d, unsigned long gfn);
+    int __must_check (*unmap_page)(struct domain *d, unsigned long dfn);
     void (*free_page_table)(struct page_info *);
 #ifdef CONFIG_X86
     void (*update_ire_from_apic)(unsigned int apic, unsigned int reg, unsigned int value);
@@ -167,7 +181,7 @@ struct iommu_ops {
     void (*resume)(void);
     void (*share_p2m)(struct domain *d);
     void (*crash_shutdown)(void);
-    int __must_check (*iotlb_flush)(struct domain *d, unsigned long gfn,
+    int __must_check (*iotlb_flush)(struct domain *d, unsigned long dfn,
                                     unsigned int page_count);
     int __must_check (*iotlb_flush_all)(struct domain *d);
     int (*get_reserved_device_memory)(iommu_grdm_t *, void *);
@@ -189,7 +203,7 @@ int iommu_do_pci_domctl(struct xen_domctl *, struct domain *d,
 int iommu_do_domctl(struct xen_domctl *, struct domain *d,
                     XEN_GUEST_HANDLE_PARAM(xen_domctl_t));
 
-int __must_check iommu_iotlb_flush(struct domain *d, unsigned long gfn,
+int __must_check iommu_iotlb_flush(struct domain *d, unsigned long dfn,
                                    unsigned int page_count);
 int __must_check iommu_iotlb_flush_all(struct domain *d);
 
xen/include/xen/mm.h +5 -0
index b3d46ab56b..720aa49b05 100644
@@ -26,6 +26,11 @@
  *   A linear idea of a guest physical address space. For an auto-translated
  *   guest, pfn == gfn while for a non-translated guest, pfn != gfn.
  *
+ * dfn: Device DMA Frame Number (definitions in include/xen/iommu.h)
+ *   The linear frame numbers of device DMA address space. All initiators for
+ *   (i.e. all devices assigned to) a guest share a single DMA address space
+ *   and, by default, Xen will ensure dfn == pfn.
+ *
  * WARNING: Some of these terms have changed over time while others have been
  * used inconsistently, meaning that a lot of existing code does not match the
  * definitions above.  New code should use these terms as described here, and
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#2 lore raw

PATCH v10 2/7 iommu: make use of type-safe DFN and MFN in exported functions

In reply to Paul Durrant

This patch modifies the declaration of the entry points to the IOMMU sub-system to use dfn_t and mfn_t in place of unsigned long. A subsequent patch will similarly modify the methods in the iommu_ops structure.

---
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: Jun Nakajima <[email protected]>
Cc: George Dunlap <[email protected]>

v9:
- Re-base.

v7:
- Re-base and re-name BFN -> DFN.
- Added Jan's A-b since re-naming was purely mechanical.

v6:
- Re-base.

v3:
- Removed most of the uses of an intermediate 'frame' variable.

v2:
- Addressed comments from Jan.
- Use intermediate 'frame' variable to avoid directly encapsulating mfn or gfn values as dfns.

---
 xen/arch/arm/p2m.c                    |  3 ++-
 xen/arch/x86/mm.c                     | 10 ++++----
 xen/arch/x86/mm/p2m-ept.c             | 10 +++++---
 xen/arch/x86/mm/p2m-pt.c              | 45 ++++++++++++++++++++---------------
 xen/arch/x86/mm/p2m.c                 | 16 ++++++++-----
 xen/arch/x86/x86_64/mm.c              |  5 ++--
 xen/common/grant_table.c              | 12 +++++-----
 xen/common/memory.c                   |  4 ++--
 xen/drivers/passthrough/iommu.c       | 25 ++++++++++---------
 xen/drivers/passthrough/vtd/x86/vtd.c |  1 -
 xen/drivers/passthrough/x86/iommu.c   |  3 ++-
 xen/include/xen/iommu.h               | 14 +++++++----
 12 files changed, 85 insertions(+), 63 deletions(-)
xen/arch/arm/p2m.c +2 -1
index 1364e5960a..0db12b01f1 100644
@@ -957,7 +957,8 @@ static int __p2m_set_entry(struct p2m_domain *p2m,
 
     if ( need_iommu(p2m->domain) &&
          (lpae_is_valid(orig_pte) || lpae_is_valid(*entry)) )
-        rc = iommu_iotlb_flush(p2m->domain, gfn_x(sgfn), 1UL << page_order);
+        rc = iommu_iotlb_flush(p2m->domain, _dfn(gfn_x(sgfn)),
+                               1UL << page_order);
     else
         rc = 0;
 
xen/arch/x86/mm.c +5 -5
index d37eea53d1..f308c97ec9 100644
@@ -2799,14 +2799,14 @@ static int _get_page_type(struct page_info *page, unsigned long type,
         struct domain *d = page_get_owner(page);
         if ( d && is_pv_domain(d) && unlikely(need_iommu(d)) )
         {
-            gfn_t gfn = _gfn(mfn_to_gmfn(d, mfn_x(page_to_mfn(page))));
+            mfn_t mfn = page_to_mfn(page);
 
             if ( (x & PGT_type_mask) == PGT_writable_page )
-                iommu_ret = iommu_unmap_page(d, gfn_x(gfn));
+                iommu_ret = iommu_unmap_page(d, _dfn(mfn_x(mfn)));
             else if ( type == PGT_writable_page )
-                iommu_ret = iommu_map_page(d, gfn_x(gfn),
-                                           mfn_x(page_to_mfn(page)),
-                                           IOMMUF_readable|IOMMUF_writable);
+                iommu_ret = iommu_map_page(d, _dfn(mfn_x(mfn)), mfn,
+                                           IOMMUF_readable |
+                                           IOMMUF_writable);
         }
     }
 
xen/arch/x86/mm/p2m-ept.c +7 -3
index 1ff4f14ae4..9a3a90e9e6 100644
@@ -870,15 +870,19 @@ out:
             rc = iommu_pte_flush(d, gfn, &ept_entry->epte, order, vtd_pte_present);
         else
         {
+            dfn_t dfn = _dfn(gfn);
+
             if ( iommu_flags )
                 for ( i = 0; i < (1 << order); i++ )
                 {
-                    rc = iommu_map_page(d, gfn + i, mfn_x(mfn) + i, iommu_flags);
+                    rc = iommu_map_page(d, dfn_add(dfn, i),
+                                        mfn_add(mfn, i), iommu_flags);
                     if ( unlikely(rc) )
                     {
                         while ( i-- )
                             /* If statement to satisfy __must_check. */
-                            if ( iommu_unmap_page(p2m->domain, gfn + i) )
+                            if ( iommu_unmap_page(p2m->domain,
+                                                  dfn_add(dfn, i)) )
                                 continue;
 
                         break;
@@ -887,7 +891,7 @@ out:
             else
                 for ( i = 0; i < (1 << order); i++ )
                 {
-                    ret = iommu_unmap_page(d, gfn + i);
+                    ret = iommu_unmap_page(d, dfn_add(dfn, i));
                     if ( !rc )
                         rc = ret;
                 }
xen/arch/x86/mm/p2m-pt.c +26 -19

55 lines not on this page · raw message

xen/arch/x86/mm/p2m.c +10 -6
index 6020553c17..801b629b95 100644
@@ -715,9 +715,11 @@ p2m_remove_page(struct p2m_domain *p2m, unsigned long gfn_l, unsigned long mfn,
 
         if ( need_iommu(p2m->domain) )
         {
+            dfn_t dfn = _dfn(mfn);
+
             for ( i = 0; i < (1 << page_order); i++ )
             {
-                int ret = iommu_unmap_page(p2m->domain, mfn + i);
+                int ret = iommu_unmap_page(p2m->domain, dfn_add(dfn, i));
 
                 if ( !rc )
                     rc = ret;
@@ -774,16 +776,17 @@ guest_physmap_add_entry(struct domain *d, gfn_t gfn, mfn_t mfn,
     {
         if ( need_iommu(d) && t == p2m_ram_rw )
         {
+            dfn_t dfn = _dfn(mfn_x(mfn));
+
             for ( i = 0; i < (1 << page_order); i++ )
             {
-                rc = iommu_map_page(d, mfn_x(mfn_add(mfn, i)),
-                                    mfn_x(mfn_add(mfn, i)),
+                rc = iommu_map_page(d, dfn_add(dfn, i), mfn_add(mfn, i),
                                     IOMMUF_readable|IOMMUF_writable);
                 if ( rc != 0 )
                 {
                     while ( i-- > 0 )
                         /* If statement to satisfy __must_check. */
-                        if ( iommu_unmap_page(d, mfn_x(mfn_add(mfn, i))) )
+                        if ( iommu_unmap_page(d, dfn_add(dfn, i)) )
                             continue;
 
                     return rc;
@@ -1158,7 +1161,8 @@ int set_identity_p2m_entry(struct domain *d, unsigned long gfn_l,
     {
         if ( !need_iommu(d) )
             return 0;
-        return iommu_map_page(d, gfn_l, gfn_l, IOMMUF_readable|IOMMUF_writable);
+        return iommu_map_page(d, _dfn(gfn_l), _mfn(gfn_l),
+                              IOMMUF_readable | IOMMUF_writable);
     }
 
     gfn_lock(p2m, gfn, 0);
@@ -1248,7 +1252,7 @@ int clear_identity_p2m_entry(struct domain *d, unsigned long gfn_l)
     {
         if ( !need_iommu(d) )
             return 0;
-        return iommu_unmap_page(d, gfn_l);
+        return iommu_unmap_page(d, _dfn(gfn_l));
     }
 
     gfn_lock(p2m, gfn, 0);
xen/arch/x86/x86_64/mm.c +3 -2
index 90a648c956..d1fce57432 100644
@@ -1430,13 +1430,14 @@ int memory_add(unsigned long spfn, unsigned long epfn, unsigned int pxm)
          !need_iommu(hardware_domain) )
     {
         for ( i = spfn; i < epfn; i++ )
-            if ( iommu_map_page(hardware_domain, i, i, IOMMUF_readable|IOMMUF_writable) )
+            if ( iommu_map_page(hardware_domain, _dfn(i), _mfn(i),
+                                IOMMUF_readable | IOMMUF_writable) )
                 break;
         if ( i != epfn )
         {
             while (i-- > old_max)
                 /* If statement to satisfy __must_check. */
-                if ( iommu_unmap_page(hardware_domain, i) )
+                if ( iommu_unmap_page(hardware_domain, _dfn(i)) )
                     continue;
 
             goto destroy_m2p;
xen/common/grant_table.c +6 -6
index 73d3ed3701..2d01cad176 100644
@@ -1143,13 +1143,13 @@ map_grant_ref(
              !(old_pin & (GNTPIN_hstw_mask|GNTPIN_devw_mask)) )
         {
             if ( !(kind & MAPKIND_WRITE) )
-                err = iommu_map_page(ld, mfn_x(mfn), mfn_x(mfn),
-                                     IOMMUF_readable|IOMMUF_writable);
+                err = iommu_map_page(ld, _dfn(mfn_x(mfn)), mfn,
+                                     IOMMUF_readable | IOMMUF_writable);
         }
         else if ( act_pin && !old_pin )
         {
             if ( !kind )
-                err = iommu_map_page(ld, mfn_x(mfn), mfn_x(mfn),
+                err = iommu_map_page(ld, _dfn(mfn_x(mfn)), mfn,
                                      IOMMUF_readable);
         }
         if ( err )
@@ -1398,10 +1398,10 @@ unmap_common(
 
         kind = mapkind(lgt, rd, op->mfn);
         if ( !kind )
-            err = iommu_unmap_page(ld, mfn_x(op->mfn));
+            err = iommu_unmap_page(ld, _dfn(mfn_x(op->mfn)));
         else if ( !(kind & MAPKIND_WRITE) )
-            err = iommu_map_page(ld, mfn_x(op->mfn),
-                                 mfn_x(op->mfn), IOMMUF_readable);
+            err = iommu_map_page(ld, _dfn(mfn_x(op->mfn)), op->mfn,
+                                 IOMMUF_readable);
 
         double_gt_unlock(lgt, rgt);
 
xen/common/memory.c +2 -2
index 996f94b103..89b211d6aa 100644
@@ -834,11 +834,11 @@ int xenmem_add_to_physmap(struct domain *d, struct xen_add_to_physmap *xatp,
 
         this_cpu(iommu_dont_flush_iotlb) = 0;
 
-        ret = iommu_iotlb_flush(d, xatp->idx - done, done);
+        ret = iommu_iotlb_flush(d, _dfn(xatp->idx - done), done);
         if ( unlikely(ret) && rc >= 0 )
             rc = ret;
 
-        ret = iommu_iotlb_flush(d, xatp->gpfn - done, done);
+        ret = iommu_iotlb_flush(d, _dfn(xatp->gpfn - done), done);
         if ( unlikely(ret) && rc >= 0 )
             rc = ret;
     }
xen/drivers/passthrough/iommu.c +12 -13
index ca8262be7c..9390b1366f 100644
@@ -295,7 +295,7 @@ void iommu_domain_destroy(struct domain *d)
     arch_iommu_domain_destroy(d);
 }
 
-int iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,
+int iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
                    unsigned int flags)
 {
     const struct domain_iommu *hd = dom_iommu(d);
@@ -304,13 +304,13 @@ int iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,
     if ( !iommu_enabled || !hd->platform_ops )
         return 0;
 
-    rc = hd->platform_ops->map_page(d, dfn, mfn, flags);
+    rc = hd->platform_ops->map_page(d, dfn_x(dfn), mfn_x(mfn), flags);
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
             printk(XENLOG_ERR
-                   "d%d: IOMMU mapping dfn %#lx to mfn %#lx failed: %d\n",
-                   d->domain_id, dfn, mfn, rc);
+                   "d%d: IOMMU mapping dfn %"PRI_dfn" to mfn %"PRI_mfn" failed: %d\n",
+                   d->domain_id, dfn_x(dfn), mfn_x(mfn), rc);
 
         if ( !is_hardware_domain(d) )
             domain_crash(d);
@@ -319,7 +319,7 @@ int iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,
     return rc;
 }
 
-int iommu_unmap_page(struct domain *d, unsigned long dfn)
+int iommu_unmap_page(struct domain *d, dfn_t dfn)
 {
     const struct domain_iommu *hd = dom_iommu(d);
     int rc;
@@ -327,13 +327,13 @@ int iommu_unmap_page(struct domain *d, unsigned long dfn)
     if ( !iommu_enabled || !hd->platform_ops )
         return 0;
 
-    rc = hd->platform_ops->unmap_page(d, dfn);
+    rc = hd->platform_ops->unmap_page(d, dfn_x(dfn));
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
             printk(XENLOG_ERR
-                   "d%d: IOMMU unmapping dfn %#lx failed: %d\n",
-                   d->domain_id, dfn, rc);
+                   "d%d: IOMMU unmapping dfn %"PRI_dfn" failed: %d\n",
+                   d->domain_id, dfn_x(dfn), rc);
 
         if ( !is_hardware_domain(d) )
             domain_crash(d);
@@ -359,8 +359,7 @@ static void iommu_free_pagetables(unsigned long unused)
                             cpumask_cycle(smp_processor_id(), &cpu_online_map));
 }
 
-int iommu_iotlb_flush(struct domain *d, unsigned long dfn,
-                      unsigned int page_count)
+int iommu_iotlb_flush(struct domain *d, dfn_t dfn, unsigned int page_count)
 {
     const struct domain_iommu *hd = dom_iommu(d);
     int rc;
@@ -368,13 +367,13 @@ int iommu_iotlb_flush(struct domain *d, unsigned long dfn,
     if ( !iommu_enabled || !hd->platform_ops || !hd->platform_ops->iotlb_flush )
         return 0;
 
-    rc = hd->platform_ops->iotlb_flush(d, dfn, page_count);
+    rc = hd->platform_ops->iotlb_flush(d, dfn_x(dfn), page_count);
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
             printk(XENLOG_ERR
-                   "d%d: IOMMU IOTLB flush failed: %d, dfn %#lx, page count %u\n",
-                   d->domain_id, rc, dfn, page_count);
+                   "d%d: IOMMU IOTLB flush failed: %d, dfn %"PRI_dfn", page count %u\n",
+                   d->domain_id, rc, dfn_x(dfn), page_count);
 
         if ( !is_hardware_domain(d) )
             domain_crash(d);
xen/drivers/passthrough/vtd/x86/vtd.c +0 -1
index 48e16f956b..ff456e1e70 100644
@@ -60,4 +60,3 @@ void flush_all_cache()
 {
     wbinvd();
 }
-
xen/drivers/passthrough/x86/iommu.c +2 -1
index 47a078272a..d6cbeae5c9 100644
@@ -227,7 +227,8 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
         if ( paging_mode_translate(d) )
             rc = set_identity_p2m_entry(d, pfn, p2m_access_rw, 0);
         else
-            rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
+            rc = iommu_map_page(d, _dfn(pfn), _mfn(pfn),
+                                IOMMUF_readable | IOMMUF_writable);
         if ( rc )
             printk(XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n",
                    d->domain_id, rc);
xen/include/xen/iommu.h +10 -4
index 290e0aada6..f9d86fc816 100644
@@ -24,6 +24,7 @@
 #include <xen/spinlock.h>
 #include <xen/pci.h>
 #include <xen/typesafe.h>
+#include <xen/mm.h>
 #include <public/hvm/ioreq.h>
 #include <public/domctl.h>
 #include <asm/device.h>
@@ -42,6 +43,11 @@ TYPE_SAFE(uint64_t, dfn);
 #undef dfn_x
 #endif
 
+static inline dfn_t dfn_add(dfn_t dfn, unsigned long i)
+{
+    return _dfn(dfn_x(dfn) + i);
+}
+
 extern bool_t iommu_enable, iommu_enabled;
 extern bool_t force_iommu, iommu_verbose;
 extern bool_t iommu_workaround_bios_bug, iommu_igfx;
@@ -78,9 +84,9 @@ void iommu_teardown(struct domain *d);
 #define IOMMUF_readable  (1u<<_IOMMUF_readable)
 #define _IOMMUF_writable 1
 #define IOMMUF_writable  (1u<<_IOMMUF_writable)
-int __must_check iommu_map_page(struct domain *d, unsigned long dfn,
-                                unsigned long mfn, unsigned int flags);
-int __must_check iommu_unmap_page(struct domain *d, unsigned long dfn);
+int __must_check iommu_map_page(struct domain *d, dfn_t dfn,
+                                mfn_t mfn, unsigned int flags);
+int __must_check iommu_unmap_page(struct domain *d, dfn_t dfn);
 
 enum iommu_feature
 {
@@ -203,7 +209,7 @@ int iommu_do_pci_domctl(struct xen_domctl *, struct domain *d,
 int iommu_do_domctl(struct xen_domctl *, struct domain *d,
                     XEN_GUEST_HANDLE_PARAM(xen_domctl_t));
 
-int __must_check iommu_iotlb_flush(struct domain *d, unsigned long dfn,
+int __must_check iommu_iotlb_flush(struct domain *d, dfn_t dfn,
                                    unsigned int page_count);
 int __must_check iommu_iotlb_flush_all(struct domain *d);
 
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH v10 3/7 iommu: push use of type-safe DFN and MFN into iommu_ops

In reply to Paul Durrant

This patch modifies the methods in struct iommu_ops to use type-safe DFN and MFN. This follows on from the prior patch that modified the functions exported in xen/iommu.h.

---
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>

v9:
- Re-base.

v7:
- Re-base and re-name BFN -> DFN.
- Added Jan's A-b since re-naming was purely mechanical.

v6:
- Re-base.

v3:
- Remove some use of intermediate 'frame' variables.

v2:
- Addressed comments from Jan.
- Extend use of intermediate 'frame' variable to avoid directly encapsulating gfn values as bfns (now dfns) or vice versa.

---
 xen/drivers/passthrough/amd/iommu_map.c       | 46 ++++++++++++++++-----------
 xen/drivers/passthrough/amd/pci_amd_iommu.c   |  2 +-
 xen/drivers/passthrough/arm/smmu.c            | 16 +++++-----
 xen/drivers/passthrough/iommu.c               |  9 +++---
 xen/drivers/passthrough/vtd/iommu.c           | 26 +++++++--------
 xen/drivers/passthrough/x86/iommu.c           |  2 +-
 xen/include/asm-x86/hvm/svm/amd-iommu-proto.h |  8 ++---
 xen/include/xen/iommu.h                       | 13 +++++---
 8 files changed, 67 insertions(+), 55 deletions(-)
xen/drivers/passthrough/amd/iommu_map.c +27 -19
index 61ade71850..c89c54fdb6 100644
@@ -631,7 +631,7 @@ static int update_paging_mode(struct domain *d, unsigned long dfn)
     return 0;
 }
 
-int amd_iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,
+int amd_iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
                        unsigned int flags)
 {
     bool_t need_flush = 0;
@@ -651,7 +651,8 @@ int amd_iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,
     if ( rc )
     {
         spin_unlock(&hd->arch.mapping_lock);
-        AMD_IOMMU_DEBUG("Root table alloc failed, dfn = %lx\n", dfn);
+        AMD_IOMMU_DEBUG("Root table alloc failed, dfn = %"PRI_dfn"\n",
+                        dfn_x(dfn));
         domain_crash(d);
         return rc;
     }
@@ -660,25 +661,27 @@ int amd_iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,
      * we might need a deeper page table for wider dfn now */
     if ( is_hvm_domain(d) )
     {
-        if ( update_paging_mode(d, dfn) )
+        if ( update_paging_mode(d, dfn_x(dfn)) )
         {
             spin_unlock(&hd->arch.mapping_lock);
-            AMD_IOMMU_DEBUG("Update page mode failed dfn = %lx\n", dfn);
+            AMD_IOMMU_DEBUG("Update page mode failed dfn = %"PRI_dfn"\n",
+                            dfn_x(dfn));
             domain_crash(d);
             return -EFAULT;
         }
     }
 
-    if ( iommu_pde_from_dfn(d, dfn, pt_mfn) || (pt_mfn[1] == 0) )
+    if ( iommu_pde_from_dfn(d, dfn_x(dfn), pt_mfn) || (pt_mfn[1] == 0) )
     {
         spin_unlock(&hd->arch.mapping_lock);
-        AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %lx\n", dfn);
+        AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n",
+                        dfn_x(dfn));
         domain_crash(d);
         return -EFAULT;
     }
 
     /* Install 4k mapping first */
-    need_flush = set_iommu_pte_present(pt_mfn[1], dfn, mfn,
+    need_flush = set_iommu_pte_present(pt_mfn[1], dfn_x(dfn), mfn_x(mfn),
                                        IOMMU_PAGING_MODE_LEVEL_1,
                                        !!(flags & IOMMUF_writable),
                                        !!(flags & IOMMUF_readable));
@@ -690,7 +693,7 @@ int amd_iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,
     /* 4K mapping for PV guests never changes, 
      * no need to flush if we trust non-present bits */
     if ( is_hvm_domain(d) )
-        amd_iommu_flush_pages(d, dfn, 0);
+        amd_iommu_flush_pages(d, dfn_x(dfn), 0);
 
     for ( merge_level = IOMMU_PAGING_MODE_LEVEL_2;
           merge_level <= hd->arch.paging_mode; merge_level++ )
@@ -698,15 +701,16 @@ int amd_iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,
         if ( pt_mfn[merge_level] == 0 )
             break;
         if ( !iommu_update_pde_count(d, pt_mfn[merge_level],
-                                     dfn, mfn, merge_level) )
+                                     dfn_x(dfn), mfn_x(mfn), merge_level) )
             break;
 
-        if ( iommu_merge_pages(d, pt_mfn[merge_level], dfn,
+        if ( iommu_merge_pages(d, pt_mfn[merge_level], dfn_x(dfn),
                                flags, merge_level) )
         {
             spin_unlock(&hd->arch.mapping_lock);
             AMD_IOMMU_DEBUG("Merge iommu page failed at level %d, "
-                            "dfn = %lx mfn = %lx\n", merge_level, dfn, mfn);
+                            "dfn = %"PRI_dfn" mfn = %"PRI_mfn"\n",
+                            merge_level, dfn_x(dfn), mfn_x(mfn));
             domain_crash(d);
             return -EFAULT;
         }
@@ -720,7 +724,7 @@ out:
     return 0;
 }
 
-int amd_iommu_unmap_page(struct domain *d, unsigned long dfn)
+int amd_iommu_unmap_page(struct domain *d, dfn_t dfn)
 {
     unsigned long pt_mfn[7];
     struct domain_iommu *hd = dom_iommu(d);
@@ -742,31 +746,33 @@ int amd_iommu_unmap_page(struct domain *d, unsigned long dfn)
      * we might need a deeper page table for lager dfn now */
     if ( is_hvm_domain(d) )
     {
-        int rc = update_paging_mode(d, dfn);
+        int rc = update_paging_mode(d, dfn_x(dfn));
 
         if ( rc )
         {
             spin_unlock(&hd->arch.mapping_lock);
-            AMD_IOMMU_DEBUG("Update page mode failed dfn = %lx\n", dfn);
+            AMD_IOMMU_DEBUG("Update page mode failed dfn = %"PRI_dfn"\n",
+                            dfn_x(dfn));
             if ( rc != -EADDRNOTAVAIL )
                 domain_crash(d);
             return rc;
         }
     }
 
-    if ( iommu_pde_from_dfn(d, dfn, pt_mfn) || (pt_mfn[1] == 0) )
+    if ( iommu_pde_from_dfn(d, dfn_x(dfn), pt_mfn) || (pt_mfn[1] == 0) )
     {
         spin_unlock(&hd->arch.mapping_lock);
-        AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %lx\n", dfn);
+        AMD_IOMMU_DEBUG("Invalid IO pagetable entry dfn = %"PRI_dfn"\n",
+                        dfn_x(dfn));
         domain_crash(d);
         return -EFAULT;
     }
 
     /* mark PTE as 'page not present' */
-    clear_iommu_pte_present(pt_mfn[1], dfn);
+    clear_iommu_pte_present(pt_mfn[1], dfn_x(dfn));
     spin_unlock(&hd->arch.mapping_lock);
 
-    amd_iommu_flush_pages(d, dfn, 0);
+    amd_iommu_flush_pages(d, dfn_x(dfn), 0);
 
     return 0;
 }
@@ -787,7 +793,9 @@ int amd_iommu_reserve_domain_unity_map(struct domain *domain,
     gfn = phys_addr >> PAGE_SHIFT;
     for ( i = 0; i < npages; i++ )
     {
-        rt = amd_iommu_map_page(domain, gfn +i, gfn +i, flags);
+        unsigned long frame = gfn + i;
+
+        rt = amd_iommu_map_page(domain, _dfn(frame), _mfn(frame), flags);
         if ( rt != 0 )
             return rt;
     }
xen/drivers/passthrough/amd/pci_amd_iommu.c +1 -1
index bcc47eb96c..2f38752ebe 100644
@@ -278,7 +278,7 @@ static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)
              */
             if ( mfn_valid(_mfn(pfn)) )
             {
-                int ret = amd_iommu_map_page(d, pfn, pfn,
+                int ret = amd_iommu_map_page(d, _dfn(pfn), _mfn(pfn),
                                              IOMMUF_readable|IOMMUF_writable);
 
                 if ( !rc )
xen/drivers/passthrough/arm/smmu.c +8 -8
index 1ff09bbb02..d66601b62e 100644
@@ -2550,8 +2550,7 @@ static int __must_check arm_smmu_iotlb_flush_all(struct domain *d)
 	return 0;
 }
 
-static int __must_check arm_smmu_iotlb_flush(struct domain *d,
-                                             unsigned long dfn,
+static int __must_check arm_smmu_iotlb_flush(struct domain *d, dfn_t dfn,
                                              unsigned int page_count)
 {
 	/* ARM SMMU v1 doesn't have flush by VMA and VMID */
@@ -2746,8 +2745,8 @@ static void arm_smmu_iommu_domain_teardown(struct domain *d)
 	xfree(xen_domain);
 }
 
-static int __must_check arm_smmu_map_page(struct domain *d, unsigned long dfn,
-			unsigned long mfn, unsigned int flags)
+static int __must_check arm_smmu_map_page(struct domain *d, dfn_t dfn,
+					  mfn_t mfn, unsigned int flags)
 {
 	p2m_type_t t;
 
@@ -2760,7 +2759,7 @@ static int __must_check arm_smmu_map_page(struct domain *d, unsigned long dfn,
 	 * function should only be used by gnttab code with gfn == mfn == dfn.
 	 */
 	BUG_ON(!is_domain_direct_mapped(d));
-	BUG_ON(mfn != dfn);
+	BUG_ON(mfn_x(mfn) != dfn_x(dfn));
 
 	/* We only support readable and writable flags */
 	if (!(flags & (IOMMUF_readable | IOMMUF_writable)))
@@ -2772,10 +2771,11 @@ static int __must_check arm_smmu_map_page(struct domain *d, unsigned long dfn,
 	 * The function guest_physmap_add_entry replaces the current mapping
 	 * if there is already one...
 	 */
-	return guest_physmap_add_entry(d, _gfn(dfn), _mfn(dfn), 0, t);
+	return guest_physmap_add_entry(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)),
+				       0, t);
 }
 
-static int __must_check arm_smmu_unmap_page(struct domain *d, unsigned long dfn)
+static int __must_check arm_smmu_unmap_page(struct domain *d, dfn_t dfn)
 {
 	/*
 	 * This function should only be used by gnttab code when the domain
@@ -2784,7 +2784,7 @@ static int __must_check arm_smmu_unmap_page(struct domain *d, unsigned long dfn)
 	if ( !is_domain_direct_mapped(d) )
 		return -EINVAL;
 
-	return guest_physmap_remove_page(d, _gfn(dfn), _mfn(dfn), 0);
+	return guest_physmap_remove_page(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), 0);
 }
 
 static const struct iommu_ops arm_smmu_iommu_ops = {
xen/drivers/passthrough/iommu.c +5 -4
index 9390b1366f..ac32a7022b 100644
@@ -224,7 +224,8 @@ void __hwdom_init iommu_hwdom_init(struct domain *d)
                   == PGT_writable_page) )
                 mapping |= IOMMUF_writable;
 
-            ret = hd->platform_ops->map_page(d, dfn, mfn, mapping);
+            ret = hd->platform_ops->map_page(d, _dfn(dfn), _mfn(mfn),
+                                             mapping);
             if ( !rc )
                 rc = ret;
 
@@ -304,7 +305,7 @@ int iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
     if ( !iommu_enabled || !hd->platform_ops )
         return 0;
 
-    rc = hd->platform_ops->map_page(d, dfn_x(dfn), mfn_x(mfn), flags);
+    rc = hd->platform_ops->map_page(d, dfn, mfn, flags);
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
@@ -327,7 +328,7 @@ int iommu_unmap_page(struct domain *d, dfn_t dfn)
     if ( !iommu_enabled || !hd->platform_ops )
         return 0;
 
-    rc = hd->platform_ops->unmap_page(d, dfn_x(dfn));
+    rc = hd->platform_ops->unmap_page(d, dfn);
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
@@ -367,7 +368,7 @@ int iommu_iotlb_flush(struct domain *d, dfn_t dfn, unsigned int page_count)
     if ( !iommu_enabled || !hd->platform_ops || !hd->platform_ops->iotlb_flush )
         return 0;
 
-    rc = hd->platform_ops->iotlb_flush(d, dfn_x(dfn), page_count);
+    rc = hd->platform_ops->iotlb_flush(d, dfn, page_count);
     if ( unlikely(rc) )
     {
         if ( !d->is_shutting_down && printk_ratelimit() )
xen/drivers/passthrough/vtd/iommu.c +12 -14
index 88720d4406..63eabb3c1b 100644
@@ -584,8 +584,7 @@ static int __must_check iommu_flush_all(void)
     return rc;
 }
 
-static int __must_check iommu_flush_iotlb(struct domain *d,
-                                          unsigned long dfn,
+static int __must_check iommu_flush_iotlb(struct domain *d, dfn_t dfn,
                                           bool_t dma_old_pte_present,
                                           unsigned int page_count)
 {
@@ -612,12 +611,12 @@ static int __must_check iommu_flush_iotlb(struct domain *d,
         if ( iommu_domid == -1 )
             continue;
 
-        if ( page_count != 1 || dfn == dfn_x(INVALID_DFN) )
+        if ( page_count != 1 || dfn_eq(dfn, INVALID_DFN) )
             rc = iommu_flush_iotlb_dsi(iommu, iommu_domid,
                                        0, flush_dev_iotlb);
         else
             rc = iommu_flush_iotlb_psi(iommu, iommu_domid,
-                                       __dfn_to_daddr(dfn),
+                                       dfn_to_daddr(dfn),
                                        PAGE_ORDER_4K,
                                        !dma_old_pte_present,
                                        flush_dev_iotlb);
@@ -633,7 +632,7 @@ static int __must_check iommu_flush_iotlb(struct domain *d,
 }
 
 static int __must_check iommu_flush_iotlb_pages(struct domain *d,
-                                                unsigned long dfn,
+                                                dfn_t dfn,
                                                 unsigned int page_count)
 {
     return iommu_flush_iotlb(d, dfn, 1, page_count);
@@ -641,7 +640,7 @@ static int __must_check iommu_flush_iotlb_pages(struct domain *d,
 
 static int __must_check iommu_flush_iotlb_all(struct domain *d)
 {
-    return iommu_flush_iotlb(d, dfn_x(INVALID_DFN), 0, 0);
+    return iommu_flush_iotlb(d, INVALID_DFN, 0, 0);
 }
 
 /* clear one page's page table */
@@ -676,7 +675,7 @@ static int __must_check dma_pte_clear_one(struct domain *domain, u64 addr)
     iommu_flush_cache_entry(pte, sizeof(struct dma_pte));
 
     if ( !this_cpu(iommu_dont_flush_iotlb) )
-        rc = iommu_flush_iotlb_pages(domain, addr >> PAGE_SHIFT_4K, 1);
+        rc = iommu_flush_iotlb_pages(domain, daddr_to_dfn(addr), 1);
 
     unmap_vtd_domain_page(page);
 
@@ -1768,8 +1767,7 @@ static void iommu_domain_teardown(struct domain *d)
 }
 
 static int __must_check intel_iommu_map_page(struct domain *d,
-                                             unsigned long dfn,
-                                             unsigned long mfn,
+                                             dfn_t dfn, mfn_t mfn,
                                              unsigned int flags)
 {
     struct domain_iommu *hd = dom_iommu(d);
@@ -1787,16 +1785,16 @@ static int __must_check intel_iommu_map_page(struct domain *d,
 
     spin_lock(&hd->arch.mapping_lock);
 
-    pg_maddr = addr_to_dma_page_maddr(d, __dfn_to_daddr(dfn), 1);
+    pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), 1);
     if ( pg_maddr == 0 )
     {
         spin_unlock(&hd->arch.mapping_lock);
         return -ENOMEM;
     }
     page = (struct dma_pte *)map_vtd_domain_page(pg_maddr);
-    pte = page + (dfn & LEVEL_MASK);
+    pte = page + (dfn_x(dfn) & LEVEL_MASK);
     old = *pte;
-    dma_set_pte_addr(new, (paddr_t)mfn << PAGE_SHIFT_4K);
+    dma_set_pte_addr(new, mfn_to_maddr(mfn));
     dma_set_pte_prot(new,
                      ((flags & IOMMUF_readable) ? DMA_PTE_READ  : 0) |
                      ((flags & IOMMUF_writable) ? DMA_PTE_WRITE : 0));
@@ -1824,13 +1822,13 @@ static int __must_check intel_iommu_map_page(struct domain *d,
 }
 
 static int __must_check intel_iommu_unmap_page(struct domain *d,
-                                               unsigned long dfn)
+                                               dfn_t dfn)
 {
     /* Do nothing if hardware domain and iommu supports pass thru. */
     if ( iommu_hwdom_passthrough && is_hardware_domain(d) )
         return 0;
 
-    return dma_pte_clear_one(d, __dfn_to_daddr(dfn));
+    return dma_pte_clear_one(d, dfn_to_daddr(dfn));
 }
 
 int iommu_pte_flush(struct domain *d, uint64_t dfn, uint64_t *pte,
xen/drivers/passthrough/x86/iommu.c +1 -1
index d6cbeae5c9..70e79c20f7 100644
@@ -68,7 +68,7 @@ int arch_iommu_populate_page_table(struct domain *d)
             {
                 ASSERT(!(gfn >> DEFAULT_DOMAIN_ADDRESS_WIDTH));
                 BUG_ON(SHARED_M2P(gfn));
-                rc = hd->platform_ops->map_page(d, gfn, mfn,
+                rc = hd->platform_ops->map_page(d, _dfn(gfn), _mfn(mfn),
                                                 IOMMUF_readable |
                                                 IOMMUF_writable);
             }
xen/include/asm-x86/hvm/svm/amd-iommu-proto.h +4 -4
index 99bc21c7b3..3083d625bd 100644
@@ -52,9 +52,9 @@ int amd_iommu_init(void);
 int amd_iommu_update_ivrs_mapping_acpi(void);
 
 /* mapping functions */
-int __must_check amd_iommu_map_page(struct domain *d, unsigned long gfn,
-                                    unsigned long mfn, unsigned int flags);
-int __must_check amd_iommu_unmap_page(struct domain *d, unsigned long gfn);
+int __must_check amd_iommu_map_page(struct domain *d, dfn_t dfn,
+                                    mfn_t mfn, unsigned int flags);
+int __must_check amd_iommu_unmap_page(struct domain *d, dfn_t dfn);
 u64 amd_iommu_get_next_table_from_pte(u32 *entry);
 int __must_check amd_iommu_alloc_root(struct domain_iommu *hd);
 int amd_iommu_reserve_domain_unity_map(struct domain *domain,
@@ -77,7 +77,7 @@ void iommu_dte_set_guest_cr3(u32 *dte, u16 dom_id, u64 gcr3,
 
 /* send cmd to iommu */
 void amd_iommu_flush_all_pages(struct domain *d);
-void amd_iommu_flush_pages(struct domain *d, unsigned long gfn,
+void amd_iommu_flush_pages(struct domain *d, unsigned long dfn,
                            unsigned int order);
 void amd_iommu_flush_iotlb(u8 devfn, const struct pci_dev *pdev,
                            uint64_t gaddr, unsigned int order);
xen/include/xen/iommu.h +9 -4
index f9d86fc816..7313957c81 100644
@@ -48,6 +48,11 @@ static inline dfn_t dfn_add(dfn_t dfn, unsigned long i)
     return _dfn(dfn_x(dfn) + i);
 }
 
+static inline bool_t dfn_eq(dfn_t x, dfn_t y)
+{
+    return dfn_x(x) == dfn_x(y);
+}
+
 extern bool_t iommu_enable, iommu_enabled;
 extern bool_t force_iommu, iommu_verbose;
 extern bool_t iommu_workaround_bios_bug, iommu_igfx;
@@ -174,9 +179,9 @@ struct iommu_ops {
 #endif /* HAS_PCI */
 
     void (*teardown)(struct domain *d);
-    int __must_check (*map_page)(struct domain *d, unsigned long dfn,
-                                 unsigned long mfn, unsigned int flags);
-    int __must_check (*unmap_page)(struct domain *d, unsigned long dfn);
+    int __must_check (*map_page)(struct domain *d, dfn_t dfn, mfn_t mfn,
+                                 unsigned int flags);
+    int __must_check (*unmap_page)(struct domain *d, dfn_t dfn);
     void (*free_page_table)(struct page_info *);
 #ifdef CONFIG_X86
     void (*update_ire_from_apic)(unsigned int apic, unsigned int reg, unsigned int value);
@@ -187,7 +192,7 @@ struct iommu_ops {
     void (*resume)(void);
     void (*share_p2m)(struct domain *d);
     void (*crash_shutdown)(void);
-    int __must_check (*iotlb_flush)(struct domain *d, unsigned long dfn,
+    int __must_check (*iotlb_flush)(struct domain *d, dfn_t dfn,
                                     unsigned int page_count);
     int __must_check (*iotlb_flush_all)(struct domain *d);
     int (*get_reserved_device_memory)(iommu_grdm_t *, void *);
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#4 lore raw

PATCH v10 4/7 iommu: don't domain_crash() inside iommu_map/unmap_page()

In reply to Paul Durrant

This patch removes the implicit domain_crash() from iommu_map(), unmap_page() and iommu_iotlb_flush() and turns them into straightforward wrappers that check the existence of the relevant iommu_op and call through to it. This makes them usable by PV IOMMU code to be delivered in future patches.
This patch adds a helper macro, domu_crash(), that will only invoke domain_crash() if the domain is not the hardware domain and modifies callers of iommu_map(), unmap_page() and iommu_iotlb_flush() to use this should an operation fail.

NOTE: This patch includes one bit of clean-up in set_identity_p2m_entry()

      replacing use of p2m->domain with the domain pointer passed into the
      function.

---
Cc: Wei Liu <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: Jun Nakajima <[email protected]>
Cc: George Dunlap <[email protected]>

v7:
- Re-base and re-name BFN -> DFN.
- Move domu_crash() outside double locked region in grant_table.c.
- Added Jan's R-b.

v6:
- Introduce domu_crash() (idea suggested by Kevin, name suggested by Jan) to crash non-hardware domains.
- Dropped Wei's and George's R-b because of change.

v2:
- New in v2.

---
 xen/arch/arm/p2m.c                  |  4 ++++
 xen/arch/x86/mm.c                   |  3 +++
 xen/arch/x86/mm/p2m-ept.c           |  3 +++
 xen/arch/x86/mm/p2m-pt.c            |  3 +++
 xen/arch/x86/mm/p2m.c               | 22 ++++++++++++++++++----
 xen/common/grant_table.c            |  4 ++++
 xen/common/memory.c                 |  3 +++
 xen/drivers/passthrough/iommu.c     | 12 ------------
 xen/drivers/passthrough/x86/iommu.c |  4 ++++
 xen/include/xen/sched.h             |  5 +++++
 10 files changed, 47 insertions(+), 16 deletions(-)
xen/arch/arm/p2m.c +4 -0
index 0db12b01f1..1c79ff7ade 100644
@@ -957,8 +957,12 @@ static int __p2m_set_entry(struct p2m_domain *p2m,
 
     if ( need_iommu(p2m->domain) &&
          (lpae_is_valid(orig_pte) || lpae_is_valid(*entry)) )
+    {
         rc = iommu_iotlb_flush(p2m->domain, _dfn(gfn_x(sgfn)),
                                1UL << page_order);
+        if ( unlikely(rc) )
+            domu_crash(p2m->domain);
+    }
     else
         rc = 0;
 
xen/arch/x86/mm.c +3 -0
index f308c97ec9..c2f20991bb 100644
@@ -2807,6 +2807,9 @@ static int _get_page_type(struct page_info *page, unsigned long type,
                 iommu_ret = iommu_map_page(d, _dfn(mfn_x(mfn)), mfn,
                                            IOMMUF_readable |
                                            IOMMUF_writable);
+
+            if ( unlikely(iommu_ret) )
+                domu_crash(d);
         }
     }
 
xen/arch/x86/mm/p2m-ept.c +3 -0
index 9a3a90e9e6..af7674f7e1 100644
@@ -895,6 +895,9 @@ out:
                     if ( !rc )
                         rc = ret;
                 }
+
+            if ( unlikely(rc) )
+                domu_crash(d);
         }
     }
 
xen/arch/x86/mm/p2m-pt.c +3 -0
index 881e9e87b8..607046f31b 100644
@@ -717,6 +717,9 @@ p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_, mfn_t mfn,
                         rc = ret;
                 }
         }
+
+        if ( unlikely(rc) )
+            domu_crash(p2m->domain);
     }
 
     /*
xen/arch/x86/mm/p2m.c +18 -4
index 801b629b95..537add65bb 100644
@@ -724,6 +724,9 @@ p2m_remove_page(struct p2m_domain *p2m, unsigned long gfn_l, unsigned long mfn,
                 if ( !rc )
                     rc = ret;
             }
+
+            if ( unlikely(rc) )
+                domu_crash(p2m->domain);
         }
 
         return rc;
@@ -789,6 +792,7 @@ guest_physmap_add_entry(struct domain *d, gfn_t gfn, mfn_t mfn,
                         if ( iommu_unmap_page(d, dfn_add(dfn, i)) )
                             continue;
 
+                    domu_crash(d);
                     return rc;
                 }
             }
@@ -1157,12 +1161,17 @@ int set_identity_p2m_entry(struct domain *d, unsigned long gfn_l,
     struct p2m_domain *p2m = p2m_get_hostp2m(d);
     int ret;
 
-    if ( !paging_mode_translate(p2m->domain) )
+    if ( !paging_mode_translate(d) )
     {
         if ( !need_iommu(d) )
             return 0;
-        return iommu_map_page(d, _dfn(gfn_l), _mfn(gfn_l),
-                              IOMMUF_readable | IOMMUF_writable);
+
+        ret = iommu_map_page(d, _dfn(gfn_l), _mfn(gfn_l),
+                             IOMMUF_readable | IOMMUF_writable);
+        if ( unlikely(ret) )
+            domu_crash(d);
+
+        return ret;
     }
 
     gfn_lock(p2m, gfn, 0);
@@ -1252,7 +1261,12 @@ int clear_identity_p2m_entry(struct domain *d, unsigned long gfn_l)
     {
         if ( !need_iommu(d) )
             return 0;
-        return iommu_unmap_page(d, _dfn(gfn_l));
+
+        ret = iommu_unmap_page(d, _dfn(gfn_l));
+        if ( unlikely(ret) )
+            domu_crash(d);
+
+        return ret;
     }
 
     gfn_lock(p2m, gfn, 0);
xen/common/grant_table.c +4 -0
index 2d01cad176..0f0b7b1a49 100644
@@ -1155,6 +1155,7 @@ map_grant_ref(
         if ( err )
         {
             double_gt_unlock(lgt, rgt);
+            domu_crash(ld);
             rc = GNTST_general_error;
             goto undo_out;
         }
@@ -1406,7 +1407,10 @@ unmap_common(
         double_gt_unlock(lgt, rgt);
 
         if ( err )
+        {
+            domu_crash(ld);
             rc = GNTST_general_error;
+        }
     }
 
     /* If just unmapped a writable mapping, mark as dirtied */
xen/common/memory.c +3 -0
index 89b211d6aa..85611ddae4 100644
@@ -841,6 +841,9 @@ int xenmem_add_to_physmap(struct domain *d, struct xen_add_to_physmap *xatp,
         ret = iommu_iotlb_flush(d, _dfn(xatp->gpfn - done), done);
         if ( unlikely(ret) && rc >= 0 )
             rc = ret;
+
+        if ( unlikely(rc < 0) )
+            domu_crash(d);
     }
 #endif
 
xen/drivers/passthrough/iommu.c +0 -12
index ac32a7022b..24f2a2a460 100644
@@ -312,9 +312,6 @@ int iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,
             printk(XENLOG_ERR
                    "d%d: IOMMU mapping dfn %"PRI_dfn" to mfn %"PRI_mfn" failed: %d\n",
                    d->domain_id, dfn_x(dfn), mfn_x(mfn), rc);
-
-        if ( !is_hardware_domain(d) )
-            domain_crash(d);
     }
 
     return rc;
@@ -335,9 +332,6 @@ int iommu_unmap_page(struct domain *d, dfn_t dfn)
             printk(XENLOG_ERR
                    "d%d: IOMMU unmapping dfn %"PRI_dfn" failed: %d\n",
                    d->domain_id, dfn_x(dfn), rc);
-
-        if ( !is_hardware_domain(d) )
-            domain_crash(d);
     }
 
     return rc;
@@ -375,9 +369,6 @@ int iommu_iotlb_flush(struct domain *d, dfn_t dfn, unsigned int page_count)
             printk(XENLOG_ERR
                    "d%d: IOMMU IOTLB flush failed: %d, dfn %"PRI_dfn", page count %u\n",
                    d->domain_id, rc, dfn_x(dfn), page_count);
-
-        if ( !is_hardware_domain(d) )
-            domain_crash(d);
     }
 
     return rc;
@@ -398,9 +389,6 @@ int iommu_iotlb_flush_all(struct domain *d)
             printk(XENLOG_ERR
                    "d%d: IOMMU IOTLB flush all failed: %d\n",
                    d->domain_id, rc);
-
-        if ( !is_hardware_domain(d) )
-            domain_crash(d);
     }
 
     return rc;
xen/drivers/passthrough/x86/iommu.c +4 -0
index 70e79c20f7..1b47bcba3f 100644
@@ -107,7 +107,11 @@ int arch_iommu_populate_page_table(struct domain *d)
     this_cpu(iommu_dont_flush_iotlb) = 0;
 
     if ( !rc )
+    {
         rc = iommu_iotlb_flush_all(d);
+        if ( unlikely(rc) )
+            domain_crash(d);
+    }
 
     if ( rc && rc != -ERESTART )
         iommu_teardown(d);
xen/include/xen/sched.h +5 -0
index 0ba80cb1a8..f2c594d197 100644
@@ -616,6 +616,11 @@ void __domain_crash(struct domain *d);
     __domain_crash(d);                                                    \
 } while (0)
 
+#define domu_crash(d) do {                \
+    if ( !is_hardware_domain(d) )         \
+        domain_crash(d);                  \
+} while (false)
+
 /*
  * Called from assembly code, with an optional address to help indicate why
  * the crash occured.  If addr is 0, look up address from last extable
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#5 lore raw

PATCH v10 5/7 memory: add check_get_page_from_gfn() as a wrapper...

In reply to Paul Durrant

...for some uses of get_page_from_gfn().

There are many occurrences of the following pattern in the code:

    q = <readonly look-up> ? P2M_ALLOC : P2M_UNSHARE;
    page = get_page_from_gfn(d, gfn, &p2mt, q);

    if ( p2m_is_paging(p2mt) )
    {
        if ( page )
            put_page(page);

        p2m_mem_paging_populate(d, gfn);
        return <-EAGAIN or equivalent>;
    }

    if ( (q & P2M_UNSHARE) && p2m_is_shared(p2mt) )
    {
        if ( page )
            put_page(page);

        return <-EAGAIN or equivalent>;
    }

    if ( !page )
        return <-EINVAL or equivalent>;

There are some small differences between the exact way the occurrences are coded but the desired semantic is the same.

This patch introduces a new common implementation of this code in check_get_page_from_gfn() and then converts the various open-coded patterns into calls to this new function.

NOTE: There are duplicate declarations of check_get_page_from_gfn() for
ARM and x86 because the function takes a p2m_type_t argument and that type is architecture specific.

---
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: Wei Liu <[email protected]>

v10:
- Expand commit comment to point out the reason for the duplicate declarations of check_get_page_from_gfn().

v9:
- Defer P2M type checks (beyond shared or paging) to the caller.

v7:
- Fix ARM build by introducing p2m_is_readonly() predicate.
- Re-name get_paged_frame() -> check_get_page_from_gfn().
- Adjust default cases of callers switch-ing on return value.

v3:
- Addressed comments from George.

v2:
- New in v2.

---
 xen/arch/x86/hvm/emulate.c | 25 ++++++++++++-----------
 xen/arch/x86/hvm/hvm.c     | 14 +------------
 xen/common/grant_table.c   | 32 ++++++++++++++----------------
 xen/common/memory.c        | 49 ++++++++++++++++++++++++++++++++++++----------
 xen/include/asm-arm/p2m.h  |  4 ++++
 xen/include/asm-x86/p2m.h  |  3 +++
 6 files changed, 74 insertions(+), 53 deletions(-)
xen/arch/x86/hvm/emulate.c +12 -13
index a577685dc6..480840b202 100644
@@ -356,22 +356,21 @@ static int hvmemul_acquire_page(unsigned long gmfn, struct page_info **page)
     struct domain *curr_d = current->domain;
     p2m_type_t p2mt;
 
-    *page = get_page_from_gfn(curr_d, gmfn, &p2mt, P2M_UNSHARE);
-
-    if ( *page == NULL )
-        return X86EMUL_UNHANDLEABLE;
-
-    if ( p2m_is_paging(p2mt) )
+    switch ( check_get_page_from_gfn(curr_d, _gfn(gmfn), false, &p2mt,
+                                     page) )
     {
-        put_page(*page);
-        p2m_mem_paging_populate(curr_d, gmfn);
-        return X86EMUL_RETRY;
-    }
+    case 0:
+        break;
 
-    if ( p2m_is_shared(p2mt) )
-    {
-        put_page(*page);
+    case -EAGAIN:
         return X86EMUL_RETRY;
+
+    default:
+        ASSERT_UNREACHABLE();
+        /* Fallthrough */
+
+    case -EINVAL:
+        return X86EMUL_UNHANDLEABLE;
     }
 
     /* This code should not be reached if the gmfn is not RAM */
xen/arch/x86/hvm/hvm.c +1 -13
index fe6c9c592f..6bb1da07eb 100644
@@ -2536,20 +2536,8 @@ static void *_hvm_map_guest_frame(unsigned long gfn, bool_t permanent,
     struct page_info *page;
     struct domain *d = current->domain;
 
-    page = get_page_from_gfn(d, gfn, &p2mt,
-                             writable ? P2M_UNSHARE : P2M_ALLOC);
-    if ( (p2m_is_shared(p2mt) && writable) || !page )
-    {
-        if ( page )
-            put_page(page);
-        return NULL;
-    }
-    if ( p2m_is_paging(p2mt) )
-    {
-        put_page(page);
-        p2m_mem_paging_populate(d, gfn);
+    if ( check_get_page_from_gfn(d, _gfn(gfn), !writable, &p2mt, &page) )
         return NULL;
-    }
 
     if ( writable )
     {
xen/common/grant_table.c +15 -17
index 0f0b7b1a49..3604a8812c 100644
@@ -374,25 +374,23 @@ static int get_paged_frame(unsigned long gfn, mfn_t *mfn,
                            struct page_info **page, bool readonly,
                            struct domain *rd)
 {
-    int rc = GNTST_okay;
     p2m_type_t p2mt;
+    int rc;
 
-    *mfn = INVALID_MFN;
-    *page = get_page_from_gfn(rd, gfn, &p2mt,
-                              readonly ? P2M_ALLOC : P2M_UNSHARE);
-    if ( !*page )
+    rc = check_get_page_from_gfn(rd, _gfn(gfn), readonly, &p2mt, page);
+    switch ( rc )
     {
-#ifdef P2M_SHARED_TYPES
-        if ( p2m_is_shared(p2mt) )
-            return GNTST_eagain;
-#endif
-#ifdef P2M_PAGES_TYPES
-        if ( p2m_is_paging(p2mt) )
-        {
-            p2m_mem_paging_populate(rd, gfn);
-            return GNTST_eagain;
-        }
-#endif
+    case 0:
+        break;
+
+    case -EAGAIN:
+        return GNTST_eagain;
+
+    default:
+        ASSERT_UNREACHABLE();
+        /* Fallthrough */
+
+    case -EINVAL:
         return GNTST_bad_page;
     }
 
@@ -406,7 +404,7 @@ static int get_paged_frame(unsigned long gfn, mfn_t *mfn,
 
     *mfn = page_to_mfn(*page);
 
-    return rc;
+    return GNTST_okay;
 }
 
 static inline void
xen/common/memory.c +39 -10

76 lines not on this page · raw message

xen/include/asm-arm/p2m.h +4 -0
index 8823707c17..377b591bc6 100644
@@ -303,6 +303,10 @@ static inline struct page_info *get_page_from_gfn(
     return page;
 }
 
+int __must_check check_get_page_from_gfn(struct domain *d, gfn_t gfn,
+                                         bool readonly, p2m_type_t *p2mt_p,
+                                         struct page_info **page_p);
+
 int get_page_type(struct page_info *page, unsigned long type);
 bool is_iomem_page(mfn_t mfn);
 static inline int get_page_and_type(struct page_info *page,
xen/include/asm-x86/p2m.h +3 -0
index d4b3cfcb6e..8613df42ce 100644
@@ -492,6 +492,9 @@ static inline struct page_info *get_page_from_gfn(
     return mfn_valid(_mfn(gfn)) && get_page(page, d) ? page : NULL;
 }
 
+int __must_check check_get_page_from_gfn(struct domain *d, gfn_t gfn,
+                                         bool readonly, p2m_type_t *p2mt_p,
+                                         struct page_info **page_p);
 
 /* General conversion function from mfn to gfn */
 static inline unsigned long mfn_to_gfn(struct domain *d, mfn_t mfn)
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#6 lore raw

Re: [PATCH v10 5/7] memory: add check_get_page_from_gfn() as a wrapper...

Hi Paul,

Quoted 30 lines · Paul Durrant

...for some uses of get_page_from_gfn().

There are many occurrences of the following pattern in the code:

     q = <readonly look-up> ? P2M_ALLOC : P2M_UNSHARE;
     page = get_page_from_gfn(d, gfn, &p2mt, q);

     if ( p2m_is_paging(p2mt) )
     {
         if ( page )
             put_page(page);

         p2m_mem_paging_populate(d, gfn);
         return <-EAGAIN or equivalent>;
     }

     if ( (q & P2M_UNSHARE) && p2m_is_shared(p2mt) )
     {
         if ( page )
             put_page(page);

         return <-EAGAIN or equivalent>;
     }

     if ( !page )
         return <-EINVAL or equivalent>;

There are some small differences between the exact way the occurrences are coded but the desired semantic is the same.

Quoted 7 lines

This patch introduces a new common implementation of this code in check_get_page_from_gfn() and then converts the various open-coded patterns into calls to this new function.

NOTE: There are duplicate declarations of check_get_page_from_gfn() for
ARM and x86 because the function takes a p2m_type_t argument and that type is architecture specific.

Wouldn't it be possible to move #include p2m-common.h later on in p2m.h?

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#7 lore raw

PATCH v10 6/7 vtd: add missing check for shared EPT...

In reply to Paul Durrant

...in intel_iommu_unmap_page().

This patch also includes some non-functional modifications in intel_iommu_map_page().

---
Cc: Wei Liu <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: George Dunlap <[email protected]>

v8:
- New in v8. (Split from the next patch in the series as requested by Kevin).

---
 xen/drivers/passthrough/vtd/iommu.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
xen/drivers/passthrough/vtd/iommu.c +10 -3
index 63eabb3c1b..0eac755ff3 100644
@@ -1771,7 +1771,7 @@ static int __must_check intel_iommu_map_page(struct domain *d,
                                              unsigned int flags)
 {
     struct domain_iommu *hd = dom_iommu(d);
-    struct dma_pte *page = NULL, *pte = NULL, old, new = { 0 };
+    struct dma_pte *page, *pte, old, new = {};
     u64 pg_maddr;
     int rc = 0;
 
@@ -1786,14 +1786,16 @@ static int __must_check intel_iommu_map_page(struct domain *d,
     spin_lock(&hd->arch.mapping_lock);
 
     pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), 1);
-    if ( pg_maddr == 0 )
+    if ( !pg_maddr )
     {
         spin_unlock(&hd->arch.mapping_lock);
         return -ENOMEM;
     }
+
     page = (struct dma_pte *)map_vtd_domain_page(pg_maddr);
-    pte = page + (dfn_x(dfn) & LEVEL_MASK);
+    pte = &page[dfn_x(dfn) & LEVEL_MASK];
     old = *pte;
+
     dma_set_pte_addr(new, mfn_to_maddr(mfn));
     dma_set_pte_prot(new,
                      ((flags & IOMMUF_readable) ? DMA_PTE_READ  : 0) |
@@ -1809,6 +1811,7 @@ static int __must_check intel_iommu_map_page(struct domain *d,
         unmap_vtd_domain_page(page);
         return 0;
     }
+
     *pte = new;
 
     iommu_flush_cache_entry(pte, sizeof(struct dma_pte));
@@ -1824,6 +1827,10 @@ static int __must_check intel_iommu_map_page(struct domain *d,
 static int __must_check intel_iommu_unmap_page(struct domain *d,
                                                dfn_t dfn)
 {
+    /* Do nothing if VT-d shares EPT page table */
+    if ( iommu_use_hap_pt(d) )
+        return 0;
+
     /* Do nothing if hardware domain and iommu supports pass thru. */
     if ( iommu_hwdom_passthrough && is_hardware_domain(d) )
         return 0;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#8 lore raw

PATCH v10 7/7 vtd: add lookup_page method to iommu_ops

In reply to Paul Durrant

This patch adds a new method to the VT-d IOMMU implementation to find the MFN currently mapped by the specified DFN along with a wrapper function in generic IOMMU code to call the implementation if it exists.

NOTE: This patch only adds a Xen-internal interface. This will be used by
a subsequent patch.
Another subsequent patch will add similar functionality for AMD IOMMUs.

---
Cc: Wei Liu <[email protected]>
Cc: George Dunlap <[email protected]>

v10:
- Adjust the locking comment.

v9:
- Add comment about locking in xen/iommu.h.

v8:
- Remove clean-up as this is now done by a prior patch.
- Make intel_iommu_lookup_page() return dfn value if using shared EPT or iommu_passthrough is set, as requested by Kevin.

v7:
- Re-base and re-name BFN -> DFN.
- Add missing checks for shared EPT and iommu_passthrough.
- Remove unnecessary initializers and use array-style dereference.
- Drop Wei's R-b because of code churn.

v3:
- Addressed comments from George.

v2:
- Addressed some comments from Jan.

---
 xen/drivers/passthrough/iommu.c     | 11 ++++++++++
 xen/drivers/passthrough/vtd/iommu.c | 44 +++++++++++++++++++++++++++++++++++++
 xen/drivers/passthrough/vtd/iommu.h |  3 +++
 xen/include/xen/iommu.h             | 10 +++++++++
 4 files changed, 68 insertions(+)
xen/drivers/passthrough/iommu.c +11 -0
index 24f2a2a460..853a14438e 100644
@@ -337,6 +337,17 @@ int iommu_unmap_page(struct domain *d, dfn_t dfn)
     return rc;
 }
 
+int iommu_lookup_page(struct domain *d, dfn_t dfn, mfn_t *mfn,
+                      unsigned int *flags)
+{
+    const struct domain_iommu *hd = dom_iommu(d);
+
+    if ( !iommu_enabled || !hd->platform_ops )
+        return -EOPNOTSUPP;
+
+    return hd->platform_ops->lookup_page(d, dfn, mfn, flags);
+}
+
 static void iommu_free_pagetables(unsigned long unused)
 {
     do {
xen/drivers/passthrough/vtd/iommu.c +44 -0

56 lines not on this page · raw message

xen/drivers/passthrough/vtd/iommu.h +3 -0
index 72c1a2e3cd..47bdfcb5ea 100644
@@ -272,6 +272,9 @@ struct dma_pte {
 #define dma_set_pte_prot(p, prot) do { \
         (p).val = ((p).val & ~DMA_PTE_PROT) | ((prot) & DMA_PTE_PROT); \
     } while (0)
+#define dma_pte_prot(p) ((p).val & DMA_PTE_PROT)
+#define dma_pte_read(p) (dma_pte_prot(p) & DMA_PTE_READ)
+#define dma_pte_write(p) (dma_pte_prot(p) & DMA_PTE_WRITE)
 #define dma_pte_addr(p) ((p).val & PADDR_MASK & PAGE_MASK_4K)
 #define dma_set_pte_addr(p, addr) do {\
             (p).val |= ((addr) & PAGE_MASK_4K); } while (0)
xen/include/xen/iommu.h +10 -0
index 7313957c81..9ae8321bb4 100644
@@ -92,6 +92,8 @@ void iommu_teardown(struct domain *d);
 int __must_check iommu_map_page(struct domain *d, dfn_t dfn,
                                 mfn_t mfn, unsigned int flags);
 int __must_check iommu_unmap_page(struct domain *d, dfn_t dfn);
+int __must_check iommu_lookup_page(struct domain *d, dfn_t dfn, mfn_t *mfn,
+                                   unsigned int *flags);
 
 enum iommu_feature
 {
@@ -179,9 +181,17 @@ struct iommu_ops {
 #endif /* HAS_PCI */
 
     void (*teardown)(struct domain *d);
+
+    /*
+     * This block of operations must be appropriately locked against each
+     * other by the caller in order to have meaningful results.
+     */
     int __must_check (*map_page)(struct domain *d, dfn_t dfn, mfn_t mfn,
                                  unsigned int flags);
     int __must_check (*unmap_page)(struct domain *d, dfn_t dfn);
+    int __must_check (*lookup_page)(struct domain *d, dfn_t dfn, mfn_t *mfn,
+                                    unsigned int *flags);
+
     void (*free_page_table)(struct page_info *);
 #ifdef CONFIG_X86
     void (*update_ire_from_apic)(unsigned int apic, unsigned int reg, unsigned int value);
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel