paravirtual IOMMU pre-requisites and clean-up
v11 cover paravirtual IOMMU pre-requisites and clean-up
This series contains pre-requisites and clean-up needed for paravirtual IOMMU support.
I have separated these patches to avoid further delaying their application whilst I re-work the implementation of paravirtual IOMMU after review of v6 of the series. Several of them already have all necessary acks.
Paul Durrant (7):
iommu: introduce the concept of DFN...
iommu: make use of type-safe DFN and MFN in exported functions
iommu: push use of type-safe DFN and MFN into iommu_ops
iommu: don't domain_crash() inside iommu_map/unmap_page()
memory: add check_get_page_from_gfn() as a wrapper...
vtd: add missing check for shared EPT...
vtd: add lookup_page method to iommu_ops
xen/arch/arm/p2m.c | 7 ++- xen/arch/x86/hvm/emulate.c | 25 ++++---- xen/arch/x86/hvm/hvm.c | 14 +---- xen/arch/x86/mm.c | 13 ++-- xen/arch/x86/mm/p2m-ept.c | 13 +++- xen/arch/x86/mm/p2m-pt.c | 48 ++++++++------ xen/arch/x86/mm/p2m.c | 32 +++++++--- xen/arch/x86/x86_64/mm.c | 5 +- xen/common/grant_table.c | 48 +++++++------- xen/common/memory.c | 56 +++++++++++++---- xen/drivers/passthrough/amd/iommu_cmd.c | 18 +++--- xen/drivers/passthrough/amd/iommu_map.c | 88 ++++++++++++++------------ xen/drivers/passthrough/amd/pci_amd_iommu.c | 4 +- xen/drivers/passthrough/arm/smmu.c | 20 +++--- xen/drivers/passthrough/iommu.c | 53 ++++++++-------- xen/drivers/passthrough/vtd/iommu.c | 91 ++++++++++++++++++++------- xen/drivers/passthrough/vtd/iommu.h | 3 + xen/drivers/passthrough/vtd/x86/vtd.c | 1 - xen/drivers/passthrough/x86/iommu.c | 9 ++- xen/include/asm-arm/p2m.h | 4 +- xen/include/asm-x86/hvm/svm/amd-iommu-proto.h | 8 +-- xen/include/asm-x86/iommu.h | 12 ++++ xen/include/asm-x86/p2m.h | 5 +- xen/include/xen/iommu.h | 51 ++++++++++++--- xen/include/xen/mm.h | 5 ++ xen/include/xen/p2m-common.h | 6 ++ xen/include/xen/sched.h | 5 ++ 27 files changed, 417 insertions(+), 227 deletions(-)
---
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Jun Nakajima <[email protected]>
Cc: Kevin Tian <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: Wei Liu <[email protected]>
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v11 6/7 vtd: add missing check for shared EPT...
...in intel_iommu_unmap_page().
This patch also includes some non-functional modifications in intel_iommu_map_page().
---
Cc: Wei Liu <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: George Dunlap <[email protected]>
v8:
- New in v8. (Split from the next patch in the series as requested by Kevin).
--- xen/drivers/passthrough/vtd/iommu.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
xen/drivers/passthrough/vtd/iommu.c +10 -3
@@ -1771,7 +1771,7 @@ static int __must_check intel_iommu_map_page(struct domain *d, unsigned int flags) {struct domain_iommu *hd =dom_iommu (d); -struct dma_pte *page =NULL , *pte =NULL , old, new = {0 }; +struct dma_pte *page, *pte, old, new = {};u64 pg_maddr;int rc =0 ;
@@ -1786,14 +1786,16 @@ static int __must_check intel_iommu_map_page(struct domain *d,spin_lock (&hd->arch.mapping_lock); pg_maddr =addr_to_dma_page_maddr (d,dfn_to_daddr (dfn),1 ); -if ( pg_maddr ==0 ) +if ( !pg_maddr ) {spin_unlock (&hd->arch.mapping_lock);return -ENOMEM ; } + page = (struct dma_pte *)map_vtd_domain_page (pg_maddr); - pte = page + (dfn_x (dfn) &LEVEL_MASK ); + pte = &page[dfn_x (dfn) &LEVEL_MASK ]; old = *pte; +dma_set_pte_addr (new,mfn_to_maddr (mfn)); dma_set_pte_prot(new, ((flags & IOMMUF_readable) ? DMA_PTE_READ : 0) |
@@ -1809,6 +1811,7 @@ static int __must_check intel_iommu_map_page(struct domain *d,unmap_vtd_domain_page (page);return 0 ; } + *pte = new;iommu_flush_cache_entry (pte,sizeof (struct dma_pte ));
@@ -1824,6 +1827,10 @@ static int __must_check intel_iommu_map_page(struct domain *d,static int __must_check intel_iommu_unmap_page (struct domain *d,dfn_t dfn) { +/* Do nothing if VT-d shares EPT page table */ +if (iommu_use_hap_pt (d) ) +return 0 ; +/* Do nothing if hardware domain and iommu supports pass thru. */ if ( iommu_hwdom_passthrough &&is_hardware_domain (d) )return 0 ;
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v11 1/7 iommu: introduce the concept of DFN...
In reply to Paul Durrant...meaning 'device DMA frame number' i.e. a frame number mapped in the IOMMU (rather than the MMU) and hence used for DMA address translation.
This patch is a largely cosmetic change that substitutes the terms 'gfn' and 'gaddr' for 'dfn' and 'daddr' in all the places where the frame number or address relate to a device rather than the CPU.
The parts that are not purely cosmetic are:
- the introduction of a type-safe declaration of dfn_t and definition of INVALID_DFN to make the substitution of gfn_x(INVALID_GFN) mechanical.
- the introduction of __dfn_to_daddr and __daddr_to_dfn (and type-safe variants without the leading __) with some use of the former.
Subsequent patches will convert code to make use of type-safe DFNs.
---
Cc: Wei Liu <[email protected]>
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Julien Grall <[email protected]>
v9:
- Re-word comment in mm.h.
- Move definitions relating to daddr into asm-x86/iommu.h since these are not used by any ARM IOMMU implementation.
- Fix __daddr_to_dfn() to properly parenthesize and remove cast.
v8:
- Correct definition of INVALID_DFN.
- Don't use _AC in definition of IOMMU_PAGE_SIZE.
v7:
- Re-name BFN -> DFN as requested by Jan.
- Dropped Wei's R-b because of name change.
v6:
- Dropped changes to 'mfn' section in xen/mm.h as suggested by Kevin.
v3:
- Get rid of intermediate 'frame' variables again.
v2:
- Addressed comments from Jan.
--- xen/drivers/passthrough/amd/iommu_cmd.c | 18 +++---- xen/drivers/passthrough/amd/iommu_map.c | 78 ++++++++++++++--------------- xen/drivers/passthrough/amd/pci_amd_iommu.c | 2 +- xen/drivers/passthrough/arm/smmu.c | 16 +++--- xen/drivers/passthrough/iommu.c | 28 +++++------ xen/drivers/passthrough/vtd/iommu.c | 30 +++++------ xen/include/asm-x86/iommu.h | 12 +++++ xen/include/xen/iommu.h | 26 +++++++--- xen/include/xen/mm.h | 5 ++ 9 files changed, 123 insertions(+), 92 deletions(-)
xen/drivers/passthrough/amd/iommu_cmd.c +9 -9
@@ -284,7 +284,7 @@ void invalidate_iommu_all(struct amd_iommu *iommu) }void amd_iommu_flush_iotlb (u8 devfn,const struct pci_dev *pdev, -uint64_t gaddr,unsigned int order) +daddr_t daddr,unsigned int order) {unsigned long flags;struct amd_iommu *iommu;
@@ -315,12 +315,12 @@ void amd_iommu_flush_iotlb(u8 devfn, const struct pci_dev *pdev,/* send INVALIDATE_IOTLB_PAGES command */ spin_lock_irqsave (&iommu->lock, flags); -invalidate_iotlb_pages (iommu, maxpend,0 , queueid, gaddr, req_id, order); +invalidate_iotlb_pages (iommu, maxpend,0 , queueid, daddr, req_id, order);flush_command_buffer (iommu);spin_unlock_irqrestore (&iommu->lock, flags); } -static void amd_iommu_flush_all_iotlbs (struct domain *d,uint64_t gaddr, +static void amd_iommu_flush_all_iotlbs (struct domain *d,daddr_t daddr,unsigned int order) {struct pci_dev *pdev;
@@ -333,7 +333,7 @@ static void amd_iommu_flush_all_iotlbs(struct domain *d, uint64_t gaddr,u8 devfn = pdev->devfn;do { -amd_iommu_flush_iotlb (devfn, pdev, gaddr, order); +amd_iommu_flush_iotlb (devfn, pdev, daddr, order); devfn += pdev->phantom_stride; }while ( devfn != pdev->devfn &&PCI_SLOT (devfn) ==PCI_SLOT (pdev->devfn) );
@@ -342,7 +342,7 @@ static void amd_iommu_flush_all_iotlbs(struct domain *d, uint64_t gaddr,/* Flush iommu cache after p2m changes. */ static void _amd_iommu_flush_pages (struct domain *d, -uint64_t gaddr,unsigned int order) +daddr_t daddr,unsigned int order) {unsigned long flags;struct amd_iommu *iommu;
@@ -352,13 +352,13 @@ static void _amd_iommu_flush_pages(struct domain *d,for_each_amd_iommu ( iommu ) {spin_lock_irqsave (&iommu->lock, flags); -invalidate_iommu_pages (iommu, gaddr, dom_id, order); +invalidate_iommu_pages (iommu, daddr, dom_id, order);flush_command_buffer (iommu);spin_unlock_irqrestore (&iommu->lock, flags); }if ( ats_enabled ) -amd_iommu_flush_all_iotlbs (d, gaddr, order); +amd_iommu_flush_all_iotlbs (d, daddr, order); }void amd_iommu_flush_all_pages (struct domain *d)
@@ -367,9 +367,9 @@ void amd_iommu_flush_all_pages(struct domain *d) }void amd_iommu_flush_pages (struct domain *d, -unsigned long gfn,unsigned int order) +unsigned long dfn,unsigned int order) { -_amd_iommu_flush_pages (d, (uint64_t ) gfn <<PAGE_SHIFT , order); +_amd_iommu_flush_pages (d,__dfn_to_daddr (dfn), order); }void amd_iommu_flush_device (struct amd_iommu *iommu,uint16_t bdf)
xen/drivers/passthrough/amd/iommu_map.c +39 -39
259 lines not on this page · raw message
xen/drivers/passthrough/amd/pci_amd_iommu.c +1 -1
@@ -585,7 +585,7 @@ static void amd_dump_p2m_table_level(struct page_info* pg, int level,maddr_to_page (next_table_maddr), next_level, address, indent + 1); else - printk("%*sgfn: %08lx mfn: %08lx\n", + printk("%*sdfn: %08lx mfn: %08lx\n", indent,"" , (unsigned long )PFN_DOWN (address), (unsigned long)PFN_DOWN(next_table_maddr));
xen/drivers/passthrough/arm/smmu.c +8 -8
@@ -2551,7 +2551,7 @@ static int __must_check arm_smmu_iotlb_flush_all(struct domain *d) }static int __must_check arm_smmu_iotlb_flush (struct domain *d, -unsigned long gfn, +unsigned long dfn,unsigned int page_count) {/* ARM SMMU v1 doesn't have flush by VMA and VMID */
@@ -2746,7 +2746,7 @@ static void arm_smmu_iommu_domain_teardown(struct domain *d)xfree (xen_domain); } -static int __must_check arm_smmu_map_page (struct domain *d,unsigned long gfn, +static int __must_check arm_smmu_map_page (struct domain *d,unsigned long dfn,unsigned long mfn,unsigned int flags) {p2m_type_t t;
@@ -2757,10 +2757,10 @@ static int __must_check arm_smmu_map_page(struct domain *d, unsigned long gfn,* protected by an IOMMU, Xen needs to add a 1:1 mapping in the domain * p2m to allow DMA request to work. * This is only valid when the domain is directed mapped. Hence this -* function should only be used by gnttab code with gfn == mfn. +* function should only be used by gnttab code with gfn == mfn == dfn. */ BUG_ON (!is_domain_direct_mapped (d)); -BUG_ON (mfn != gfn); +BUG_ON (mfn != dfn);/* We only support readable and writable flags */ if (!(flags & (IOMMUF_readable | IOMMUF_writable)))
@@ -2772,19 +2772,19 @@ static int __must_check arm_smmu_map_page(struct domain *d, unsigned long gfn,* The function guest_physmap_add_entry replaces the current mapping * if there is already one... */ -return guest_physmap_add_entry (d,_gfn (gfn),_mfn (mfn),0 , t); +return guest_physmap_add_entry (d,_gfn (dfn),_mfn (dfn),0 , t); } -static int __must_check arm_smmu_unmap_page (struct domain *d,unsigned long gfn) +static int __must_check arm_smmu_unmap_page (struct domain *d,unsigned long dfn) {/* * This function should only be used by gnttab code when the domain -* is direct mapped +* is direct mapped (i.e. gfn == mfn == dfn). */ if ( !is_domain_direct_mapped (d) )return -EINVAL ; -return guest_physmap_remove_page (d,_gfn (gfn),_mfn (gfn),0 ); +return guest_physmap_remove_page (d,_gfn (dfn),_mfn (dfn),0 ); }static const struct iommu_ops arm_smmu_iommu_ops = {
xen/drivers/passthrough/iommu.c +14 -14
@@ -215,7 +215,7 @@ void __hwdom_init iommu_hwdom_init(struct domain *d) page_list_for_each ( page, &d->page_list ) {unsigned long mfn =mfn_x (page_to_mfn (page)); -unsigned long gfn =mfn_to_gmfn (d, mfn); +unsigned long dfn =mfn_to_gmfn (d, mfn);unsigned int mapping = IOMMUF_readable;int ret;
@@ -224,7 +224,7 @@ void __hwdom_init iommu_hwdom_init(struct domain *d) == PGT_writable_page) ) mapping |= IOMMUF_writable; - ret = hd->platform_ops->map_page (d, gfn, mfn, mapping); + ret = hd->platform_ops->map_page (d, dfn, mfn, mapping);if ( !rc ) rc = ret;
@@ -295,7 +295,7 @@ void iommu_domain_destroy(struct domain *d)arch_iommu_domain_destroy (d); } -int iommu_map_page (struct domain *d,unsigned long gfn,unsigned long mfn, +int iommu_map_page (struct domain *d,unsigned long dfn,unsigned long mfn,unsigned int flags) {const struct domain_iommu *hd =dom_iommu (d);
@@ -304,13 +304,13 @@ int iommu_map_page(struct domain *d, unsigned long gfn, unsigned long mfn,if ( !iommu_enabled || !hd->platform_ops )return 0 ; - rc = hd->platform_ops->map_page (d, gfn, mfn, flags); + rc = hd->platform_ops->map_page (d, dfn, mfn, flags);if (unlikely (rc) ) {if ( !d->is_shutting_down &&printk_ratelimit () )printk (XENLOG_ERR -"d%d: IOMMU mapping gfn %#lx to mfn %#lx failed: %d\n" , - d->domain_id, gfn, mfn, rc); +"d%d: IOMMU mapping dfn %#lx to mfn %#lx failed: %d\n" , + d->domain_id, dfn, mfn, rc);if ( !is_hardware_domain (d) )domain_crash (d);
@@ -319,7 +319,7 @@ int iommu_map_page(struct domain *d, unsigned long gfn, unsigned long mfn,return rc; } -int iommu_unmap_page (struct domain *d,unsigned long gfn) +int iommu_unmap_page (struct domain *d,unsigned long dfn) {const struct domain_iommu *hd =dom_iommu (d);int rc;
@@ -327,13 +327,13 @@ int iommu_unmap_page(struct domain *d, unsigned long gfn)if ( !iommu_enabled || !hd->platform_ops )return 0 ; - rc = hd->platform_ops->unmap_page (d, gfn); + rc = hd->platform_ops->unmap_page (d, dfn);if (unlikely (rc) ) {if ( !d->is_shutting_down &&printk_ratelimit () )printk (XENLOG_ERR -"d%d: IOMMU unmapping gfn %#lx failed: %d\n" , - d->domain_id, gfn, rc); +"d%d: IOMMU unmapping dfn %#lx failed: %d\n" , + d->domain_id, dfn, rc);if ( !is_hardware_domain (d) )domain_crash (d);
@@ -359,7 +359,7 @@ static void iommu_free_pagetables(unsigned long unused) cpumask_cycle(smp_processor_id(), &cpu_online_map)); } -int iommu_iotlb_flush (struct domain *d,unsigned long gfn, +int iommu_iotlb_flush (struct domain *d,unsigned long dfn,unsigned int page_count) {const struct domain_iommu *hd =dom_iommu (d);
@@ -368,13 +368,13 @@ int iommu_iotlb_flush(struct domain *d, unsigned long gfn,if ( !iommu_enabled || !hd->platform_ops || !hd->platform_ops->iotlb_flush )return 0 ; - rc = hd->platform_ops->iotlb_flush (d, gfn, page_count); + rc = hd->platform_ops->iotlb_flush (d, dfn, page_count);if (unlikely (rc) ) {if ( !d->is_shutting_down &&printk_ratelimit () )printk (XENLOG_ERR -"d%d: IOMMU IOTLB flush failed: %d, gfn %#lx, page count %u\n" , - d->domain_id, rc, gfn, page_count); +"d%d: IOMMU IOTLB flush failed: %d, dfn %#lx, page count %u\n" , + d->domain_id, rc, dfn, page_count);if ( !is_hardware_domain (d) )domain_crash (d);
xen/drivers/passthrough/vtd/iommu.c +15 -15
@@ -585,7 +585,7 @@ static int __must_check iommu_flush_all(void) }static int __must_check iommu_flush_iotlb (struct domain *d, -unsigned long gfn, +unsigned long dfn,bool_t dma_old_pte_present,unsigned int page_count) {
@@ -612,12 +612,12 @@ static int __must_check iommu_flush_iotlb(struct domain *d,if ( iommu_domid ==-1 )continue ; -if ( page_count !=1 || gfn ==gfn_x (INVALID_GFN ) ) +if ( page_count !=1 || dfn ==dfn_x (INVALID_DFN ) ) rc =iommu_flush_iotlb_dsi (iommu, iommu_domid,0 , flush_dev_iotlb);else rc =iommu_flush_iotlb_psi (iommu, iommu_domid, - (paddr_t )gfn <<PAGE_SHIFT_4K , +__dfn_to_daddr (dfn),PAGE_ORDER_4K , !dma_old_pte_present, flush_dev_iotlb);
@@ -633,15 +633,15 @@ static int __must_check iommu_flush_iotlb(struct domain *d, }static int __must_check iommu_flush_iotlb_pages (struct domain *d, -unsigned long gfn, +unsigned long dfn,unsigned int page_count) { -return iommu_flush_iotlb (d, gfn,1 , page_count); +return iommu_flush_iotlb (d, dfn,1 , page_count); }static int __must_check iommu_flush_iotlb_all (struct domain *d) { -return iommu_flush_iotlb (d,gfn_x (INVALID_GFN ),0 ,0 ); +return iommu_flush_iotlb (d,dfn_x (INVALID_DFN ),0 ,0 ); }/* clear one page's page table */
@@ -1768,7 +1768,7 @@ static void iommu_domain_teardown(struct domain *d) }static int __must_check intel_iommu_map_page (struct domain *d, -unsigned long gfn, +unsigned long dfn,unsigned long mfn,unsigned int flags) {
@@ -1787,14 +1787,14 @@ static int __must_check intel_iommu_map_page(struct domain *d,spin_lock (&hd->arch.mapping_lock); - pg_maddr =addr_to_dma_page_maddr (d, (paddr_t )gfn <<PAGE_SHIFT_4K ,1 ); + pg_maddr =addr_to_dma_page_maddr (d,__dfn_to_daddr (dfn),1 );if ( pg_maddr ==0 ) {spin_unlock (&hd->arch.mapping_lock);return -ENOMEM ; } page = (struct dma_pte *)map_vtd_domain_page (pg_maddr); - pte = page + (gfn &LEVEL_MASK ); + pte = page + (dfn &LEVEL_MASK ); old = *pte;dma_set_pte_addr (new, (paddr_t )mfn <<PAGE_SHIFT_4K ); dma_set_pte_prot(new,
@@ -1818,22 +1818,22 @@ static int __must_check intel_iommu_map_page(struct domain *d,unmap_vtd_domain_page (page);if ( !this_cpu (iommu_dont_flush_iotlb) ) - rc =iommu_flush_iotlb (d, gfn,dma_pte_present (old),1 ); + rc =iommu_flush_iotlb (d, dfn,dma_pte_present (old),1 );return rc; }static int __must_check intel_iommu_unmap_page (struct domain *d, -unsigned long gfn) +unsigned long dfn) {/* Do nothing if hardware domain and iommu supports pass thru. */ if ( iommu_hwdom_passthrough &&is_hardware_domain (d) )return 0 ; -return dma_pte_clear_one (d, (paddr_t )gfn <<PAGE_SHIFT_4K ); +return dma_pte_clear_one (d,__dfn_to_daddr (dfn)); } -int iommu_pte_flush (struct domain *d,u64 gfn,u64 *pte, +int iommu_pte_flush (struct domain *d,uint64_t dfn,uint64_t *pte,int order,int present) {struct acpi_drhd_unit *drhd;
@@ -1857,7 +1857,7 @@ int iommu_pte_flush(struct domain *d, u64 gfn, u64 *pte,continue ; rc =iommu_flush_iotlb_psi (iommu, iommu_domid, - (paddr_t )gfn <<PAGE_SHIFT_4K , +__dfn_to_daddr (dfn), order, !present, flush_dev_iotlb);if ( rc >0 ) {
@@ -2627,7 +2627,7 @@ static void vtd_dump_p2m_table_level(paddr_t pt_maddr, int level, paddr_t gpa,vtd_dump_p2m_table_level (dma_pte_addr (*pte), next_level, address, indent +1 ); else - printk("%*sgfn: %08lx mfn: %08lx\n", + printk("%*sdfn: %08lx mfn: %08lx\n", indent,"" , (unsigned long )(address >>PAGE_SHIFT_4K ), (unsigned long)(dma_pte_addr(*pte) >> PAGE_SHIFT_4K));
xen/include/asm-x86/iommu.h +12 -0
@@ -30,6 +30,18 @@ struct g2m_ioport {unsigned int np; }; +#define IOMMU_PAGE_SHIFT 12 +#define IOMMU_PAGE_SIZE (1 << IOMMU_PAGE_SHIFT) +#define IOMMU_PAGE_MASK (~(IOMMU_PAGE_SIZE - 1)) + +typedef uint64_t daddr_t ; + +#define __dfn_to_daddr (dfn) ((daddr_t)(dfn) << IOMMU_PAGE_SHIFT) +#define __daddr_to_dfn (daddr) ((daddr) >> IOMMU_PAGE_SHIFT) + +#define dfn_to_daddr (dfn) __dfn_to_daddr(dfn_x(dfn)) +#define daddr_to_dfn (daddr) _dfn(__daddr_to_dfn(daddr)) +struct arch_iommu {u64 pgd_maddr;/* io page directory machine address */
xen/include/xen/iommu.h +20 -6
@@ -23,11 +23,25 @@#include <xen/page-defs.h> #include <xen/spinlock.h> #include <xen/pci.h> +#include <xen/typesafe.h> #include <public/hvm/ioreq.h> #include <public/domctl.h> #include <asm/device.h> #include <asm/iommu.h> +TYPE_SAFE (uint64_t, dfn); +#define PRI_dfn PRIx64 +#define INVALID_DFN _dfn(~0ULL) + +#ifndef dfn_t +#define dfn_t/* Grep fodder: dfn_t, _dfn() and dfn_x() are defined above */ +#define _dfn +#define dfn_x +#undef dfn_t +#undef _dfn +#undef dfn_x +#endif +extern bool_t iommu_enable, iommu_enabled;extern bool_t force_iommu, iommu_verbose;extern bool_t iommu_workaround_bios_bug, iommu_igfx;
@@ -64,9 +78,9 @@ void iommu_teardown(struct domain *d);#define IOMMUF_readable (1u<<_IOMMUF_readable)#define _IOMMUF_writable 1#define IOMMUF_writable (1u<<_IOMMUF_writable) -int __must_check iommu_map_page (struct domain *d,unsigned long gfn, +int __must_check iommu_map_page (struct domain *d,unsigned long dfn,unsigned long mfn,unsigned int flags); -int __must_check iommu_unmap_page (struct domain *d,unsigned long gfn); +int __must_check iommu_unmap_page (struct domain *d,unsigned long dfn);enum iommu_feature {
@@ -154,9 +168,9 @@ struct iommu_ops {#endif /* HAS_PCI */ void (*teardown)(struct domain *d); -int __must_check (*map_page)(struct domain *d,unsigned long gfn, +int __must_check (*map_page)(struct domain *d,unsigned long dfn,unsigned long mfn,unsigned int flags); -int __must_check (*unmap_page)(struct domain *d,unsigned long gfn); +int __must_check (*unmap_page)(struct domain *d,unsigned long dfn);void (*free_page_table)(struct page_info *);#ifdef CONFIG_X86 void (*update_ire_from_apic)(unsigned int apic,unsigned int reg,unsigned int value);
@@ -167,7 +181,7 @@ struct iommu_ops {void (*resume)(void );void (*share_p2m)(struct domain *d);void (*crash_shutdown)(void ); -int __must_check (*iotlb_flush)(struct domain *d,unsigned long gfn, +int __must_check (*iotlb_flush)(struct domain *d,unsigned long dfn,unsigned int page_count);int __must_check (*iotlb_flush_all)(struct domain *d);int (*get_reserved_device_memory)(iommu_grdm_t *,void *);
@@ -189,7 +203,7 @@ int iommu_do_pci_domctl(struct xen_domctl *, struct domain *d,int iommu_do_domctl (struct xen_domctl *,struct domain *d,XEN_GUEST_HANDLE_PARAM (xen_domctl_t )); -int __must_check iommu_iotlb_flush (struct domain *d,unsigned long gfn, +int __must_check iommu_iotlb_flush (struct domain *d,unsigned long dfn,unsigned int page_count);int __must_check iommu_iotlb_flush_all (struct domain *d);
xen/include/xen/mm.h +5 -0
@@ -26,6 +26,11 @@* A linear idea of a guest physical address space. For an auto-translated * guest, pfn == gfn while for a non-translated guest, pfn != gfn. * +* dfn: Device DMA Frame Number (definitions in include/xen/iommu.h) +* The linear frame numbers of device DMA address space. All initiators for +* (i.e. all devices assigned to) a guest share a single DMA address space +* and, by default, Xen will ensure dfn == pfn. +* * WARNING: Some of these terms have changed over time while others have been * used inconsistently, meaning that a lot of existing code does not match the * definitions above. New code should use these terms as described here, and
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v11 3/7 iommu: push use of type-safe DFN and MFN into iommu_ops
In reply to Paul DurrantThis patch modifies the methods in struct iommu_ops to use type-safe DFN and MFN. This follows on from the prior patch that modified the functions exported in xen/iommu.h.
---
Cc: Suravee Suthikulpanit <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
v9:
- Re-base.
v7:
- Re-base and re-name BFN -> DFN.
- Added Jan's A-b since re-naming was purely mechanical.
v6:
- Re-base.
v3:
- Remove some use of intermediate 'frame' variables.
v2:
- Addressed comments from Jan.
- Extend use of intermediate 'frame' variable to avoid directly encapsulating gfn values as bfns (now dfns) or vice versa.
--- xen/drivers/passthrough/amd/iommu_map.c | 46 ++++++++++++++++----------- xen/drivers/passthrough/amd/pci_amd_iommu.c | 2 +- xen/drivers/passthrough/arm/smmu.c | 16 +++++----- xen/drivers/passthrough/iommu.c | 9 +++--- xen/drivers/passthrough/vtd/iommu.c | 26 +++++++-------- xen/drivers/passthrough/x86/iommu.c | 2 +- xen/include/asm-x86/hvm/svm/amd-iommu-proto.h | 8 ++--- xen/include/xen/iommu.h | 13 +++++--- 8 files changed, 67 insertions(+), 55 deletions(-)
xen/drivers/passthrough/amd/iommu_map.c +27 -19
@@ -631,7 +631,7 @@ static int update_paging_mode(struct domain *d, unsigned long dfn)return 0 ; } -int amd_iommu_map_page (struct domain *d,unsigned long dfn,unsigned long mfn, +int amd_iommu_map_page (struct domain *d,dfn_t dfn,mfn_t mfn,unsigned int flags) {bool_t need_flush =0 ;
@@ -651,7 +651,8 @@ int amd_iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,if ( rc ) {spin_unlock (&hd->arch.mapping_lock); -AMD_IOMMU_DEBUG ("Root table alloc failed, dfn = %lx\n" , dfn); +AMD_IOMMU_DEBUG ("Root table alloc failed, dfn = %" PRI_dfn"\n" , +dfn_x (dfn));domain_crash (d);return rc; }
@@ -660,25 +661,27 @@ int amd_iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,* we might need a deeper page table for wider dfn now */ if (is_hvm_domain (d) ) { -if (update_paging_mode (d, dfn) ) +if (update_paging_mode (d,dfn_x (dfn)) ) {spin_unlock (&hd->arch.mapping_lock); -AMD_IOMMU_DEBUG ("Update page mode failed dfn = %lx\n" , dfn); +AMD_IOMMU_DEBUG ("Update page mode failed dfn = %" PRI_dfn"\n" , +dfn_x (dfn));domain_crash (d);return -EFAULT ; } } -if (iommu_pde_from_dfn (d, dfn, pt_mfn) || (pt_mfn[1 ] ==0 ) ) +if (iommu_pde_from_dfn (d,dfn_x (dfn), pt_mfn) || (pt_mfn[1 ] ==0 ) ) {spin_unlock (&hd->arch.mapping_lock); -AMD_IOMMU_DEBUG ("Invalid IO pagetable entry dfn = %lx\n" , dfn); +AMD_IOMMU_DEBUG ("Invalid IO pagetable entry dfn = %" PRI_dfn"\n" , +dfn_x (dfn));domain_crash (d);return -EFAULT ; }/* Install 4k mapping first */ - need_flush =set_iommu_pte_present (pt_mfn[1 ], dfn, mfn, + need_flush =set_iommu_pte_present (pt_mfn[1 ],dfn_x (dfn),mfn_x (mfn),IOMMU_PAGING_MODE_LEVEL_1 , !!(flags & IOMMUF_writable), !!(flags & IOMMUF_readable));
@@ -690,7 +693,7 @@ int amd_iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,/* 4K mapping for PV guests never changes, * no need to flush if we trust non-present bits */ if (is_hvm_domain (d) ) -amd_iommu_flush_pages (d, dfn,0 ); +amd_iommu_flush_pages (d,dfn_x (dfn),0 );for ( merge_level =IOMMU_PAGING_MODE_LEVEL_2 ; merge_level <= hd->arch.paging_mode; merge_level++ )
@@ -698,15 +701,16 @@ int amd_iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,if ( pt_mfn[merge_level] ==0 )break ;if ( !iommu_update_pde_count (d, pt_mfn[merge_level], - dfn, mfn, merge_level) ) +dfn_x (dfn),mfn_x (mfn), merge_level) )break ; -if (iommu_merge_pages (d, pt_mfn[merge_level], dfn, +if (iommu_merge_pages (d, pt_mfn[merge_level],dfn_x (dfn), flags, merge_level) ) {spin_unlock (&hd->arch.mapping_lock);AMD_IOMMU_DEBUG ("Merge iommu page failed at level %d, " -"dfn = %lx mfn = %lx\n" , merge_level, dfn, mfn); +"dfn = %" PRI_dfn" mfn = %" PRI_mfn"\n" , + merge_level,dfn_x (dfn),mfn_x (mfn));domain_crash (d);return -EFAULT ; }
@@ -720,7 +724,7 @@ out:return 0 ; } -int amd_iommu_unmap_page (struct domain *d,unsigned long dfn) +int amd_iommu_unmap_page (struct domain *d,dfn_t dfn) {unsigned long pt_mfn[7 ];struct domain_iommu *hd =dom_iommu (d);
@@ -742,31 +746,33 @@ int amd_iommu_unmap_page(struct domain *d, unsigned long dfn)* we might need a deeper page table for lager dfn now */ if (is_hvm_domain (d) ) { -int rc =update_paging_mode (d, dfn); +int rc =update_paging_mode (d,dfn_x (dfn));if ( rc ) {spin_unlock (&hd->arch.mapping_lock); -AMD_IOMMU_DEBUG ("Update page mode failed dfn = %lx\n" , dfn); +AMD_IOMMU_DEBUG ("Update page mode failed dfn = %" PRI_dfn"\n" , +dfn_x (dfn));if ( rc != -EADDRNOTAVAIL )domain_crash (d);return rc; } } -if (iommu_pde_from_dfn (d, dfn, pt_mfn) || (pt_mfn[1 ] ==0 ) ) +if (iommu_pde_from_dfn (d,dfn_x (dfn), pt_mfn) || (pt_mfn[1 ] ==0 ) ) {spin_unlock (&hd->arch.mapping_lock); -AMD_IOMMU_DEBUG ("Invalid IO pagetable entry dfn = %lx\n" , dfn); +AMD_IOMMU_DEBUG ("Invalid IO pagetable entry dfn = %" PRI_dfn"\n" , +dfn_x (dfn));domain_crash (d);return -EFAULT ; }/* mark PTE as 'page not present' */ -clear_iommu_pte_present (pt_mfn[1 ], dfn); +clear_iommu_pte_present (pt_mfn[1 ],dfn_x (dfn));spin_unlock (&hd->arch.mapping_lock); -amd_iommu_flush_pages (d, dfn,0 ); +amd_iommu_flush_pages (d,dfn_x (dfn),0 );return 0 ; }
@@ -787,7 +793,9 @@ int amd_iommu_reserve_domain_unity_map(struct domain *domain, gfn = phys_addr >>PAGE_SHIFT ;for ( i =0 ; i < npages; i++ ) { - rt =amd_iommu_map_page (domain, gfn +i, gfn +i, flags); +unsigned long frame = gfn + i; + + rt =amd_iommu_map_page (domain,_dfn (frame),_mfn (frame), flags);if ( rt !=0 )return rt; }
xen/drivers/passthrough/amd/pci_amd_iommu.c +1 -1
@@ -278,7 +278,7 @@ static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)*/ if (mfn_valid (_mfn (pfn)) ) { -int ret =amd_iommu_map_page (d, pfn, pfn, +int ret =amd_iommu_map_page (d,_dfn (pfn),_mfn (pfn), IOMMUF_readable|IOMMUF_writable);if ( !rc )
xen/drivers/passthrough/arm/smmu.c +8 -8
@@ -2550,8 +2550,7 @@ static int __must_check arm_smmu_iotlb_flush_all(struct domain *d)return 0 ; } -static int __must_check arm_smmu_iotlb_flush (struct domain *d, -unsigned long dfn, +static int __must_check arm_smmu_iotlb_flush (struct domain *d,dfn_t dfn,unsigned int page_count) {/* ARM SMMU v1 doesn't have flush by VMA and VMID */
@@ -2746,8 +2745,8 @@ static void arm_smmu_iommu_domain_teardown(struct domain *d)xfree (xen_domain); } -static int __must_check arm_smmu_map_page (struct domain *d,unsigned long dfn, -unsigned long mfn,unsigned int flags) +static int __must_check arm_smmu_map_page (struct domain *d,dfn_t dfn, +mfn_t mfn,unsigned int flags) {p2m_type_t t;
@@ -2760,7 +2759,7 @@ static int __must_check arm_smmu_map_page(struct domain *d, unsigned long dfn,* function should only be used by gnttab code with gfn == mfn == dfn. */ BUG_ON (!is_domain_direct_mapped (d)); -BUG_ON (mfn != dfn); +BUG_ON (mfn_x (mfn) !=dfn_x (dfn));/* We only support readable and writable flags */ if (!(flags & (IOMMUF_readable | IOMMUF_writable)))
@@ -2772,10 +2771,11 @@ static int __must_check arm_smmu_map_page(struct domain *d, unsigned long dfn,* The function guest_physmap_add_entry replaces the current mapping * if there is already one... */ -return guest_physmap_add_entry (d,_gfn (dfn),_mfn (dfn),0 , t); +return guest_physmap_add_entry (d,_gfn (dfn_x (dfn)),_mfn (dfn_x (dfn)), +0 , t); } -static int __must_check arm_smmu_unmap_page (struct domain *d,unsigned long dfn) +static int __must_check arm_smmu_unmap_page (struct domain *d,dfn_t dfn) {/* * This function should only be used by gnttab code when the domain
@@ -2784,7 +2784,7 @@ static int __must_check arm_smmu_unmap_page(struct domain *d, unsigned long dfn)if ( !is_domain_direct_mapped (d) )return -EINVAL ; -return guest_physmap_remove_page (d,_gfn (dfn),_mfn (dfn),0 ); +return guest_physmap_remove_page (d,_gfn (dfn_x (dfn)),_mfn (dfn_x (dfn)),0 ); }static const struct iommu_ops arm_smmu_iommu_ops = {
xen/drivers/passthrough/iommu.c +5 -4
@@ -224,7 +224,8 @@ void __hwdom_init iommu_hwdom_init(struct domain *d) == PGT_writable_page) ) mapping |= IOMMUF_writable; - ret = hd->platform_ops->map_page (d, dfn, mfn, mapping); + ret = hd->platform_ops->map_page (d,_dfn (dfn),_mfn (mfn), + mapping);if ( !rc ) rc = ret;
@@ -304,7 +305,7 @@ int iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,if ( !iommu_enabled || !hd->platform_ops )return 0 ; - rc = hd->platform_ops->map_page (d,dfn_x (dfn),mfn_x (mfn), flags); + rc = hd->platform_ops->map_page (d, dfn, mfn, flags);if (unlikely (rc) ) {if ( !d->is_shutting_down &&printk_ratelimit () )
@@ -327,7 +328,7 @@ int iommu_unmap_page(struct domain *d, dfn_t dfn)if ( !iommu_enabled || !hd->platform_ops )return 0 ; - rc = hd->platform_ops->unmap_page (d,dfn_x (dfn)); + rc = hd->platform_ops->unmap_page (d, dfn);if (unlikely (rc) ) {if ( !d->is_shutting_down &&printk_ratelimit () )
@@ -367,7 +368,7 @@ int iommu_iotlb_flush(struct domain *d, dfn_t dfn, unsigned int page_count)if ( !iommu_enabled || !hd->platform_ops || !hd->platform_ops->iotlb_flush )return 0 ; - rc = hd->platform_ops->iotlb_flush (d,dfn_x (dfn), page_count); + rc = hd->platform_ops->iotlb_flush (d, dfn, page_count);if (unlikely (rc) ) {if ( !d->is_shutting_down &&printk_ratelimit () )
xen/drivers/passthrough/vtd/iommu.c +12 -14
@@ -584,8 +584,7 @@ static int __must_check iommu_flush_all(void)return rc; } -static int __must_check iommu_flush_iotlb (struct domain *d, -unsigned long dfn, +static int __must_check iommu_flush_iotlb (struct domain *d,dfn_t dfn,bool_t dma_old_pte_present,unsigned int page_count) {
@@ -612,12 +611,12 @@ static int __must_check iommu_flush_iotlb(struct domain *d,if ( iommu_domid ==-1 )continue ; -if ( page_count !=1 || dfn ==dfn_x (INVALID_DFN ) ) +if ( page_count !=1 ||dfn_eq (dfn,INVALID_DFN ) ) rc =iommu_flush_iotlb_dsi (iommu, iommu_domid,0 , flush_dev_iotlb);else rc =iommu_flush_iotlb_psi (iommu, iommu_domid, -__dfn_to_daddr (dfn), +dfn_to_daddr (dfn),PAGE_ORDER_4K , !dma_old_pte_present, flush_dev_iotlb);
@@ -633,7 +632,7 @@ static int __must_check iommu_flush_iotlb(struct domain *d, }static int __must_check iommu_flush_iotlb_pages (struct domain *d, -unsigned long dfn, +dfn_t dfn,unsigned int page_count) {return iommu_flush_iotlb (d, dfn,1 , page_count);
@@ -641,7 +640,7 @@ static int __must_check iommu_flush_iotlb_pages(struct domain *d,static int __must_check iommu_flush_iotlb_all (struct domain *d) { -return iommu_flush_iotlb (d,dfn_x (INVALID_DFN ),0 ,0 ); +return iommu_flush_iotlb (d,INVALID_DFN ,0 ,0 ); }/* clear one page's page table */
@@ -676,7 +675,7 @@ static int __must_check dma_pte_clear_one(struct domain *domain, u64 addr)iommu_flush_cache_entry (pte,sizeof (struct dma_pte ));if ( !this_cpu (iommu_dont_flush_iotlb) ) - rc =iommu_flush_iotlb_pages (domain, addr >>PAGE_SHIFT_4K ,1 ); + rc =iommu_flush_iotlb_pages (domain,daddr_to_dfn (addr),1 );unmap_vtd_domain_page (page);
@@ -1768,8 +1767,7 @@ static void iommu_domain_teardown(struct domain *d) }static int __must_check intel_iommu_map_page (struct domain *d, -unsigned long dfn, -unsigned long mfn, +dfn_t dfn,mfn_t mfn,unsigned int flags) {struct domain_iommu *hd =dom_iommu (d);
@@ -1787,16 +1785,16 @@ static int __must_check intel_iommu_map_page(struct domain *d,spin_lock (&hd->arch.mapping_lock); - pg_maddr =addr_to_dma_page_maddr (d,__dfn_to_daddr (dfn),1 ); + pg_maddr =addr_to_dma_page_maddr (d,dfn_to_daddr (dfn),1 );if ( pg_maddr ==0 ) {spin_unlock (&hd->arch.mapping_lock);return -ENOMEM ; } page = (struct dma_pte *)map_vtd_domain_page (pg_maddr); - pte = page + (dfn &LEVEL_MASK ); + pte = page + (dfn_x (dfn) &LEVEL_MASK ); old = *pte; -dma_set_pte_addr (new, (paddr_t )mfn <<PAGE_SHIFT_4K ); +dma_set_pte_addr (new,mfn_to_maddr (mfn));dma_set_pte_prot (new, ((flags & IOMMUF_readable) ?DMA_PTE_READ :0 ) | ((flags & IOMMUF_writable) ?DMA_PTE_WRITE :0 ));
@@ -1824,13 +1822,13 @@ static int __must_check intel_iommu_map_page(struct domain *d, }static int __must_check intel_iommu_unmap_page (struct domain *d, -unsigned long dfn) +dfn_t dfn) {/* Do nothing if hardware domain and iommu supports pass thru. */ if ( iommu_hwdom_passthrough &&is_hardware_domain (d) )return 0 ; -return dma_pte_clear_one (d,__dfn_to_daddr (dfn)); +return dma_pte_clear_one (d,dfn_to_daddr (dfn)); } int iommu_pte_flush(struct domain *d, uint64_t dfn, uint64_t *pte,
xen/drivers/passthrough/x86/iommu.c +1 -1
@@ -68,7 +68,7 @@ int arch_iommu_populate_page_table(struct domain *d) {ASSERT (!(gfn >>DEFAULT_DOMAIN_ADDRESS_WIDTH ));BUG_ON (SHARED_M2P (gfn)); - rc = hd->platform_ops->map_page (d, gfn, mfn, + rc = hd->platform_ops->map_page (d,_dfn (gfn),_mfn (mfn), IOMMUF_readable | IOMMUF_writable); }
xen/include/asm-x86/hvm/svm/amd-iommu-proto.h +4 -4
@@ -52,9 +52,9 @@ int amd_iommu_init(void);int amd_iommu_update_ivrs_mapping_acpi (void );/* mapping functions */ -int __must_check amd_iommu_map_page (struct domain *d,unsigned long gfn, -unsigned long mfn,unsigned int flags); -int __must_check amd_iommu_unmap_page (struct domain *d,unsigned long gfn); +int __must_check amd_iommu_map_page (struct domain *d,dfn_t dfn, +mfn_t mfn,unsigned int flags); +int __must_check amd_iommu_unmap_page (struct domain *d,dfn_t dfn);u64 amd_iommu_get_next_table_from_pte (u32 *entry);int __must_check amd_iommu_alloc_root (struct domain_iommu *hd); int amd_iommu_reserve_domain_unity_map(struct domain *domain,
@@ -77,7 +77,7 @@ void iommu_dte_set_guest_cr3(u32 *dte, u16 dom_id, u64 gcr3,/* send cmd to iommu */ void amd_iommu_flush_all_pages (struct domain *d); -void amd_iommu_flush_pages (struct domain *d,unsigned long gfn, +void amd_iommu_flush_pages (struct domain *d,unsigned long dfn,unsigned int order);void amd_iommu_flush_iotlb (u8 devfn,const struct pci_dev *pdev,uint64_t gaddr,unsigned int order);
xen/include/xen/iommu.h +9 -4
@@ -48,6 +48,11 @@ static inline dfn_t dfn_add(dfn_t dfn, unsigned long i)return _dfn (dfn_x (dfn) + i); } +static inline bool_t dfn_eq (dfn_t x,dfn_t y) +{ +return dfn_x (x) ==dfn_x (y); +} +extern bool_t iommu_enable, iommu_enabled;extern bool_t force_iommu, iommu_verbose;extern bool_t iommu_workaround_bios_bug, iommu_igfx;
@@ -174,9 +179,9 @@ struct iommu_ops {#endif /* HAS_PCI */ void (*teardown)(struct domain *d); -int __must_check (*map_page)(struct domain *d,unsigned long dfn, -unsigned long mfn,unsigned int flags); -int __must_check (*unmap_page)(struct domain *d,unsigned long dfn); +int __must_check (*map_page)(struct domain *d,dfn_t dfn,mfn_t mfn, +unsigned int flags); +int __must_check (*unmap_page)(struct domain *d,dfn_t dfn);void (*free_page_table)(struct page_info *);#ifdef CONFIG_X86 void (*update_ire_from_apic)(unsigned int apic,unsigned int reg,unsigned int value);
@@ -187,7 +192,7 @@ struct iommu_ops {void (*resume)(void );void (*share_p2m)(struct domain *d);void (*crash_shutdown)(void ); -int __must_check (*iotlb_flush)(struct domain *d,unsigned long dfn, +int __must_check (*iotlb_flush)(struct domain *d,dfn_t dfn,unsigned int page_count);int __must_check (*iotlb_flush_all)(struct domain *d);int (*get_reserved_device_memory)(iommu_grdm_t *,void *);
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v11 4/7 iommu: don't domain_crash() inside iommu_map/unmap_page()
In reply to Paul DurrantThis patch removes the implicit domain_crash() from iommu_map(), unmap_page() and iommu_iotlb_flush() and turns them into straightforward wrappers that check the existence of the relevant iommu_op and call through to it. This makes them usable by PV IOMMU code to be delivered in future patches.
This patch adds a helper macro, domu_crash(), that will only invoke domain_crash() if the domain is not the hardware domain and modifies callers of iommu_map(), unmap_page() and iommu_iotlb_flush() to use this should an operation fail.
NOTE: This patch includes one bit of clean-up in set_identity_p2m_entry()
replacing use of p2m->domain with the domain pointer passed into the
function.---
Cc: Wei Liu <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: Jun Nakajima <[email protected]>
Cc: George Dunlap <[email protected]>
v7:
- Re-base and re-name BFN -> DFN.
- Move domu_crash() outside double locked region in grant_table.c.
- Added Jan's R-b.
v6:
- Introduce domu_crash() (idea suggested by Kevin, name suggested by Jan) to crash non-hardware domains.
- Dropped Wei's and George's R-b because of change.
v2:
- New in v2.
--- xen/arch/arm/p2m.c | 4 ++++ xen/arch/x86/mm.c | 3 +++ xen/arch/x86/mm/p2m-ept.c | 3 +++ xen/arch/x86/mm/p2m-pt.c | 3 +++ xen/arch/x86/mm/p2m.c | 22 ++++++++++++++++++---- xen/common/grant_table.c | 4 ++++ xen/common/memory.c | 3 +++ xen/drivers/passthrough/iommu.c | 12 ------------ xen/drivers/passthrough/x86/iommu.c | 4 ++++ xen/include/xen/sched.h | 5 +++++ 10 files changed, 47 insertions(+), 16 deletions(-)
xen/arch/arm/p2m.c +4 -0
@@ -957,8 +957,12 @@ static int __p2m_set_entry(struct p2m_domain *p2m,if (need_iommu (p2m->domain) && (lpae_is_valid (orig_pte) ||lpae_is_valid (*entry)) ) + { rc =iommu_iotlb_flush (p2m->domain,_dfn (gfn_x (sgfn)),1UL << page_order); +if (unlikely (rc) ) +domu_crash (p2m->domain); + }else rc =0 ;
xen/arch/x86/mm.c +3 -0
@@ -2807,6 +2807,9 @@ static int _get_page_type(struct page_info *page, unsigned long type, iommu_ret =iommu_map_page (d,_dfn (mfn_x (mfn)), mfn, IOMMUF_readable | IOMMUF_writable); + +if (unlikely (iommu_ret) ) +domu_crash (d); } }
xen/arch/x86/mm/p2m-ept.c +3 -0
@@ -895,6 +895,9 @@ out:if ( !rc ) rc = ret; } + +if (unlikely (rc) ) +domu_crash (d); } }
xen/arch/x86/mm/p2m-pt.c +3 -0
@@ -717,6 +717,9 @@ p2m_pt_set_entry(struct p2m_domain *p2m, gfn_t gfn_, mfn_t mfn, rc = ret; } } + +if (unlikely (rc) ) +domu_crash (p2m->domain); }/*
xen/arch/x86/mm/p2m.c +18 -4
@@ -724,6 +724,9 @@ p2m_remove_page(struct p2m_domain *p2m, unsigned long gfn_l, unsigned long mfn,if ( !rc ) rc = ret; } + +if (unlikely (rc) ) +domu_crash (p2m->domain); }return rc;
@@ -789,6 +792,7 @@ guest_physmap_add_entry(struct domain *d, gfn_t gfn, mfn_t mfn,if (iommu_unmap_page (d,dfn_add (dfn, i)) )continue ; +domu_crash (d);return rc; } }
@@ -1157,12 +1161,17 @@ int set_identity_p2m_entry(struct domain *d, unsigned long gfn_l,struct p2m_domain *p2m =p2m_get_hostp2m (d);int ret; -if ( !paging_mode_translate (p2m->domain) ) +if ( !paging_mode_translate (d) ) {if ( !need_iommu (d) )return 0 ; -return iommu_map_page (d,_dfn (gfn_l),_mfn (gfn_l), - IOMMUF_readable | IOMMUF_writable); + + ret =iommu_map_page (d,_dfn (gfn_l),_mfn (gfn_l), + IOMMUF_readable | IOMMUF_writable); +if (unlikely (ret) ) +domu_crash (d); + +return ret; }gfn_lock (p2m, gfn,0 );
@@ -1252,7 +1261,12 @@ int clear_identity_p2m_entry(struct domain *d, unsigned long gfn_l) {if ( !need_iommu (d) )return 0 ; -return iommu_unmap_page (d,_dfn (gfn_l)); + + ret =iommu_unmap_page (d,_dfn (gfn_l)); +if (unlikely (ret) ) +domu_crash (d); + +return ret; }gfn_lock (p2m, gfn,0 );
xen/common/grant_table.c +4 -0
@@ -1155,6 +1155,7 @@ map_grant_ref(if ( err ) {double_gt_unlock (lgt, rgt); +domu_crash (ld); rc = GNTST_general_error; goto undo_out; }
@@ -1406,7 +1407,10 @@ unmap_common(double_gt_unlock (lgt, rgt);if ( err ) + { +domu_crash (ld); rc = GNTST_general_error; + } }/* If just unmapped a writable mapping, mark as dirtied */
xen/common/memory.c +3 -0
@@ -841,6 +841,9 @@ int xenmem_add_to_physmap(struct domain *d, struct xen_add_to_physmap *xatp, ret =iommu_iotlb_flush (d,_dfn (xatp->gpfn - done), done);if (unlikely (ret) && rc >=0 ) rc = ret; + +if (unlikely (rc <0 ) ) + domu_crash(d); }#endif
xen/drivers/passthrough/iommu.c +0 -12
@@ -312,9 +312,6 @@ int iommu_map_page(struct domain *d, dfn_t dfn, mfn_t mfn,printk (XENLOG_ERR "d%d: IOMMU mapping dfn %" PRI_dfn" to mfn %" PRI_mfn" failed: %d\n" , d->domain_id,dfn_x (dfn),mfn_x (mfn), rc); - -if ( !is_hardware_domain (d) ) -domain_crash (d); }return rc;
@@ -335,9 +332,6 @@ int iommu_unmap_page(struct domain *d, dfn_t dfn)printk (XENLOG_ERR "d%d: IOMMU unmapping dfn %" PRI_dfn" failed: %d\n" , d->domain_id,dfn_x (dfn), rc); - -if ( !is_hardware_domain (d) ) -domain_crash (d); }return rc;
@@ -375,9 +369,6 @@ int iommu_iotlb_flush(struct domain *d, dfn_t dfn, unsigned int page_count)printk (XENLOG_ERR "d%d: IOMMU IOTLB flush failed: %d, dfn %" PRI_dfn", page count %u\n" , d->domain_id, rc,dfn_x (dfn), page_count); - -if ( !is_hardware_domain (d) ) -domain_crash (d); }return rc;
@@ -398,9 +389,6 @@ int iommu_iotlb_flush_all(struct domain *d)printk (XENLOG_ERR "d%d: IOMMU IOTLB flush all failed: %d\n" , d->domain_id, rc); - -if ( !is_hardware_domain (d) ) -domain_crash (d); }return rc;
xen/drivers/passthrough/x86/iommu.c +4 -0
@@ -107,7 +107,11 @@ int arch_iommu_populate_page_table(struct domain *d)this_cpu (iommu_dont_flush_iotlb) =0 ;if ( !rc ) + { rc =iommu_iotlb_flush_all (d); +if (unlikely (rc) ) +domain_crash (d); + }if ( rc && rc != -ERESTART )iommu_teardown (d);
xen/include/xen/sched.h +5 -0
@@ -616,6 +616,11 @@ void __domain_crash(struct domain *d);__domain_crash (d); \ } while (0) +#define domu_crash (d) do { \ + if ( !is_hardware_domain(d) ) \ + domain_crash(d); \ +} while (false) +/* * Called from assembly code, with an optional address to help indicate why * the crash occured. If addr is 0, look up address from last extable
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v11 5/7 memory: add check_get_page_from_gfn() as a wrapper...
In reply to Paul Durrant...for some uses of get_page_from_gfn().
There are many occurrences of the following pattern in the code:
q = <readonly look-up> ? P2M_ALLOC : P2M_UNSHARE;
page = get_page_from_gfn(d, gfn, &p2mt, q);
if ( p2m_is_paging(p2mt) )
{
if ( page )
put_page(page);
p2m_mem_paging_populate(d, gfn);
return <-EAGAIN or equivalent>;
}
if ( (q & P2M_UNSHARE) && p2m_is_shared(p2mt) )
{
if ( page )
put_page(page);
return <-EAGAIN or equivalent>;
}
if ( !page )
return <-EINVAL or equivalent>;There are some small differences between the exact way the occurrences are coded but the desired semantic is the same.
This patch introduces a new common implementation of this code in check_get_page_from_gfn() and then converts the various open-coded patterns into calls to this new function.
---
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: Wei Liu <[email protected]>
v11:
- Forward declare p2m_type_t in p2m-common.h to allow the duplicate declarations of check_get_page_from_gfn() to be removed, and hence add Jan's R-b.
v10:
- Expand commit comment to point out the reason for the duplicate declarations of check_get_page_from_gfn().
v9:
- Defer P2M type checks (beyond shared or paging) to the caller.
v7:
- Fix ARM build by introducing p2m_is_readonly() predicate.
- Re-name get_paged_frame() -> check_get_page_from_gfn().
- Adjust default cases of callers switch-ing on return value.
v3:
- Addressed comments from George.
v2:
- New in v2.
--- xen/arch/x86/hvm/emulate.c | 25 +++++++++++----------- xen/arch/x86/hvm/hvm.c | 14 +------------ xen/common/grant_table.c | 32 ++++++++++++++--------------- xen/common/memory.c | 49 +++++++++++++++++++++++++++++++++++--------- xen/include/asm-arm/p2m.h | 4 ++-- xen/include/asm-x86/p2m.h | 5 ++--- xen/include/xen/p2m-common.h | 6 ++++++ 7 files changed, 77 insertions(+), 58 deletions(-)
xen/arch/x86/hvm/emulate.c +12 -13
@@ -356,22 +356,21 @@ static int hvmemul_acquire_page(unsigned long gmfn, struct page_info **page)struct domain *curr_d = current->domain;p2m_type_t p2mt; - *page =get_page_from_gfn (curr_d, gmfn, &p2mt,P2M_UNSHARE ); - -if ( *page ==NULL ) -return X86EMUL_UNHANDLEABLE ; - -if (p2m_is_paging (p2mt) ) +switch (check_get_page_from_gfn (curr_d,_gfn (gmfn), false, &p2mt, + page) ) { -put_page (*page); -p2m_mem_paging_populate (curr_d, gmfn); -return X86EMUL_RETRY ; - } +case 0 : +break ; -if (p2m_is_shared (p2mt) ) - { -put_page (*page); +case -EAGAIN :return X86EMUL_RETRY ; + +default : +ASSERT_UNREACHABLE (); +/* Fallthrough */ + +case -EINVAL : +return X86EMUL_UNHANDLEABLE ; }/* This code should not be reached if the gmfn is not RAM */
xen/arch/x86/hvm/hvm.c +1 -13
@@ -2536,20 +2536,8 @@ static void *_hvm_map_guest_frame(unsigned long gfn, bool_t permanent,struct page_info *page;struct domain *d = current->domain; - page =get_page_from_gfn (d, gfn, &p2mt, - writable ?P2M_UNSHARE :P2M_ALLOC ); -if ( (p2m_is_shared (p2mt) && writable) || !page ) - { -if ( page ) -put_page (page); -return NULL ; - } -if (p2m_is_paging (p2mt) ) - { -put_page (page); -p2m_mem_paging_populate (d, gfn); +if (check_get_page_from_gfn (d,_gfn (gfn), !writable, &p2mt, &page) )return NULL ; - }if ( writable ) {
xen/common/grant_table.c +15 -17
@@ -374,25 +374,23 @@ static int get_paged_frame(unsigned long gfn, mfn_t *mfn, struct page_info **page, bool readonly, struct domain *rd) { - int rc = GNTST_okay; p2m_type_t p2mt; +int rc; - *mfn =INVALID_MFN ; - *page =get_page_from_gfn (rd, gfn, &p2mt, - readonly ?P2M_ALLOC :P2M_UNSHARE ); -if ( !*page ) + rc =check_get_page_from_gfn (rd,_gfn (gfn), readonly, &p2mt, page); +switch ( rc ) { -#ifdef P2M_SHARED_TYPES -if (p2m_is_shared (p2mt) ) -return GNTST_eagain; -#endif -#ifdef P2M_PAGES_TYPES -if (p2m_is_paging (p2mt) ) - { -p2m_mem_paging_populate (rd, gfn); -return GNTST_eagain; - } -#endif +case 0 : +break ; + +case -EAGAIN : +return GNTST_eagain; + +default : +ASSERT_UNREACHABLE (); +/* Fallthrough */ + +case -EINVAL :return GNTST_bad_page; }
@@ -406,7 +404,7 @@ static int get_paged_frame(unsigned long gfn, mfn_t *mfn, *mfn =page_to_mfn (*page); -return rc; +return GNTST_okay; } static inline void
xen/common/memory.c +39 -10
76 lines not on this page · raw message
xen/include/asm-arm/p2m.h +2 -2
@@ -110,7 +110,7 @@ struct p2m_domain {* future, it's possible to use higher value for pseudo-type and don't store * them in the p2m entry. */ -typedef enum { +enum p2m_type { p2m_invalid =0 ,/* Nothing mapped here */ p2m_ram_rw,/* Normal read/write guest RAM */ p2m_ram_ro,/* Read-only; writes are silently dropped */
@@ -124,7 +124,7 @@ typedef enum { p2m_iommu_map_rw,/* Read/write iommu mapping */ p2m_iommu_map_ro,/* Read-only iommu mapping */ p2m_max_real_type, /* Types after this won't be store in the p2m */ -} p2m_type_t; +};/* We use bitmaps and mask to handle groups of types */ #define p2m_to_mask (_t) (1UL << (_t))
xen/include/asm-x86/p2m.h +2 -3
@@ -52,7 +52,7 @@ extern bool_t opt_hap_1gb, opt_hap_2mb;* cannot be non-zero, otherwise, hardware generates io page faults when * device access those pages. Therefore, p2m_ram_rw has to be defined as 0. */ -typedef enum { +enum p2m_type { p2m_ram_rw =0 ,/* Normal read/write guest RAM */ p2m_invalid =1 ,/* Nothing mapped here */ p2m_ram_logdirty =2 ,/* Temporarily read-only for log-dirty */
@@ -72,7 +72,7 @@ typedef enum { p2m_ram_broken =13 ,/* Broken page, access cause domain crash */ p2m_map_foreign =14 ,/* ram pages from foreign domain */ p2m_ioreq_server = 15, -} p2m_type_t; +};/* Modifiers to the query */ typedef unsigned int p2m_query_t ;
@@ -492,7 +492,6 @@ static inline struct page_info *get_page_from_gfn(return mfn_valid (_mfn (gfn)) &&get_page (page, d) ? page :NULL ; } -/* General conversion function from mfn to gfn */ static inline unsigned long mfn_to_gfn (struct domain *d,mfn_t mfn) {
xen/include/xen/p2m-common.h +6 -0
@@ -32,5 +32,11 @@ unsigned long p2m_pod_decrease_reservation(struct domain *d, gfn_t gfn, unsigned int order); +typedef enum p2m_type p2m_type_t ; + +int __must_check check_get_page_from_gfn (struct domain *d,gfn_t gfn, +bool readonly,p2m_type_t *p2mt_p, +struct page_info **page_p); +#endif /* _XEN_P2M_COMMON_H */
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v11 7/7 vtd: add lookup_page method to iommu_ops
In reply to Paul DurrantThis patch adds a new method to the VT-d IOMMU implementation to find the MFN currently mapped by the specified DFN along with a wrapper function in generic IOMMU code to call the implementation if it exists.
NOTE: This patch only adds a Xen-internal interface. This will be used by
a subsequent patch.
Another subsequent patch will add similar functionality for AMD IOMMUs.
---
Cc: Wei Liu <[email protected]>
Cc: George Dunlap <[email protected]>
v10:
- Adjust the locking comment.
v9:
- Add comment about locking in xen/iommu.h.
v8:
- Remove clean-up as this is now done by a prior patch.
- Make intel_iommu_lookup_page() return dfn value if using shared EPT or iommu_passthrough is set, as requested by Kevin.
v7:
- Re-base and re-name BFN -> DFN.
- Add missing checks for shared EPT and iommu_passthrough.
- Remove unnecessary initializers and use array-style dereference.
- Drop Wei's R-b because of code churn.
v3:
- Addressed comments from George.
v2:
- Addressed some comments from Jan.
--- xen/drivers/passthrough/iommu.c | 11 ++++++++++ xen/drivers/passthrough/vtd/iommu.c | 44 +++++++++++++++++++++++++++++++++++++ xen/drivers/passthrough/vtd/iommu.h | 3 +++ xen/include/xen/iommu.h | 10 +++++++++ 4 files changed, 68 insertions(+)
xen/drivers/passthrough/iommu.c +11 -0
@@ -337,6 +337,17 @@ int iommu_unmap_page(struct domain *d, dfn_t dfn)return rc; } +int iommu_lookup_page (struct domain *d,dfn_t dfn,mfn_t *mfn, +unsigned int *flags) +{ +const struct domain_iommu *hd =dom_iommu (d); + +if ( !iommu_enabled || !hd->platform_ops ) +return -EOPNOTSUPP ; + +return hd->platform_ops->lookup_page (d, dfn, mfn, flags); +} +static void iommu_free_pagetables (unsigned long unused) { do {
xen/drivers/passthrough/vtd/iommu.c +44 -0
56 lines not on this page · raw message
xen/drivers/passthrough/vtd/iommu.h +3 -0
@@ -272,6 +272,9 @@ struct dma_pte {#define dma_set_pte_prot (p, prot) do { \ (p).val = ((p).val & ~DMA_PTE_PROT) | ((prot) & DMA_PTE_PROT); \ } while (0) +#define dma_pte_prot (p) ((p).val & DMA_PTE_PROT) +#define dma_pte_read (p) (dma_pte_prot(p) & DMA_PTE_READ) +#define dma_pte_write (p) (dma_pte_prot(p) & DMA_PTE_WRITE)#define dma_pte_addr (p) ((p).val & PADDR_MASK & PAGE_MASK_4K)#define dma_set_pte_addr (p, addr) do {\ (p).val |= ((addr) & PAGE_MASK_4K); } while (0)
xen/include/xen/iommu.h +10 -0
@@ -92,6 +92,8 @@ void iommu_teardown(struct domain *d);int __must_check iommu_map_page (struct domain *d,dfn_t dfn,mfn_t mfn,unsigned int flags);int __must_check iommu_unmap_page (struct domain *d,dfn_t dfn); +int __must_check iommu_lookup_page (struct domain *d,dfn_t dfn,mfn_t *mfn, +unsigned int *flags);enum iommu_feature {
@@ -179,9 +181,17 @@ struct iommu_ops {#endif /* HAS_PCI */ void (*teardown)(struct domain *d); + +/* +* This block of operations must be appropriately locked against each +* other by the caller in order to have meaningful results. +*/ int __must_check (*map_page)(struct domain *d,dfn_t dfn,mfn_t mfn,unsigned int flags);int __must_check (*unmap_page)(struct domain *d,dfn_t dfn); +int __must_check (*lookup_page)(struct domain *d,dfn_t dfn,mfn_t *mfn, +unsigned int *flags); +void (*free_page_table)(struct page_info *);#ifdef CONFIG_X86 void (*update_ire_from_apic)(unsigned int apic,unsigned int reg,unsigned int value);
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v11 2/7 iommu: make use of type-safe DFN and MFN in exported functions
In reply to Paul DurrantThis patch modifies the declaration of the entry points to the IOMMU sub-system to use dfn_t and mfn_t in place of unsigned long. A subsequent patch will similarly modify the methods in the iommu_ops structure.
---
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Tim Deegan <[email protected]>
Cc: Jun Nakajima <[email protected]>
Cc: George Dunlap <[email protected]>
v9:
- Re-base.
v7:
- Re-base and re-name BFN -> DFN.
- Added Jan's A-b since re-naming was purely mechanical.
v6:
- Re-base.
v3:
- Removed most of the uses of an intermediate 'frame' variable.
v2:
- Addressed comments from Jan.
- Use intermediate 'frame' variable to avoid directly encapsulating mfn or gfn values as dfns.
--- xen/arch/arm/p2m.c | 3 ++- xen/arch/x86/mm.c | 10 ++++---- xen/arch/x86/mm/p2m-ept.c | 10 +++++--- xen/arch/x86/mm/p2m-pt.c | 45 ++++++++++++++++++++--------------- xen/arch/x86/mm/p2m.c | 16 ++++++++----- xen/arch/x86/x86_64/mm.c | 5 ++-- xen/common/grant_table.c | 12 +++++----- xen/common/memory.c | 4 ++-- xen/drivers/passthrough/iommu.c | 25 ++++++++++--------- xen/drivers/passthrough/vtd/x86/vtd.c | 1 - xen/drivers/passthrough/x86/iommu.c | 3 ++- xen/include/xen/iommu.h | 14 +++++++---- 12 files changed, 85 insertions(+), 63 deletions(-)
xen/arch/arm/p2m.c +2 -1
@@ -957,7 +957,8 @@ static int __p2m_set_entry(struct p2m_domain *p2m,if (need_iommu (p2m->domain) && (lpae_is_valid (orig_pte) ||lpae_is_valid (*entry)) ) - rc =iommu_iotlb_flush (p2m->domain,gfn_x (sgfn),1UL << page_order); + rc =iommu_iotlb_flush (p2m->domain,_dfn (gfn_x (sgfn)), +1UL << page_order);else rc =0 ;
xen/arch/x86/mm.c +5 -5
@@ -2799,14 +2799,14 @@ static int _get_page_type(struct page_info *page, unsigned long type,struct domain *d =page_get_owner (page);if ( d &&is_pv_domain (d) &&unlikely (need_iommu (d)) ) { -gfn_t gfn =_gfn (mfn_to_gmfn (d,mfn_x (page_to_mfn (page)))); +mfn_t mfn =page_to_mfn (page);if ( (x & PGT_type_mask) == PGT_writable_page ) - iommu_ret =iommu_unmap_page (d,gfn_x (gfn)); + iommu_ret =iommu_unmap_page (d,_dfn (mfn_x (mfn)));else if ( type == PGT_writable_page ) - iommu_ret =iommu_map_page (d,gfn_x (gfn), -mfn_x (page_to_mfn (page)), - IOMMUF_readable|IOMMUF_writable); + iommu_ret =iommu_map_page (d,_dfn (mfn_x (mfn)), mfn, + IOMMUF_readable | + IOMMUF_writable); } }
xen/arch/x86/mm/p2m-ept.c +7 -3
@@ -870,15 +870,19 @@ out: rc =iommu_pte_flush (d, gfn, &ept_entry->epte, order, vtd_pte_present); else { +dfn_t dfn =_dfn (gfn); +if ( iommu_flags )for ( i =0 ; i < (1 << order); i++ ) { - rc =iommu_map_page (d, gfn + i,mfn_x (mfn) + i, iommu_flags); + rc =iommu_map_page (d,dfn_add (dfn, i), +mfn_add (mfn, i), iommu_flags);if (unlikely (rc) ) {while ( i-- )/* If statement to satisfy __must_check. */ -if (iommu_unmap_page (p2m->domain, gfn + i) ) +if (iommu_unmap_page (p2m->domain, +dfn_add (dfn, i)) )continue ;break ;
@@ -887,7 +891,7 @@ out:else for ( i =0 ; i < (1 << order); i++ ) { - ret =iommu_unmap_page (d, gfn + i); + ret =iommu_unmap_page (d,dfn_add (dfn, i));if ( !rc ) rc = ret; }
xen/arch/x86/mm/p2m-pt.c +26 -19
55 lines not on this page · raw message
xen/arch/x86/mm/p2m.c +10 -6
@@ -715,9 +715,11 @@ p2m_remove_page(struct p2m_domain *p2m, unsigned long gfn_l, unsigned long mfn,if (need_iommu (p2m->domain) ) { +dfn_t dfn =_dfn (mfn); +for ( i =0 ; i < (1 << page_order); i++ ) { -int ret =iommu_unmap_page (p2m->domain, mfn + i); +int ret =iommu_unmap_page (p2m->domain,dfn_add (dfn, i));if ( !rc ) rc = ret;
@@ -774,16 +776,17 @@ guest_physmap_add_entry(struct domain *d, gfn_t gfn, mfn_t mfn, {if (need_iommu (d) && t == p2m_ram_rw ) { +dfn_t dfn =_dfn (mfn_x (mfn)); +for ( i =0 ; i < (1 << page_order); i++ ) { - rc =iommu_map_page (d,mfn_x (mfn_add (mfn, i)), -mfn_x (mfn_add (mfn, i)), + rc =iommu_map_page (d,dfn_add (dfn, i),mfn_add (mfn, i), IOMMUF_readable|IOMMUF_writable);if ( rc !=0 ) {while ( i-- >0 )/* If statement to satisfy __must_check. */ -if (iommu_unmap_page (d,mfn_x (mfn_add (mfn, i))) ) +if (iommu_unmap_page (d,dfn_add (dfn, i)) )continue ;return rc;
@@ -1158,7 +1161,8 @@ int set_identity_p2m_entry(struct domain *d, unsigned long gfn_l, {if ( !need_iommu (d) )return 0 ; -return iommu_map_page (d, gfn_l, gfn_l, IOMMUF_readable|IOMMUF_writable); +return iommu_map_page (d,_dfn (gfn_l),_mfn (gfn_l), + IOMMUF_readable | IOMMUF_writable); }gfn_lock (p2m, gfn,0 );
@@ -1248,7 +1252,7 @@ int clear_identity_p2m_entry(struct domain *d, unsigned long gfn_l) {if ( !need_iommu (d) )return 0 ; -return iommu_unmap_page (d, gfn_l); +return iommu_unmap_page (d,_dfn (gfn_l)); }gfn_lock (p2m, gfn,0 );
xen/arch/x86/x86_64/mm.c +3 -2
@@ -1430,13 +1430,14 @@ int memory_add(unsigned long spfn, unsigned long epfn, unsigned int pxm) !need_iommu(hardware_domain) ) {for ( i = spfn; i < epfn; i++ ) -if (iommu_map_page (hardware_domain, i, i, IOMMUF_readable|IOMMUF_writable) ) +if (iommu_map_page (hardware_domain,_dfn (i),_mfn (i), + IOMMUF_readable | IOMMUF_writable) )break ;if ( i != epfn ) {while (i-- > old_max)/* If statement to satisfy __must_check. */ -if (iommu_unmap_page (hardware_domain, i) ) +if (iommu_unmap_page (hardware_domain,_dfn (i)) )continue ; goto destroy_m2p;
xen/common/grant_table.c +6 -6
@@ -1143,13 +1143,13 @@ map_grant_ref( !(old_pin & (GNTPIN_hstw_mask|GNTPIN_devw_mask)) ) {if ( !(kind &MAPKIND_WRITE ) ) - err =iommu_map_page (ld,mfn_x (mfn),mfn_x (mfn), - IOMMUF_readable|IOMMUF_writable); + err =iommu_map_page (ld,_dfn (mfn_x (mfn)), mfn, + IOMMUF_readable | IOMMUF_writable); } else if ( act_pin && !old_pin ) {if ( !kind ) - err =iommu_map_page (ld,mfn_x (mfn),mfn_x (mfn), + err =iommu_map_page (ld,_dfn (mfn_x (mfn)), mfn, IOMMUF_readable); }if ( err )
@@ -1398,10 +1398,10 @@ unmap_common( kind =mapkind (lgt, rd, op->mfn);if ( !kind ) - err =iommu_unmap_page (ld,mfn_x (op->mfn)); + err =iommu_unmap_page (ld,_dfn (mfn_x (op->mfn)));else if ( !(kind &MAPKIND_WRITE ) ) - err =iommu_map_page (ld,mfn_x (op->mfn), -mfn_x (op->mfn), IOMMUF_readable); + err =iommu_map_page (ld,_dfn (mfn_x (op->mfn)), op->mfn, + IOMMUF_readable);double_gt_unlock (lgt, rgt);
xen/common/memory.c +2 -2
@@ -834,11 +834,11 @@ int xenmem_add_to_physmap(struct domain *d, struct xen_add_to_physmap *xatp,this_cpu (iommu_dont_flush_iotlb) =0 ; - ret =iommu_iotlb_flush (d, xatp->idx - done, done); + ret =iommu_iotlb_flush (d,_dfn (xatp->idx - done), done);if (unlikely (ret) && rc >=0 ) rc = ret; - ret =iommu_iotlb_flush (d, xatp->gpfn - done, done); + ret =iommu_iotlb_flush (d,_dfn (xatp->gpfn - done), done);if (unlikely (ret) && rc >=0 ) rc = ret; }
xen/drivers/passthrough/iommu.c +12 -13
@@ -295,7 +295,7 @@ void iommu_domain_destroy(struct domain *d)arch_iommu_domain_destroy (d); } -int iommu_map_page (struct domain *d,unsigned long dfn,unsigned long mfn, +int iommu_map_page (struct domain *d,dfn_t dfn,mfn_t mfn,unsigned int flags) {const struct domain_iommu *hd =dom_iommu (d);
@@ -304,13 +304,13 @@ int iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,if ( !iommu_enabled || !hd->platform_ops )return 0 ; - rc = hd->platform_ops->map_page (d, dfn, mfn, flags); + rc = hd->platform_ops->map_page (d,dfn_x (dfn),mfn_x (mfn), flags);if (unlikely (rc) ) {if ( !d->is_shutting_down &&printk_ratelimit () )printk (XENLOG_ERR -"d%d: IOMMU mapping dfn %#lx to mfn %#lx failed: %d\n" , - d->domain_id, dfn, mfn, rc); +"d%d: IOMMU mapping dfn %" PRI_dfn" to mfn %" PRI_mfn" failed: %d\n" , + d->domain_id,dfn_x (dfn),mfn_x (mfn), rc);if ( !is_hardware_domain (d) )domain_crash (d);
@@ -319,7 +319,7 @@ int iommu_map_page(struct domain *d, unsigned long dfn, unsigned long mfn,return rc; } -int iommu_unmap_page (struct domain *d,unsigned long dfn) +int iommu_unmap_page (struct domain *d,dfn_t dfn) {const struct domain_iommu *hd =dom_iommu (d);int rc;
@@ -327,13 +327,13 @@ int iommu_unmap_page(struct domain *d, unsigned long dfn)if ( !iommu_enabled || !hd->platform_ops )return 0 ; - rc = hd->platform_ops->unmap_page (d, dfn); + rc = hd->platform_ops->unmap_page (d,dfn_x (dfn));if (unlikely (rc) ) {if ( !d->is_shutting_down &&printk_ratelimit () )printk (XENLOG_ERR -"d%d: IOMMU unmapping dfn %#lx failed: %d\n" , - d->domain_id, dfn, rc); +"d%d: IOMMU unmapping dfn %" PRI_dfn" failed: %d\n" , + d->domain_id,dfn_x (dfn), rc);if ( !is_hardware_domain (d) )domain_crash (d);
@@ -359,8 +359,7 @@ static void iommu_free_pagetables(unsigned long unused) cpumask_cycle(smp_processor_id(), &cpu_online_map)); } -int iommu_iotlb_flush (struct domain *d,unsigned long dfn, -unsigned int page_count) +int iommu_iotlb_flush (struct domain *d,dfn_t dfn,unsigned int page_count) {const struct domain_iommu *hd =dom_iommu (d);int rc;
@@ -368,13 +367,13 @@ int iommu_iotlb_flush(struct domain *d, unsigned long dfn,if ( !iommu_enabled || !hd->platform_ops || !hd->platform_ops->iotlb_flush )return 0 ; - rc = hd->platform_ops->iotlb_flush (d, dfn, page_count); + rc = hd->platform_ops->iotlb_flush (d,dfn_x (dfn), page_count);if (unlikely (rc) ) {if ( !d->is_shutting_down &&printk_ratelimit () )printk (XENLOG_ERR -"d%d: IOMMU IOTLB flush failed: %d, dfn %#lx, page count %u\n" , - d->domain_id, rc, dfn, page_count); +"d%d: IOMMU IOTLB flush failed: %d, dfn %" PRI_dfn", page count %u\n" , + d->domain_id, rc,dfn_x (dfn), page_count);if ( !is_hardware_domain (d) )domain_crash (d);
xen/drivers/passthrough/vtd/x86/vtd.c +0 -1
@@ -60,4 +60,3 @@ void flush_all_cache() {wbinvd (); } -
xen/drivers/passthrough/x86/iommu.c +2 -1
@@ -227,7 +227,8 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)if (paging_mode_translate (d) ) rc =set_identity_p2m_entry (d, pfn, p2m_access_rw,0 );else - rc =iommu_map_page (d, pfn, pfn, IOMMUF_readable|IOMMUF_writable); + rc =iommu_map_page (d,_dfn (pfn),_mfn (pfn), + IOMMUF_readable | IOMMUF_writable);if ( rc )printk (XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n" , d->domain_id, rc);
xen/include/xen/iommu.h +10 -4
@@ -24,6 +24,7 @@#include <xen/spinlock.h> #include <xen/pci.h> #include <xen/typesafe.h> +#include <xen/mm.h> #include <public/hvm/ioreq.h> #include <public/domctl.h> #include <asm/device.h>
@@ -42,6 +43,11 @@ TYPE_SAFE(uint64_t, dfn);#undef dfn_x#endif +static inline dfn_t dfn_add (dfn_t dfn,unsigned long i) +{ +return _dfn (dfn_x (dfn) + i); +} +extern bool_t iommu_enable, iommu_enabled;extern bool_t force_iommu, iommu_verbose;extern bool_t iommu_workaround_bios_bug, iommu_igfx;
@@ -78,9 +84,9 @@ void iommu_teardown(struct domain *d);#define IOMMUF_readable (1u<<_IOMMUF_readable)#define _IOMMUF_writable 1#define IOMMUF_writable (1u<<_IOMMUF_writable) -int __must_check iommu_map_page (struct domain *d,unsigned long dfn, -unsigned long mfn,unsigned int flags); -int __must_check iommu_unmap_page (struct domain *d,unsigned long dfn); +int __must_check iommu_map_page (struct domain *d,dfn_t dfn, +mfn_t mfn,unsigned int flags); +int __must_check iommu_unmap_page (struct domain *d,dfn_t dfn);enum iommu_feature {
@@ -203,7 +209,7 @@ int iommu_do_pci_domctl(struct xen_domctl *, struct domain *d,int iommu_do_domctl (struct xen_domctl *,struct domain *d,XEN_GUEST_HANDLE_PARAM (xen_domctl_t )); -int __must_check iommu_iotlb_flush (struct domain *d,unsigned long dfn, +int __must_check iommu_iotlb_flush (struct domain *d,dfn_t dfn,unsigned int page_count);int __must_check iommu_iotlb_flush_all (struct domain *d);
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v11 1/7] iommu: introduce the concept of DFN...
In reply to Paul DurrantQuoted 14 lines · Paul Durrant
...meaning 'device DMA frame number' i.e. a frame number mapped in the IOMMU (rather than the MMU) and hence used for DMA address translation.
This patch is a largely cosmetic change that substitutes the terms 'gfn' and 'gaddr' for 'dfn' and 'daddr' in all the places where the frame number or address relate to a device rather than the CPU.
The parts that are not purely cosmetic are:
- the introduction of a type-safe declaration of
dfn_tand definition ofINVALID_DFNto make the substitution ofgfn_x(INVALID_GFN)mechanical.
- the introduction of__dfn_to_daddrand__daddr_to_dfn(and type-safe variants without the leading __) with some use of the former.
Quoted 5 lines
Subsequent patches will convert code to make use of type-safe DFNs.
Signed-off-by Paul Durrant <[email protected]>Acked-by Jan Beulich <[email protected]>Reviewed-by Kevin Tian <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v11 4/7] iommu: don't domain_crash() inside iommu_map/unmap_page()
In reply to Paul DurrantQuoted 10 lines · Paul Durrant
This patch removes the implicit
domain_crash()fromiommu_map(),unmap_page()andiommu_iotlb_flush()and turns them into straightforward wrappers that check the existence of the relevantiommu_opand call through to it. This makes them usable by PV IOMMU code to be delivered in future patches.
This patch adds a helper macro,domu_crash(), that will only invokedomain_crash()if the domain is not the hardware domain and modifies callers ofiommu_map(),unmap_page()andiommu_iotlb_flush()to use this should an operation fail.
Quoted 8 lines
NOTE: This patch includes one bit of clean-up in
set_identity_p2m_entry()replacing use of p2m->domain with the domain pointer passed into the function.Signed-off-by Paul Durrant <[email protected]>Reviewed-by Jan Beulich <[email protected]>Reviewed-by Kevin Tian <[email protected]>Reviewed-by Roger Pau Monne <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v11 5/7] memory: add check_get_page_from_gfn() as a wrapper...
In reply to Paul DurrantQuoted 30 lines · Paul Durrant
...for some uses of
get_page_from_gfn().There are many occurrences of the following pattern in the code:
q = <readonly look-up> ? P2M_ALLOC : P2M_UNSHARE; page = get_page_from_gfn(d, gfn, &p2mt, q); if ( p2m_is_paging(p2mt) ) { if ( page ) put_page(page); p2m_mem_paging_populate(d, gfn); return <-EAGAIN or equivalent>; } if ( (q & P2M_UNSHARE) && p2m_is_shared(p2mt) ) { if ( page ) put_page(page); return <-EAGAIN or equivalent>; } if ( !page ) return <-EINVAL or equivalent>;There are some small differences between the exact way the occurrences are coded but the desired semantic is the same.
Quoted 7 lines
This patch introduces a new common implementation of this code in
check_get_page_from_gfn()and then converts the various open-coded patterns into calls to this new function.Signed-off-by Paul Durrant <[email protected]>Reviewed-by Roger Pau Monne <[email protected]>Reviewed-by Jan Beulich <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v11 6/7] vtd: add missing check for shared EPT...
In reply to Paul DurrantQuoted 7 lines · Paul Durrant
...in
intel_iommu_unmap_page().This patch also includes some non-functional modifications in
intel_iommu_map_page().Signed-off-by Paul Durrant <[email protected]>Acked-by Kevin Tian <[email protected]>
This patch contains more cleanup changes then its meat, Heh.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v11 7/7] vtd: add lookup_page method to iommu_ops
In reply to Paul DurrantQuoted 4 lines · Paul Durrant
This patch adds a new method to the VT-d IOMMU implementation to find the MFN currently mapped by the specified DFN along with a wrapper function in generic IOMMU code to call the implementation if it exists.
Quoted 8 lines
NOTE: This patch only adds a Xen-internal interface. This will be used by
a subsequent patch.
Another subsequent patch will add similar functionality for AMD IOMMUs.Signed-off-by Paul Durrant <[email protected]>Reviewed-by Jan Beulich <[email protected]>Reviewed-by Kevin Tian <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel