intel-iommu: Don't be too aggressive when clearing one context entry
PATCH 1/1 intel-iommu: Don't be too aggressive when clearing one context entry
Previously, we were invalidating context cache and IOTLB globally when clearing one context entry. This is a tad too aggressive. Invalidate the context cache and IOTLB for the interested device only.
--- drivers/iommu/intel-iommu.c | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)
drivers/iommu/intel-iommu.c +22 -3
@@ -2351,13 +2351,32 @@ static inline int domain_pfn_mapping(struct dmar_domain *domain, unsigned long istatic void domain_context_clear_one (struct intel_iommu *iommu,u8 bus,u8 devfn) { +unsigned long flags; +struct context_entry *context; +u16 did_old; +if (!iommu)return ; +spin_lock_irqsave (&iommu->lock, flags); + context =iommu_context_addr (iommu, bus, devfn,0 ); +if (!context) { +spin_unlock_irqrestore (&iommu->lock, flags); +return ; + } + did_old =context_domain_id (context); +spin_unlock_irqrestore (&iommu->lock, flags);clear_context_table (iommu, bus, devfn); - iommu->flush.flush_context (iommu,0 ,0 ,0 , -DMA_CCMD_GLOBAL_INVL ); - iommu->flush.flush_iotlb (iommu,0 ,0 ,0 ,DMA_TLB_GLOBAL_FLUSH ); + iommu->flush.flush_context (iommu, + did_old, + (((u16 )bus) <<8 ) | devfn, +DMA_CCMD_MASK_NOBIT , +DMA_CCMD_DEVICE_INVL ); + iommu->flush.flush_iotlb (iommu, + did_old, +0 , +0 , +DMA_TLB_DSI_FLUSH ); }static inline void unlink_domain_info (struct device_domain_info *info)
Signature · 1 line
2.7.4
Joerg Roedel
maintainerHi Filippo,
please change the subject to:
iommu/vt-d: Don't be too aggressive when clearing one context entry
to follow the convention used in the iommu-tree. Another comment below.
Quoted 8 lines · Filippo Sironi
static void
domain_context_clear_one(struct intel_iommu *iommu, u8 bus, u8 devfn){drivers/iommu/intel-iommu.c +4
-0+unsigned long flags; +struct context_entry *context; +u16 did_old; +if (!iommu)return ;
drivers/iommu/intel-iommu.c static inline int domain_pfn_mapping(struct dmar_domain *domain, unsigned long i · line 2369
Quoted 10 lines
drivers/iommu/intel-iommu.c +8
-0+spin_lock_irqsave (&iommu->lock, flags); + context =iommu_context_addr (iommu, bus, devfn,0 ); +if (!context) { +spin_unlock_irqrestore (&iommu->lock, flags); +return ; + } + did_old =context_domain_id (context); +spin_unlock_irqrestore (&iommu->lock, flags);clear_context_table (iommu, bus, devfn);
This function is the only caller of clear_context_table(), which does similar things (like fetching the context-entry) as you are adding above.
So you can either make clear_context_table() return the old domain-id so that you don't need to do it here, or you get rid of the function entirely and add the context_clear_entry() and __iommu_flush_cache() calls into this code-path.
Regards,
Joerg
Hi Joerg,
Quoted 9 lines · Joerg Roedel
Hi Filippo,
please change the subject to:
iommu/vt-d: Don't be too aggressive when clearing one context entry
to follow the convention used in the iommu-tree. Another comment below.
Will do.
Quoted 24 lines · Filippo Sironi
Quoted 17 lines
static void
domain_context_clear_one(struct intel_iommu *iommu, u8 bus, u8 devfn){drivers/iommu/intel-iommu.c +4
-0+unsigned long flags; +struct context_entry *context; +u16 did_old; +if (!iommu) return;drivers/iommu/intel-iommu.c +8
-0+spin_lock_irqsave (&iommu->lock, flags); + context =iommu_context_addr (iommu, bus, devfn,0 ); +if (!context) { +spin_unlock_irqrestore (&iommu->lock, flags); +return ; + } + did_old =context_domain_id (context); +spin_unlock_irqrestore (&iommu->lock, flags);Quoted 1 line
clear_context_table(iommu, bus, devfn);This function is the only caller of
clear_context_table(), which does similar things (like fetching the context-entry) as you are adding above.
Quoted 8 lines
So you can either make
clear_context_table()return the old domain-id so that you don't need to do it here, or you get rid of the function entirely and add thecontext_clear_entry()and__iommu_flush_cache()calls into this code-path.Regards,
Joerg
I went for merging domain_context_clear_one() with context_clear_one().
Regards,
Filippo
Amazon Development Center Germany GmbH
Berlin - Dresden - Aachen
main office: Krausenstr. 38, 10117 Berlin
Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
Ust-ID: DE289237879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
Quoted 25 lines · On Mon, 28 Aug 2017 16:16:29 +0200 Filippo Sironi via iommu…
Previously, we were invalidating context cache and IOTLB globally when clearing one context entry. This is a tad too aggressive. Invalidate the context cache and IOTLB for the interested device only.
Signed-off-by Filippo Sironi <[email protected]>Cc David Woodhouse <[email protected]>Cc David Woodhouse <[email protected]>Cc Joerg Roedel <[email protected]>--- drivers/iommu/intel-iommu.c | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)drivers/iommu/intel-iommu.c +0
-0@@ -2351,13 +2351,32 @@ static inline int domain_pfn_mapping(struct
dmar_domain*domain, unsigned long i
static void domain_context_clear_one(structintel_iommu*iommu, u8
bus, u8 devfn) {drivers/iommu/intel-iommu.c +3
-0+unsigned long flags; +struct context_entry *context; +u16 did_old;
drivers/iommu/intel-iommu.c static inline int domain_pfn_mapping(struct dmar_domain *domain, unsigned long i · line 2366
Quoted 10 lines
drivers/iommu/intel-iommu.c +7
-0+if (!iommu)return ; +spin_lock_irqsave (&iommu->lock, flags); + context =iommu_context_addr (iommu, bus, devfn,0 ); +if (!context) { +spin_unlock_irqrestore (&iommu->lock, flags); +return ; + }
perhaps check with device_context_mapped()?
Quoted 20 lines
drivers/iommu/intel-iommu.c +2
-3+ did_old =context_domain_id (context); +spin_unlock_irqrestore (&iommu->lock, flags);clear_context_table (iommu, bus, devfn); - iommu->flush.flush_context (iommu,0 ,0 ,0 , -DMA_CCMD_GLOBAL_INVL ); - iommu->flush.flush_iotlb(iommu, 0, 0, 0,
DMA_TLB_GLOBAL_FLUSH);drivers/iommu/intel-iommu.c +10
-0+ iommu->flush.flush_context (iommu, + did_old, + (((u16 )bus) <<8 ) | devfn, +DMA_CCMD_MASK_NOBIT , +DMA_CCMD_DEVICE_INVL ); + iommu->flush.flush_iotlb (iommu, + did_old, +0 , +0 , +DMA_TLB_DSI_FLUSH ); } static inline void unlink_domain_info(struct device_domain_info
Quoted 1 line
*info)
[Jacob Pan]
Hi Jacob,
Quoted 34 lines · Jacob Pan
Quoted 29 lines · On Mon, 28 Aug 2017 16:16:29 +0200 Filippo Sironi via iommu…
Previously, we were invalidating context cache and IOTLB globally when clearing one context entry. This is a tad too aggressive. Invalidate the context cache and IOTLB for the interested device only.
Signed-off-by Filippo Sironi <[email protected]>Cc David Woodhouse <[email protected]>Cc David Woodhouse <[email protected]>Cc Joerg Roedel <[email protected]>---
drivers/iommu/intel-iommu.c| 25 ++++++++++++++++++++++---1 file changed, 22 insertions(+), 3 deletions(-)drivers/iommu/intel-iommu.c +0
-0@@ -2351,13 +2351,32 @@ static inline int domain_pfn_mapping(struct
dmar_domain*domain, unsigned long i
static voiddomain_context_clear_one(struct intel_iommu *iommu, u8 bus, u8 devfn){drivers/iommu/intel-iommu.c +4
-0+unsigned long flags; +struct context_entry *context; +u16 did_old; +if (!iommu) return;
Quoted 7 lines
drivers/iommu/intel-iommu.c static inline int domain_pfn_mapping(struct dmar_domain *domain, unsigned long i · line 2366
Quoted 6 lines
drivers/iommu/intel-iommu.c +6
-0+spin_lock_irqsave (&iommu->lock, flags); + context =iommu_context_addr (iommu, bus, devfn,0 ); +if (!context) { +spin_unlock_irqrestore (&iommu->lock, flags); +return ; + }perhaps check with
device_context_mapped()?
Using device_context_mapped() wouldn't simplify the code since it would just tell me that at the time of check there was a context. I would still need to lock, get the context, check if the context is valid, do the work, and unlock.
Modifying device_context_mapped() to return the context isn't going to work either because it may go away in the meantime since I wouldn't hold the lock.
Quoted 19 lines
Quoted 19 lines
drivers/iommu/intel-iommu.c +2
-0+ did_old =context_domain_id (context); +spin_unlock_irqrestore (&iommu->lock, flags);clear_context_table(iommu, bus, devfn);drivers/iommu/intel-iommu.c +0
-3- iommu->flush.flush_context (iommu,0 ,0 ,0 , -DMA_CCMD_GLOBAL_INVL ); - iommu->flush.flush_iotlb(iommu, 0, 0, 0,
DMA_TLB_GLOBAL_FLUSH);drivers/iommu/intel-iommu.c +10
-0+ iommu->flush.flush_context (iommu, + did_old, + (((u16 )bus) <<8 ) | devfn, +DMA_CCMD_MASK_NOBIT , +DMA_CCMD_DEVICE_INVL ); + iommu->flush.flush_iotlb (iommu, + did_old, +0 , +0 , +DMA_TLB_DSI_FLUSH );}
Quoted 4 lines
Quoted 2 lines
static inline void
unlink_domain_info(struct device_domain_info *info)[Jacob Pan]
Regards,
Filippo
Amazon Development Center Germany GmbH
Berlin - Dresden - Aachen
main office: Krausenstr. 38, 10117 Berlin
Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
Ust-ID: DE289237879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
PATCH v2 1/1 iommu/vt-d: Don't be too aggressive when clearing one context entry
In reply to Filippo SironiPreviously, we were invalidating context cache and IOTLB globally when clearing one context entry. This is a tad too aggressive. Invalidate the context cache and IOTLB for the interested device only.
--- drivers/iommu/intel-iommu.c | 42 ++++++++++++++++++++++++------------------ 1 file changed, 24 insertions(+), 18 deletions(-)
drivers/iommu/intel-iommu.c +24 -18
@@ -974,20 +974,6 @@ static int device_context_mapped(struct intel_iommu *iommu, u8 bus, u8 devfn)return ret; } -static void clear_context_table (struct intel_iommu *iommu,u8 bus,u8 devfn) -{ -struct context_entry *context; -unsigned long flags; - -spin_lock_irqsave (&iommu->lock, flags); - context =iommu_context_addr (iommu, bus, devfn,0 ); -if (context) { -context_clear_entry (context); -__iommu_flush_cache (iommu, context,sizeof (*context)); - } -spin_unlock_irqrestore (&iommu->lock, flags); -} -static void free_context_table (struct intel_iommu *iommu) {int i;
@@ -2351,13 +2337,33 @@ static inline int domain_pfn_mapping(struct dmar_domain *domain, unsigned long istatic void domain_context_clear_one (struct intel_iommu *iommu,u8 bus,u8 devfn) { +unsigned long flags; +struct context_entry *context; +u16 did_old; +if (!iommu)return ; -clear_context_table (iommu, bus, devfn); - iommu->flush.flush_context (iommu,0 ,0 ,0 , -DMA_CCMD_GLOBAL_INVL ); - iommu->flush.flush_iotlb (iommu,0 ,0 ,0 ,DMA_TLB_GLOBAL_FLUSH ); +spin_lock_irqsave (&iommu->lock, flags); + context =iommu_context_addr (iommu, bus, devfn,0 ); +if (!context) { +spin_unlock_irqrestore (&iommu->lock, flags); +return ; + } + did_old =context_domain_id (context); +context_clear_entry (context); +__iommu_flush_cache (iommu, context,sizeof (*context)); +spin_unlock_irqrestore (&iommu->lock, flags); + iommu->flush.flush_context (iommu, + did_old, + (((u16 )bus) <<8 ) | devfn, +DMA_CCMD_MASK_NOBIT , +DMA_CCMD_DEVICE_INVL ); + iommu->flush.flush_iotlb (iommu, + did_old, +0 , +0 , +DMA_TLB_DSI_FLUSH ); }static inline void unlink_domain_info (struct device_domain_info *info)
Signature · 1 line
2.7.4
A message that is not in the archive (sent off-list or not yet received): 1504169891-9480-1-git-send-email-sironi-ebkRAfMGSJGzQB+pC5nmwQ@public.gmane.org
Woodhouse, David via iommu
Acked-by David WoodhouseRe: [PATCH v2] iommu/vt-d: Don't be too aggressive when clearing one context entry
Quoted 11 lines · Filippo Sironi
Previously, we were invalidating context cache and IOTLB globally when clearing one context entry. This is a tad too aggressive. Invalidate the context cache and IOTLB for the interested device only.
Signed-off-by Filippo Sironi <[email protected]>Cc David Woodhouse <[email protected]>Cc David Woodhouse <[email protected]>Cc Joerg Roedel <[email protected]>Cc Jacob Pan <jacob.jun.pan-VuQAYsv1563Yd54FQh9/[email protected]>Cc iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
I'd actually quite like to see us tidy this up some more. If there are aliases, we're doing the flush multiple times. It might be nicer to do no flushing in domain_contact_clear_one(), which is invoked for each alias, and just to update a (u16 *)did. If *did was empty, set it. If it was set (i.e. if there are aliases), then set it to a magic value like 0xFFFF.
Then we can do the actual flush — either device-specific, or global — just once in domain_context_clear().
We'd probably just move domain_context_clear_one() into its only caller domain_context_clear_one_cb() while we're at it.
But in the meantime I'm more than happy with this patch which turns multiple global flushes into multiple domain-specific flushes, and does the right thing in the common case.
Quoted 1 line
---
v2: Changed subject, killed clear_context_table(). Anything else?
Quoted 46 lines
drivers/iommu/intel-iommu.c | 42 ++++++++++++++++++++++++------------------
1 file changed, 24 insertions(+), 18 deletions(-)drivers/iommu/intel-iommu.c +0
-0@@ -974,20 +974,6 @@ static int device_context_mapped(struct intel_iommu *iommu, u8 bus, u8 devfn) return ret; }drivers/iommu/intel-iommu.c +0
-14-static void clear_context_table (struct intel_iommu *iommu,u8 bus,u8 devfn) -{ -struct context_entry *context; -unsigned long flags; - -spin_lock_irqsave (&iommu->lock, flags); - context =iommu_context_addr (iommu, bus, devfn,0 ); -if (context) { -context_clear_entry (context); -__iommu_flush_cache (iommu, context,sizeof (*context)); - } -spin_unlock_irqrestore (&iommu->lock, flags); -} -static void
free_context_table(struct intel_iommu *iommu){
int i;drivers/iommu/intel-iommu.c +0
-0@@ -2351,13 +2337,33 @@ static inline int domain_pfn_mapping(struct dmar_domain *domain, unsigned long i
static voiddomain_context_clear_one(struct intel_iommu *iommu, u8 bus, u8 devfn){drivers/iommu/intel-iommu.c +4
-0+unsigned long flags; +struct context_entry *context; +u16 did_old; +if (!iommu)
return;drivers/iommu/intel-iommu.c +3
-4-clear_context_table (iommu, bus, devfn); - iommu->flush.flush_context (iommu,0 ,0 ,0 , - DMA_CCMD_GLOBAL_INVL); - iommu->flush.flush_iotlb (iommu,0 ,0 ,0 ,DMA_TLB_GLOBAL_FLUSH ); +spin_lock_irqsave (&iommu->lock, flags); + context =iommu_context_addr (iommu, bus, devfn,0 ); +if (!context) {
drivers/iommu/intel-iommu.c static inline int domain_pfn_mapping(struct dmar_domain *domain, unsigned long i · line 2359
Quoted 10 lines
drivers/iommu/intel-iommu.c +10
-0+spin_unlock_irqrestore (&iommu->lock, flags); +return ; + } + did_old =context_domain_id (context); +context_clear_entry (context); +__iommu_flush_cache (iommu, context,sizeof (*context)); +spin_unlock_irqrestore (&iommu->lock, flags); + iommu->flush.flush_context (iommu, + did_old, + (((u16)bus) << 8) | devfn,
Arguably we ought to be PCI_DEVID() there, but I feel bad pointing that out when you're adding about the fifth instance of open-coding it... :)
--
dwmw2
Amazon Web Services UK Limited. Registered in England and Wales with registration number 08650665 and which has its registered office at 60 Holborn Viaduct, London EC1A 2FD, United Kingdom.
Re: [PATCH v2] iommu/vt-d: Don't be too aggressive when clearing one context entry
In reply to Filippo SironiQuoted 11 lines · Filippo Sironi
Previously, we were invalidating context cache and IOTLB globally when clearing one context entry. This is a tad too aggressive. Invalidate the context cache and IOTLB for the interested device only.
Signed-off-by Filippo Sironi <[email protected]>Cc David Woodhouse <[email protected]>Cc David Woodhouse <[email protected]>Cc Joerg Roedel <[email protected]>Cc Jacob Pan <[email protected]>
Quoted 3 lines
--- drivers/iommu/intel-iommu.c | 42 ++++++++++++++++++++++++------------------ 1 file changed, 24 insertions(+), 18 deletions(-)
Applied, thanks.