SMMUv3.2 Range-based TLB Invalidation Support
cover SMMUv3.2 Range-based TLB Invalidation Support
SMMU3.2 brings the support of range-based TLB invalidation and level hint. When this feature is supported, the SMMUv3 driver is allowed to send TLB invalidations for a range of IOVAs instead of using page based invalidation.
Implementing this feature in the virtual SMMUv3 device is mandated for DPDK on guest use case: DPDK uses hugepage buffers and guest sends invalidations for blocks. Without this feature, a guest invalidation of a block of 1GB for instance translates into a storm of page invalidations. Each of them is trapped by the VMM and cascaded downto the physical IOMMU. This completely stalls the execution. This integration issue was initially reported in [1].
Now SMMUv3.2 specifies additional parameters to NH_VA and NH_VAA stage 1 invalidation commands so we can support those extensions.
patches [1, 3] are cleanup patches.
patches [4, 6] changes the implementation of the VSMMUV3 IOTLB
This IOTLB is a minimalist IOTLB implementation that avoids to do the page table walk in case we have an entry in the TLB. Previously entries were page mappings only. Now they can be blocks.
patches [7, 9] bring support for range invalidation.
Supporting block mappings in the IOTLB look sensible in terms of TLB entry consumption. However looking at virtio/vhost device usage, without block mapping and without range invalidation (< 5.7 kernels it may be less performant. However for recent guest kernels supporting range invalidations [2], the performance should be similar.
Best Regards
Eric
This series can be found at:
https://github.com/eauger/qemu.git
branch: v5.0.0-smmuv3-ril-v1
References:
[1] [RFC v2 4/4] iommu/arm-smmu-v3: add CMD_TLBI_NH_VA_AM command for iova range invalidation
(https://lists.linuxfoundation.org/pipermail/iommu/2017-August/023679.html
[2] 5.7+ kernels featuring6a481a95d4c1 iommu/arm-smmu-v3: Add SMMUv3.2 range invalidation support
Eric Auger (9):
hw/arm/smmu-common: Factorize some code in smmu_ptw_64() hw/arm/smmu-common: Add IOTLB helpers
hw/arm/smmu: Simplify the IOTLB key format hw/arm/smmu: Introduce SMMUTLBEntry for PTW and IOTLB value hw/arm/smmuv3: Store the starting level in SMMUTransTableInfo hw/arm/smmu-common: Manage IOTLB block entries hw/arm/smmuv3: Introduce smmuv3_s1_range_inval() helper hw/arm/smmuv3: Get prepared for range invalidation hw/arm/smmuv3: Advertise SMMUv3.2 range invalidation
hw/arm/smmu-internal.h | 14 +++ hw/arm/smmuv3-internal.h | 5 + include/hw/arm/smmu-common.h | 24 ++-- hw/arm/smmu-common.c | 205 +++++++++++++++++++++++------------ hw/arm/smmuv3.c | 142 ++++++++++++------------ hw/arm/trace-events | 12 +- 6 files changed, 246 insertions(+), 156 deletions(-)
Signature · 1 line
2.20.1
PATCH RESEND 1/9 hw/arm/smmu-common: Factorize some code in smmu_ptw_64()
Page and block PTE decoding can share some code. Let's first handle table PTE and factorize some code shared by page and block PTEs.
--- hw/arm/smmu-common.c | 51 ++++++++++++++++---------------------------- 1 file changed, 18 insertions(+), 33 deletions(-)
hw/arm/smmu-common.c +18 -33
@@ -186,12 +186,12 @@ static int smmu_ptw_64(SMMUTransCfg *cfg,uint64_t subpage_size =1ULL <<level_shift (level, granule_sz);uint64_t mask = subpage_size -1 ;uint32_t offset =iova_level_offset (iova, inputsize, level, granule_sz); -uint64_t pte; +uint64_t pte, gpa;dma_addr_t pte_addr = baseaddr + offset *sizeof (pte);uint8_t ap;if (get_pte (baseaddr, offset, &pte, info)) { - goto error; +break ; }trace_smmu_ptw_level (level, iova, subpage_size, baseaddr, offset, pte);
@@ -199,58 +199,43 @@ static int smmu_ptw_64(SMMUTransCfg *cfg,if (is_invalid_pte (pte) ||is_reserved_pte (pte, level)) {trace_smmu_ptw_invalid_pte (stage, level, baseaddr, pte_addr, offset, pte); - info->type =SMMU_PTW_ERR_TRANSLATION ; - goto error; +break ; } -if (is_page_pte (pte, level)) { -uint64_t gpa =get_page_pte_address (pte, granule_sz); +if (is_table_pte (pte, level)) { + ap =PTE_APTABLE (pte); - ap =PTE_AP (pte);if (is_permission_fault (ap, perm)) { info->type =SMMU_PTW_ERR_PERMISSION ; goto error; } - - tlbe->translated_addr = gpa + (iova & mask); - tlbe->perm =PTE_AP_TO_PERM (ap); + baseaddr =get_table_pte_address (pte, granule_sz); + level++; +continue ; + }else if (is_page_pte (pte, level)) { + gpa =get_page_pte_address (pte, granule_sz);trace_smmu_ptw_page_pte (stage, level, iova, baseaddr, pte_addr, pte, gpa); -return 0 ; - } -if (is_block_pte (pte, level)) { + }else {uint64_t block_size; -hwaddr gpa =get_block_pte_address (pte, level, granule_sz, - &block_size); - - ap =PTE_AP (pte); -if (is_permission_fault (ap, perm)) { - info->type =SMMU_PTW_ERR_PERMISSION ; - goto error; - } + gpa =get_block_pte_address (pte, level, granule_sz, + &block_size);trace_smmu_ptw_block_pte (stage, level, baseaddr, pte_addr, pte, iova, gpa, block_size >>20 ); - - tlbe->translated_addr = gpa + (iova & mask); - tlbe->perm =PTE_AP_TO_PERM (ap); -return 0 ; } - -/* table pte */ - ap =PTE_APTABLE (pte); - + ap =PTE_AP (pte);if (is_permission_fault (ap, perm)) { info->type =SMMU_PTW_ERR_PERMISSION ; goto error; } - baseaddr =get_table_pte_address (pte, granule_sz); - level++; + + tlbe->translated_addr = gpa + (iova & mask); + tlbe->perm =PTE_AP_TO_PERM (ap); +return 0 ; } - info->type =SMMU_PTW_ERR_TRANSLATION ; - error: tlbe->perm =IOMMU_NONE ;return -EINVAL ;
Signature · 1 line
2.20.1
Re: [PATCH RESEND 1/9] hw/arm/smmu-common: Factorize some code in smmu_ptw_64()
Quoted 16 lines · Eric Auger
Page and block PTE decoding can share some code. Let's first handle table PTE and factorize some code shared by page and block PTEs.
Signed-off-by Eric Auger <[email protected]>--- hw/arm/smmu-common.c | 51 ++++++++++++++++---------------------------- 1 file changed, 18 insertions(+), 33 deletions(-)hw/arm/smmu-common.c +0
-0@@ -186,12 +186,12 @@ static int smmu_ptw_64(SMMUTransCfg *cfg,uint64_t subpage_size =1ULL <<level_shift (level, granule_sz);
hw/arm/smmu-common.c static int smmu_ptw_64(SMMUTransCfg *cfg, · line 194
Quoted 10 lines
hw/arm/smmu-common.c +2
-2uint64_t mask = subpage_size -1 ;uint32_t offset =iova_level_offset (iova, inputsize, level, granule_sz); -uint64_t pte; +uint64_t pte, gpa;dma_addr_t pte_addr = baseaddr + offset *sizeof (pte);uint8_t ap;if (get_pte (baseaddr, offset, &pte, info)) { - goto error; +break ;
get_pte() fills in info->type (to SMMU_PTW_ERR_WALK_EABT) on error; changing this from "goto error" to "break" means we'll now execute the "info->type = SMMU_PTW_ERR_TRANSLATION" that comes between the end of the while loop and the error: label, overwriting the wrong error type.
thanks
-- PMM
Re: [PATCH RESEND 1/9] hw/arm/smmu-common: Factorize some code in smmu_ptw_64()
Hi Peter,
Quoted 24 lines · Peter Maydell
Quoted 23 lines · Eric Auger
Page and block PTE decoding can share some code. Let's first handle table PTE and factorize some code shared by page and block PTEs.
Signed-off-by Eric Auger <[email protected]>--- hw/arm/smmu-common.c | 51 ++++++++++++++++---------------------------- 1 file changed, 18 insertions(+), 33 deletions(-)hw/arm/smmu-common.c +1
-1@@ -186,12 +186,12 @@ static int smmu_ptw_64(SMMUTransCfg *cfg,uint64_t subpage_size =1ULL <<level_shift (level, granule_sz);uint64_t mask = subpage_size -1 ;uint32_t offset =iova_level_offset (iova, inputsize, level, granule_sz); -uint64_t pte; +uint64_t pte, gpa;dma_addr_t pte_addr = baseaddr + offset *sizeof (pte);uint8_t ap;
Quoted 9 lines
hw/arm/smmu-common.c static int smmu_ptw_64(SMMUTransCfg *cfg, · line 194
Quoted 3 lines
hw/arm/smmu-common.c +1
-1if (get_pte (baseaddr, offset, &pte, info)) { - goto error; +break ;
get_pte()fills ininfo->type(toSMMU_PTW_ERR_WALK_EABT) on error; changing this from "goto error" to "break" means we'll now execute the "info->type=SMMU_PTW_ERR_TRANSLATION" that comes between the end of the while loop and the error: label, overwriting the wrong error type.
Agreed.
Thanks
Eric
Quoted 4 lines
thanks
-- PMM
PATCH RESEND 2/9 hw/arm/smmu-common: Add IOTLB helpers
In reply to Eric AugerAdd two helpers to lookup for a given IOTLB entry and an one. We also more the tracing there.
--- include/hw/arm/smmu-common.h | 2 ++ hw/arm/smmu-common.c | 36 ++++++++++++++++++++++++++++++++++++ hw/arm/smmuv3.c | 26 ++------------------------ hw/arm/trace-events | 5 +++-- 4 files changed, 43 insertions(+), 26 deletions(-)
include/hw/arm/smmu-common.h +2 -0
@@ -153,6 +153,8 @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid);#define SMMU_IOTLB_MAX_SIZE 256 +IOMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg,hwaddr iova); +void smmu_iotlb_insert (SMMUState *bs,SMMUTransCfg *cfg,IOMMUTLBEntry *entry);void smmu_iotlb_inv_all (SMMUState *s);void smmu_iotlb_inv_asid (SMMUState *s,uint16_t asid);void smmu_iotlb_inv_iova (SMMUState *s,uint16_t asid,dma_addr_t iova);
hw/arm/smmu-common.c +36 -0
@@ -32,6 +32,42 @@/* IOTLB Management */ +IOMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg, +hwaddr iova) +{ +SMMUIOTLBKey key = {.asid = cfg->asid, .iova = iova}; +IOMMUTLBEntry *entry =g_hash_table_lookup (bs->iotlb, &key); + +if (entry) { + cfg->iotlb_hits++; +trace_smmu_iotlb_lookup_hit (cfg->asid, iova, + cfg->iotlb_hits, cfg->iotlb_misses, +100 * cfg->iotlb_hits / + (cfg->iotlb_hits + cfg->iotlb_misses)); + }else { + cfg->iotlb_misses++; +trace_smmu_iotlb_lookup_miss (cfg->asid, iova, + cfg->iotlb_hits, cfg->iotlb_misses, +100 * cfg->iotlb_hits / + (cfg->iotlb_hits + cfg->iotlb_misses)); + } +return entry; +} + +void smmu_iotlb_insert (SMMUState *bs,SMMUTransCfg *cfg,IOMMUTLBEntry *entry) +{ +SMMUIOTLBKey *key =g_new0 (SMMUIOTLBKey,1 ); + +if (g_hash_table_size (bs->iotlb) >=SMMU_IOTLB_MAX_SIZE ) { +smmu_iotlb_inv_all (bs); + } + + key->asid = cfg->asid; + key->iova = entry->iova; +trace_smmu_iotlb_insert (cfg->asid, entry->iova); +g_hash_table_insert (bs->iotlb, key, entry); +} +inline void smmu_iotlb_inv_all (SMMUState *s) {trace_smmu_iotlb_inv_all ();
hw/arm/smmuv3.c +2 -24
@@ -636,7 +636,6 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, .addr_mask = ~(hwaddr)0, .perm = IOMMU_NONE, }; -SMMUIOTLBKey key, *new_key;qemu_mutex_lock (&s->mutex);
@@ -675,16 +674,8 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, page_mask = (1ULL << (tt->granule_sz)) -1 ; aligned_addr = addr & ~page_mask; - key.asid = cfg->asid; - key.iova = aligned_addr; - - cached_entry =g_hash_table_lookup (bs->iotlb, &key); + cached_entry =smmu_iotlb_lookup (bs, cfg, aligned_addr);if (cached_entry) { - cfg->iotlb_hits++; -trace_smmu_iotlb_cache_hit (cfg->asid, aligned_addr, - cfg->iotlb_hits, cfg->iotlb_misses, -100 * cfg->iotlb_hits / - (cfg->iotlb_hits + cfg->iotlb_misses));if ((flag &IOMMU_WO ) && !(cached_entry->perm &IOMMU_WO )) { status =SMMU_TRANS_ERROR ;if (event.record_trans_faults) {
@@ -698,16 +689,6 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, goto epilogue; } - cfg->iotlb_misses++; -trace_smmu_iotlb_cache_miss (cfg->asid, addr & ~page_mask, - cfg->iotlb_hits, cfg->iotlb_misses, -100 * cfg->iotlb_hits / - (cfg->iotlb_hits + cfg->iotlb_misses)); - -if (g_hash_table_size (bs->iotlb) >=SMMU_IOTLB_MAX_SIZE ) { -smmu_iotlb_inv_all (bs); - } - cached_entry =g_new0 (IOMMUTLBEntry,1 );if (smmu_ptw (cfg, aligned_addr, flag, cached_entry, &ptw_info)) {
@@ -753,10 +734,7 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, } status =SMMU_TRANS_ERROR ; }else { - new_key =g_new0 (SMMUIOTLBKey,1 ); - new_key->asid = cfg->asid; - new_key->iova = aligned_addr; -g_hash_table_insert (bs->iotlb, new_key, cached_entry); +smmu_iotlb_insert (bs, cfg, cached_entry); status =SMMU_TRANS_SUCCESS ; }
hw/arm/trace-events +3 -2
@@ -14,6 +14,9 @@ smmu_iotlb_inv_all(void) "IOTLB invalidate all" smmu_iotlb_inv_asid(uint16_t asid) "IOTLB invalidate asid=%d" smmu_iotlb_inv_iova(uint16_t asid, uint64_t addr) "IOTLB invalidate asid=%d addr=0x%"PRIx64 smmu_inv_notifiers_mr(const char *name) "iommu mr=%s" +smmu_iotlb_lookup_hit(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache HIT asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" +smmu_iotlb_lookup_miss(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache MISS asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" +smmu_iotlb_insert(uint16_t asid, uint64_t addr) "IOTLB ++ asid=%d addr=0x%"PRIx64 # smmuv3.c smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)"
@@ -46,8 +49,6 @@ smmuv3_cmdq_tlbi_nh_va(int vmid, int asid, uint64_t addr, bool leaf) "vmid =%d a smmuv3_cmdq_tlbi_nh_vaa(int vmid, uint64_t addr) "vmid =%d addr=0x%"PRIx64 smmuv3_cmdq_tlbi_nh(void) "" smmuv3_cmdq_tlbi_nh_asid(uint16_t asid) "asid=%d" -smmu_iotlb_cache_hit(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache HIT asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" -smmu_iotlb_cache_miss(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache MISS asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid %d" smmuv3_notify_flag_add(const char *iommu) "ADD SMMUNotifier node for iommu mr=%s" smmuv3_notify_flag_del(const char *iommu) "DEL SMMUNotifier node for iommu mr=%s"
Signature · 1 line
2.20.1
Re: [PATCH RESEND 2/9] hw/arm/smmu-common: Add IOTLB helpers
Quoted 3 lines · Eric Auger
Add two helpers to lookup for a given IOTLB entry and an one. We also more the tracing there.
Missing word between "an" and "one" ? Is "more" a typo for "move" ?
Quoted 2 lines
Signed-off-by Eric Auger <[email protected]>---
Otherwise
thanks
-- PMM
PATCH RESEND 3/9 hw/arm/smmu: Simplify the IOTLB key format
In reply to Eric AugerInstead of using a Jenkins hash function to generate the key let's just use a 64 bit unsigned integer that contains the asid and the 40 upper bits of the iova. A maximum of 52-bit IOVA is supported. This change in the key format also prepares for the addition of new fields in subsequent patches (granule and level).
--- hw/arm/smmu-internal.h | 4 +++ include/hw/arm/smmu-common.h | 6 +--- hw/arm/smmu-common.c | 53 ++++++++++++++---------------------- 3 files changed, 26 insertions(+), 37 deletions(-)
hw/arm/smmu-internal.h +4 -0
@@ -96,4 +96,8 @@ uint64_t iova_level_offset(uint64_t iova, int inputsize, MAKE_64BIT_MASK(0, gsz - 3); } +#define SMMU_IOTLB_ASID_SHIFT 40 + +#define SMMU_IOTLB_ASID (key) (((key) >> SMMU_IOTLB_ASID_SHIFT) & 0xFFFF) +#define SMMU_IOTLB_IOVA (key) (((key) & MAKE_64BIT_MASK(0, 40)) << 12)#endif
include/hw/arm/smmu-common.h +1 -5
@@ -88,11 +88,6 @@ typedef struct SMMUPciBus {SMMUDevice *pbdev[];/* Parent array is sparse, so dynamically alloc */ } SMMUPciBus; -typedef struct SMMUIOTLBKey { -uint64_t iova; -uint16_t asid; -}SMMUIOTLBKey ; - typedef struct SMMUState { /* <private> */ SysBusDevice dev;
@@ -155,6 +150,7 @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid);IOMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg,hwaddr iova);void smmu_iotlb_insert (SMMUState *bs,SMMUTransCfg *cfg,IOMMUTLBEntry *entry); +uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova);void smmu_iotlb_inv_all (SMMUState *s);void smmu_iotlb_inv_asid (SMMUState *s,uint16_t asid);void smmu_iotlb_inv_iova (SMMUState *s,uint16_t asid,dma_addr_t iova);
hw/arm/smmu-common.c +21 -32
90 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH RESEND 3/9] hw/arm/smmu: Simplify the IOTLB key format
Quoted 15 lines · Eric Auger
Instead of using a Jenkins hash function to generate the key let's just use a 64 bit unsigned integer that contains the asid and the 40 upper bits of the iova. A maximum of 52-bit IOVA is supported. This change in the key format also prepares for the addition of new fields in subsequent patches (granule and level).
Signed-off-by Eric Auger <[email protected]>include/hw/arm/smmu-common.h +0
-0
include/hw/arm/smmu-common.h typedef struct SMMUPciBus { · line 94
Quoted 7 lines
include/hw/arm/smmu-common.h +0
-4@@ -88,11 +88,6 @@ typedef struct SMMUPciBus {SMMUDevice *pbdev[];/* Parent array is sparse, so dynamically alloc */ } SMMUPciBus; -typedef struct SMMUIOTLBKey { -uint64_t iova; -uint16_t asid; -}SMMUIOTLBKey ;
I think we should keep the SMMUIOTLBKey type to abstract out what the key type is under the hood, so it would now be
typedef uint64_t SMMUIOTLBKey;
(and then the code that works with SMMUIOTLBKeys should never directly look at it as a uint64_t. If you wanted you could put the abstraction layer into place with the existing SMMUIOTLBKey type and then change the type in a second patch.)
include/hw/arm/smmu-common.h IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid); · line 153
Quoted 1 line
include/hw/arm/smmu-common.h +1
-0+uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova);
This should return SMMUIOTLBKey rather than uint64_t,
or pass in the pointer, like:
smmu_get_iotlb_key(SMMUIOTLBKey *key, uint16_t asid, uint64_t iova);
thanks
-- PMM
Re: [PATCH RESEND 3/9] hw/arm/smmu: Simplify the IOTLB key format
Hi Peter,
Quoted 19 lines · Peter Maydell
Quoted 18 lines · Eric Auger
Instead of using a Jenkins hash function to generate the key let's just use a 64 bit unsigned integer that contains the asid and the 40 upper bits of the iova. A maximum of 52-bit IOVA is supported. This change in the key format also prepares for the addition of new fields in subsequent patches (granule and level).
Signed-off-by Eric Auger <[email protected]>include/hw/arm/smmu-common.h +0
-0@@ -88,11 +88,6 @@ typedef struct SMMUPciBus {SMMUDevice *pbdev[];/* Parent array is sparse, so dynamically alloc */ } SMMUPciBus;
Quoted 8 lines
include/hw/arm/smmu-common.h typedef struct SMMUPciBus { · line 94
Quoted 4 lines
include/hw/arm/smmu-common.h +0
-4-typedef struct SMMUIOTLBKey { -uint64_t iova; -uint16_t asid; -}SMMUIOTLBKey ;I think we should keep the SMMUIOTLBKey type to abstract out what the key type is under the hood, so it would now be
typedefuint64_tSMMUIOTLBKey;
OK
Quoted 5 lines
(and then the code that works with SMMUIOTLBKeys should never directly look at it as a
uint64_t. If you wanted you could put the abstraction layer into place with the existing SMMUIOTLBKey type and then change the type in a second patch.)
done this way
Quoted 6 lines
include/hw/arm/smmu-common.h IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid); · line 153
Quoted 2 lines
include/hw/arm/smmu-common.h +1
-0+uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova);This should return SMMUIOTLBKey rather than
uint64_t, or pass in the pointer, like:smmu_get_iotlb_key(SMMUIOTLBKey *key, uint16_t asid, uint64_t iova);
sure
Thanks
Eric
Quoted 4 lines
thanks
-- PMM
PATCH RESEND 4/9 hw/arm/smmu: Introduce SMMUTLBEntry for PTW and IOTLB value
In reply to Eric AugerIntroduce a specialized SMMUTLBEntry to store the result of the PTW and cache in the IOTLB. This structure extends the generic IOMMUTLBEntry struct with the level of the entry and the granule size.
Those latter will be useful when implementing range invalidation.
--- include/hw/arm/smmu-common.h | 14 +++++++++++--- hw/arm/smmu-common.c | 30 ++++++++++++++++-------------- hw/arm/smmuv3.c | 10 +++++----- 3 files changed, 32 insertions(+), 22 deletions(-)
include/hw/arm/smmu-common.h +11 -3
@@ -52,6 +52,14 @@ typedef struct SMMUTransTableInfo {uint8_t granule_sz;/* granule page shift */ } SMMUTransTableInfo; +struct SMMUTLBEntry { +IOMMUTLBEntry entry; +uint8_t level; +uint8_t granule; +}; + +typedef struct SMMUTLBEntry SMMUTLBEntry ; +/* * Generic structure populated by derived SMMU devices * after decoding the configuration information and used as
@@ -135,7 +143,7 @@ static inline uint16_t smmu_get_sid(SMMUDevice *sdev)* pair, according to @cfg translation config */ int smmu_ptw (SMMUTransCfg *cfg,dma_addr_t iova,IOMMUAccessFlags perm, -IOMMUTLBEntry *tlbe,SMMUPTWEventInfo *info); +SMMUTLBEntry *tlbe,SMMUPTWEventInfo *info);/** * select_tt - compute which translation table shall be used according to
@@ -148,8 +156,8 @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid);#define SMMU_IOTLB_MAX_SIZE 256 -IOMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg,hwaddr iova); -void smmu_iotlb_insert (SMMUState *bs,SMMUTransCfg *cfg,IOMMUTLBEntry *entry); +SMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg,hwaddr iova); +void smmu_iotlb_insert (SMMUState *bs,SMMUTransCfg *cfg,SMMUTLBEntry *entry);uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova);void smmu_iotlb_inv_all (SMMUState *s);void smmu_iotlb_inv_asid (SMMUState *s,uint16_t asid);
hw/arm/smmu-common.c +16 -14
@@ -47,11 +47,11 @@ uint64_t smmu_get_iotlb_key(uint16_t asid, uint64_t iova)return iova >>12 | (uint64_t )(asid) <<SMMU_IOTLB_ASID_SHIFT ; } -IOMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg, +SMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg,hwaddr iova) {uint64_t key =smmu_get_iotlb_key (cfg->asid, iova); -IOMMUTLBEntry *entry =g_hash_table_lookup (bs->iotlb, &key); +SMMUTLBEntry *entry =g_hash_table_lookup (bs->iotlb, &key);if (entry) { cfg->iotlb_hits++;
@@ -69,7 +69,7 @@ IOMMUTLBEntry *smmu_iotlb_lookup(SMMUState *bs, SMMUTransCfg *cfg,return entry; } -void smmu_iotlb_insert (SMMUState *bs,SMMUTransCfg *cfg,IOMMUTLBEntry *entry) +void smmu_iotlb_insert (SMMUState *bs,SMMUTransCfg *cfg,SMMUTLBEntry *new) {uint64_t *key =g_new0 (uint64_t,1 );
@@ -77,9 +77,9 @@ void smmu_iotlb_insert(SMMUState *bs, SMMUTransCfg *cfg, IOMMUTLBEntry *entry)smmu_iotlb_inv_all (bs); } - *key =smmu_get_iotlb_key (cfg->asid, entry->iova); -trace_smmu_iotlb_insert (cfg->asid, entry->iova); -g_hash_table_insert (bs->iotlb, key, entry); + *key =smmu_get_iotlb_key (cfg->asid, new->entry.iova); +trace_smmu_iotlb_insert (cfg->asid, new->entry.iova); +g_hash_table_insert (bs->iotlb, key, new); }inline void smmu_iotlb_inv_all (SMMUState *s)
@@ -199,7 +199,7 @@ SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova)* @cfg: translation config * @iova: iova to translate * @perm: access type -* @tlbe: IOMMUTLBEntry (out) +* @tlbe: SMMUTLBEntry (out) * @info: handle to an error info * * Return 0 on success, < 0 on error. In case of error, @info is filled
@@ -209,7 +209,7 @@ SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova)*/ static int smmu_ptw_64 (SMMUTransCfg *cfg,dma_addr_t iova,IOMMUAccessFlags perm, -IOMMUTLBEntry *tlbe,SMMUPTWEventInfo *info) +SMMUTLBEntry *tlbe,SMMUPTWEventInfo *info) {dma_addr_t baseaddr, indexmask;int stage = cfg->stage;
@@ -229,8 +229,8 @@ static int smmu_ptw_64(SMMUTransCfg *cfg, baseaddr =extract64 (tt->ttb,0 ,48 ); baseaddr &= ~indexmask; - tlbe->iova = iova; - tlbe->addr_mask = (1 << granule_sz) -1 ; + tlbe->entry.iova = iova; + tlbe->entry.addr_mask = (1 << granule_sz) -1 ;while (level <=3 ) {uint64_t subpage_size =1ULL <<level_shift (level, granule_sz);
@@ -281,13 +281,15 @@ static int smmu_ptw_64(SMMUTransCfg *cfg, goto error; } - tlbe->translated_addr = gpa + (iova & mask); - tlbe->perm =PTE_AP_TO_PERM (ap); + tlbe->entry.translated_addr = gpa + (iova & mask); + tlbe->entry.perm =PTE_AP_TO_PERM (ap); + tlbe->level = level; + tlbe->granule = granule_sz;return 0 ; } info->type =SMMU_PTW_ERR_TRANSLATION ; error: - tlbe->perm =IOMMU_NONE ; + tlbe->entry.perm =IOMMU_NONE ;return -EINVAL ; }
@@ -303,7 +305,7 @@ error:* return 0 on success */ inline int smmu_ptw (SMMUTransCfg *cfg,dma_addr_t iova,IOMMUAccessFlags perm, -IOMMUTLBEntry *tlbe,SMMUPTWEventInfo *info) +SMMUTLBEntry *tlbe,SMMUPTWEventInfo *info) {if (!cfg->aa64) {/*
hw/arm/smmuv3.c +5 -5
@@ -626,7 +626,7 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr,SMMUTranslationStatus status;SMMUState *bs =ARM_SMMU (s);uint64_t page_mask, aligned_addr; -IOMMUTLBEntry *cached_entry =NULL ; +SMMUTLBEntry *cached_entry =NULL ;SMMUTransTableInfo *tt;SMMUTransCfg *cfg =NULL ;IOMMUTLBEntry entry = {
@@ -676,7 +676,7 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, cached_entry =smmu_iotlb_lookup (bs, cfg, aligned_addr);if (cached_entry) { -if ((flag &IOMMU_WO ) && !(cached_entry->perm &IOMMU_WO )) { +if ((flag &IOMMU_WO ) && !(cached_entry->entry.perm &IOMMU_WO )) { status =SMMU_TRANS_ERROR ;if (event.record_trans_faults) { event.type =SMMU_EVT_F_PERMISSION ;
@@ -689,7 +689,7 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, goto epilogue; } - cached_entry =g_new0 (IOMMUTLBEntry,1 ); + cached_entry =g_new0 (SMMUTLBEntry,1 );if (smmu_ptw (cfg, aligned_addr, flag, cached_entry, &ptw_info)) {g_free (cached_entry);
@@ -743,9 +743,9 @@ epilogue:switch (status) {case SMMU_TRANS_SUCCESS : entry.perm = flag; - entry.translated_addr = cached_entry->translated_addr + + entry.translated_addr = cached_entry->entry.translated_addr + (addr & page_mask); - entry.addr_mask = cached_entry->addr_mask; + entry.addr_mask = cached_entry->entry.addr_mask;trace_smmuv3_translate_success (mr->parent_obj.name, sid, addr, entry.translated_addr, entry.perm);break ;
Signature · 1 line
2.20.1
Re: [PATCH RESEND 4/9] hw/arm/smmu: Introduce SMMUTLBEntry for PTW and IOTLB value
Quoted 20 lines · Eric Auger
Introduce a specialized SMMUTLBEntry to store the result of the PTW and cache in the IOTLB. This structure extends the generic IOMMUTLBEntry struct with the level of the entry and the granule size.
Those latter will be useful when implementing range invalidation.
Signed-off-by Eric Auger <[email protected]>--- include/hw/arm/smmu-common.h | 14 +++++++++++--- hw/arm/smmu-common.c | 30 ++++++++++++++++-------------- hw/arm/smmuv3.c | 10 +++++----- 3 files changed, 32 insertions(+), 22 deletions(-)include/hw/arm/smmu-common.h +0
-0
include/hw/arm/smmu-common.h typedef struct SMMUTransTableInfo { · line 61
Quoted 10 lines
include/hw/arm/smmu-common.h +7
-0@@ -52,6 +52,14 @@ typedef struct SMMUTransTableInfo {uint8_t granule_sz;/* granule page shift */ } SMMUTransTableInfo; +struct SMMUTLBEntry { +IOMMUTLBEntry entry; +uint8_t level; +uint8_t granule; +}; + +typedef struct SMMUTLBEntry SMMUTLBEntry ;
Every other typedef in this header uses
typedef struct Foo {
...} Foo;
rather than a separate typedef line; could we be consistent with that?
hw/arm/smmu-common.c uint64_t smmu_get_iotlb_key(uint16_t asid, uint64_t iova) · line 51
Quoted 11 lines
hw/arm/smmu-common.c +1
-1@@ -47,11 +47,11 @@ uint64_t smmu_get_iotlb_key(uint16_t asid, uint64_t iova)return iova >>12 | (uint64_t )(asid) <<SMMU_IOTLB_ASID_SHIFT ; } -IOMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg, +SMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg,hwaddr iova)
Indent on this second line needs to move in one space now.
Quoted 9 lines
hw/arm/smmu-common.c +1
-1{uint64_t key =smmu_get_iotlb_key (cfg->asid, iova); -IOMMUTLBEntry *entry =g_hash_table_lookup (bs->iotlb, &key); +SMMUTLBEntry *entry =g_hash_table_lookup (bs->iotlb, &key);if (entry) { cfg->iotlb_hits++;
hw/arm/smmu-common.c error: · line 308
Quoted 5 lines
hw/arm/smmu-common.c +1
-1@@ -303,7 +305,7 @@ error:* return 0 on success */ inline int smmu_ptw (SMMUTransCfg *cfg,dma_addr_t iova,IOMMUAccessFlags perm, -IOMMUTLBEntry *tlbe,SMMUPTWEventInfo *info) +SMMUTLBEntry *tlbe,SMMUPTWEventInfo *info)
might as well fix this indent while we're editing the line...
hw/arm/smmu-common.c error: · line 311
Quoted 3 lines
hw/arm/smmu-common.c +0
-0{if (!cfg->aa64) {/*
otherwise
thanks
-- PMM
PATCH RESEND 5/9 hw/arm/smmuv3: Store the starting level in SMMUTransTableInfo
In reply to Eric AugerCompute the starting level on CD decoding and store it into SMMUTransTableInfo. We will need this information on IOTLB lookup so let's avoid to recompute it each time.
--- include/hw/arm/smmu-common.h | 1 + hw/arm/smmu-common.c | 2 +- hw/arm/smmuv3.c | 8 ++++++-- hw/arm/trace-events | 2 +- 4 files changed, 9 insertions(+), 4 deletions(-)
include/hw/arm/smmu-common.h +1 -0
@@ -50,6 +50,7 @@ typedef struct SMMUTransTableInfo {uint64_t ttb;/* TT base address */ uint8_t tsz;/* input range, ie. 2^(64 -tsz)*/ uint8_t granule_sz;/* granule page shift */ +uint8_t starting_level;/* starting level */ } SMMUTransTableInfo;struct SMMUTLBEntry {
hw/arm/smmu-common.c +1 -1
@@ -224,7 +224,7 @@ static int smmu_ptw_64(SMMUTransCfg *cfg, granule_sz = tt->granule_sz; stride = granule_sz -3 ; inputsize =64 - tt->tsz; - level =4 - (inputsize -4 ) / stride; + level = tt->starting_level; indexmask = (1ULL << (inputsize - (stride * (4 - level)))) -1 ; baseaddr =extract64 (tt->ttb,0 ,48 ); baseaddr &= ~indexmask;
hw/arm/smmuv3.c +6 -2
@@ -482,7 +482,7 @@ static int decode_cd(SMMUTransCfg *cfg, CD *cd, SMMUEventInfo *event)/* decode data dependent on TT */ for (i =0 ; i <=1 ; i++) { -int tg, tsz; +int tg, tsz, input_size, stride;SMMUTransTableInfo *tt = &cfg->tt[i]; cfg->tt[i].disabled =CD_EPD (cd, i);
@@ -502,11 +502,15 @@ static int decode_cd(SMMUTransCfg *cfg, CD *cd, SMMUEventInfo *event) } tt->tsz = tsz; + input_size =64 - tt->tsz; + stride = tt->granule_sz -3 ; tt->ttb =CD_TTB (cd, i);if (tt->ttb & ~(MAKE_64BIT_MASK (0 , cfg->oas))) { goto bad_cd; } -trace_smmuv3_decode_cd_tt (i, tt->tsz, tt->ttb, tt->granule_sz); + tt->starting_level =4 - (input_size -4 ) / stride; +trace_smmuv3_decode_cd_tt (i, tt->tsz, tt->ttb, + tt->granule_sz, tt->starting_level); } event->record_trans_faults =CD_R (cd);
hw/arm/trace-events +1 -1
@@ -39,7 +39,7 @@ smmuv3_translate_abort(const char *n, uint16_t sid, uint64_t addr, bool is_write smmuv3_translate_success(const char *n, uint16_t sid, uint64_t iova, uint64_t translated, int perm) "%s sid=%d iova=0x%"PRIx64" translated=0x%"PRIx64" perm=0x%x" smmuv3_get_cd(uint64_t addr) "CD addr: 0x%"PRIx64 smmuv3_decode_cd(uint32_t oas) "oas=%d" -smmuv3_decode_cd_tt(int i, uint32_t tsz, uint64_t ttb, uint32_t granule_sz) "TT[%d]:tsz:%d ttb:0x%"PRIx64" granule_sz:%d" +smmuv3_decode_cd_tt(int i, uint32_t tsz, uint64_t ttb, uint32_t granule_sz, uint8_t starting_level) "TT[%d]:tsz:%d ttb:0x%"PRIx64" granule_sz:%d starting_level=%d" smmuv3_cmdq_cfgi_ste(int streamid) "streamid =%d" smmuv3_cmdq_cfgi_ste_range(int start, int end) "start=0x%d - end=0x%d" smmuv3_cmdq_cfgi_cd(uint32_t sid) "streamid = %d"
Signature · 1 line
2.20.1
Re: [PATCH RESEND 5/9] hw/arm/smmuv3: Store the starting level in SMMUTransTableInfo
Quoted 10 lines · Eric Auger
Compute the starting level on CD decoding and store it into SMMUTransTableInfo. We will need this information on IOTLB lookup so let's avoid to recompute it each time.
Signed-off-by Eric Auger <[email protected]>
hw/arm/smmu-common.c static int smmu_ptw_64(SMMUTransCfg *cfg, · line 227
Quoted 5 lines
hw/arm/smmu-common.c +1
-1@@ -224,7 +224,7 @@ static int smmu_ptw_64(SMMUTransCfg *cfg, granule_sz = tt->granule_sz; stride = granule_sz -3 ; inputsize =64 - tt->tsz; - level =4 - (inputsize -4 ) / stride; + level = tt->starting_level;
"4 - (x - 4) / y" doesn't really seem like it's complicated enough to be worth caching given everything else we do on a page table walk. Do you have perf figures to indicate that this change is worthwhile?
thanks
-- PMM
Re: [PATCH RESEND 5/9] hw/arm/smmuv3: Store the starting level in SMMUTransTableInfo
Hi Peter,
Quoted 8 lines · Peter Maydell
Quoted 6 lines · Eric Auger
Compute the starting level on CD decoding and store it into SMMUTransTableInfo. We will need this information on IOTLB lookup so let's avoid to recompute it each time.
Signed-off-by Eric Auger <[email protected]>
Quoted 13 lines
hw/arm/smmu-common.c static int smmu_ptw_64(SMMUTransCfg *cfg, · line 227
Quoted 8 lines
hw/arm/smmu-common.c +1
-1@@ -224,7 +224,7 @@ static int smmu_ptw_64(SMMUTransCfg *cfg, granule_sz = tt->granule_sz; stride = granule_sz -3 ; inputsize =64 - tt->tsz; - level =4 - (inputsize -4 ) / stride; + level = tt->starting_level;"4 - (x - 4) / y" doesn't really seem like it's complicated enough to be worth caching given everything else we do on a page table walk. Do you have perf figures to indicate that this change is worthwhile?
no I don't have any figure or any setup at the moment to perform that bench.
I thought it could only help, was not complicated to store and was of the same kind as the granule size, the input range, ... already stored in SMMUTransTableInfo.
Thanks
Eric
Quoted 4 lines
thanks
-- PMM
PATCH RESEND 6/9 hw/arm/smmu-common: Manage IOTLB block entries
In reply to Eric AugerAt the moment each entry in the IOTLB corresponds to a page sized mapping (4K, 16K or 64K), even if the page belongs to a mapped block. In case of block mapping this unefficiently consume IOTLB entries.
Change the value of the entry so that it reflects the actual mapping it belongs to (block or page start address and size).
Also the level/tg of the entry is encoded in the key. In subsequent patches we will enable range invalidation. This latter is able to provide the level/tg of the entry.
--- hw/arm/smmu-internal.h | 10 ++++++ include/hw/arm/smmu-common.h | 8 +++-- hw/arm/smmu-common.c | 61 +++++++++++++++++++++++++++--------- hw/arm/smmuv3.c | 6 ++-- hw/arm/trace-events | 2 +- 5 files changed, 66 insertions(+), 21 deletions(-)
hw/arm/smmu-internal.h +10 -0
@@ -97,7 +97,17 @@ uint64_t iova_level_offset(uint64_t iova, int inputsize, }#define SMMU_IOTLB_ASID_SHIFT 40 +#define SMMU_IOTLB_LEVEL_SHIFT 56 +#define SMMU_IOTLB_TG_SHIFT 58#define SMMU_IOTLB_ASID (key) (((key) >> SMMU_IOTLB_ASID_SHIFT) & 0xFFFF)#define SMMU_IOTLB_IOVA (key) (((key) & MAKE_64BIT_MASK(0, 40)) << 12) + +struct SMMUIOTLBPageInvInfo { +int asid; +uint64_t iova; +uint64_t mask; +}; +typedef struct SMMUIOTLBPageInvInfo SMMUIOTLBPageInvInfo ; +#endif
include/hw/arm/smmu-common.h +5 -3
@@ -157,12 +157,14 @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid);#define SMMU_IOTLB_MAX_SIZE 256 -SMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg,hwaddr iova); +SMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg, +SMMUTransTableInfo *tt,hwaddr iova);void smmu_iotlb_insert (SMMUState *bs,SMMUTransCfg *cfg,SMMUTLBEntry *entry); -uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova); +uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova, +uint8_t tg,uint8_t level);void smmu_iotlb_inv_all (SMMUState *s);void smmu_iotlb_inv_asid (SMMUState *s,uint16_t asid); -void smmu_iotlb_inv_iova (SMMUState *s,uint16_t asid,dma_addr_t iova); +void smmu_iotlb_inv_iova (SMMUState *s,int asid,dma_addr_t iova);/* Unmap the range of all the notifiers registered to any IOMMU mr */ void smmu_inv_notifiers_all (SMMUState *s);
hw/arm/smmu-common.c +47 -14
104 lines not on this page · raw message
hw/arm/smmuv3.c +3 -3
@@ -678,7 +678,7 @@ static IOMMUTLBEntry smmuv3_translate(IOMMUMemoryRegion *mr, hwaddr addr, page_mask = (1ULL << (tt->granule_sz)) -1 ; aligned_addr = addr & ~page_mask; - cached_entry =smmu_iotlb_lookup (bs, cfg, aligned_addr); + cached_entry =smmu_iotlb_lookup (bs, cfg, tt, aligned_addr);if (cached_entry) {if ((flag &IOMMU_WO ) && !(cached_entry->entry.perm &IOMMU_WO )) { status =SMMU_TRANS_ERROR ;
@@ -748,7 +748,7 @@ epilogue:case SMMU_TRANS_SUCCESS : entry.perm = flag; entry.translated_addr = cached_entry->entry.translated_addr + - (addr & page_mask); + (addr & cached_entry->entry.addr_mask); entry.addr_mask = cached_entry->entry.addr_mask;trace_smmuv3_translate_success (mr->parent_obj.name, sid, addr, entry.translated_addr, entry.perm);
@@ -976,7 +976,7 @@ static int smmuv3_cmdq_consume(SMMUv3State *s)trace_smmuv3_cmdq_tlbi_nh_vaa (vmid, addr);smmuv3_inv_notifiers_iova (bs,-1 , addr); -smmu_iotlb_inv_all (bs); +smmu_iotlb_inv_iova (bs,-1 , addr);break ; }case SMMU_CMD_TLBI_NH_VA :
hw/arm/trace-events +1 -1
@@ -16,7 +16,7 @@ smmu_iotlb_inv_iova(uint16_t asid, uint64_t addr) "IOTLB invalidate asid=%d addr smmu_inv_notifiers_mr(const char *name) "iommu mr=%s" smmu_iotlb_lookup_hit(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache HIT asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" smmu_iotlb_lookup_miss(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, uint32_t p) "IOTLB cache MISS asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%d" -smmu_iotlb_insert(uint16_t asid, uint64_t addr) "IOTLB ++ asid=%d addr=0x%"PRIx64 +smmu_iotlb_insert(uint16_t asid, uint64_t addr, uint8_t tg, uint8_t level) "IOTLB ++ asid=%d addr=0x%"PRIx64" tg=%d level=%d" # smmuv3.c smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)"
Signature · 1 line
2.20.1
Re: [PATCH RESEND 6/9] hw/arm/smmu-common: Manage IOTLB block entries
Quoted 37 lines · Eric Auger
At the moment each entry in the IOTLB corresponds to a page sized mapping (4K, 16K or 64K), even if the page belongs to a mapped block. In case of block mapping this unefficiently consume IOTLB entries.
Change the value of the entry so that it reflects the actual mapping it belongs to (block or page start address and size).
Also the level/tg of the entry is encoded in the key. In subsequent patches we will enable range invalidation. This latter is able to provide the level/tg of the entry.
Signed-off-by Eric Auger <[email protected]>hw/arm/smmu-common.c +11
-5-uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova) +uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova, +uint8_t tg,uint8_t level) { -return iova >>12 | (uint64_t )(asid) <<SMMU_IOTLB_ASID_SHIFT ; +return iova >>12 | (uint64_t )(asid) <<SMMU_IOTLB_ASID_SHIFT | + (uint64_t )(level) <<SMMU_IOTLB_LEVEL_SHIFT | + (uint64_t )(tg) <<SMMU_IOTLB_TG_SHIFT ; }SMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg, -hwaddr iova) +SMMUTransTableInfo *tt,hwaddr iova) { -uint64_t key =smmu_get_iotlb_key (cfg->asid, iova); -SMMUTLBEntry *entry =g_hash_table_lookup (bs->iotlb, &key); +uint8_t tg = (tt->granule_sz -10 ) /2 ; +int level = tt->starting_level; +SMMUTLBEntry *entry =NULL ; + +while (level <=3 ) {
hw/arm/smmu-common.c static gboolean smmu_iotlb_key_equal(gconstpointer v1, gconstpointer v2) · line 70
Quoted 10 lines
hw/arm/smmu-common.c +10
-0+uint64_t subpage_size =1ULL <<level_shift (level, tt->granule_sz); +uint64_t mask = subpage_size -1 ; +uint64_t key; + + key =smmu_get_iotlb_key (cfg->asid, iova & ~mask, tg, level); + entry =g_hash_table_lookup (bs->iotlb, &key); +if (entry) { +break ; + } + level++;
Rather than looping around doing multiple hash table lookups like this, why not just avoid including the tg and level in the key equality test?
If I understand the range-based-invalidation feature correctly, the only time we care about the TG/LVL is if we're processing an invalidate-range command that specifies them. But in that case there should never be multiple entries in the bs->iotlb with the same iova, so we can just check whether the entry matches the requested TG/LVL once we've pulled it out of the hash table. (Or we could architecturally validly just blow it away regardless of requested TG/LVL -- they are only hints, not required-to-match.)
thanks
-- PMM
Re: [PATCH RESEND 6/9] hw/arm/smmu-common: Manage IOTLB block entries
Hi Peter,
Quoted 53 lines · Peter Maydell
Quoted 37 lines · Eric Auger
At the moment each entry in the IOTLB corresponds to a page sized mapping (4K, 16K or 64K), even if the page belongs to a mapped block. In case of block mapping this unefficiently consume IOTLB entries.
Change the value of the entry so that it reflects the actual mapping it belongs to (block or page start address and size).
Also the level/tg of the entry is encoded in the key. In subsequent patches we will enable range invalidation. This latter is able to provide the level/tg of the entry.
Signed-off-by Eric Auger <[email protected]>hw/arm/smmu-common.c +11
-5-uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova) +uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova, +uint8_t tg,uint8_t level) { -return iova >>12 | (uint64_t )(asid) <<SMMU_IOTLB_ASID_SHIFT ; +return iova >>12 | (uint64_t )(asid) <<SMMU_IOTLB_ASID_SHIFT | + (uint64_t )(level) <<SMMU_IOTLB_LEVEL_SHIFT | + (uint64_t )(tg) <<SMMU_IOTLB_TG_SHIFT ; }SMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg, -hwaddr iova) +SMMUTransTableInfo *tt,hwaddr iova) { -uint64_t key =smmu_get_iotlb_key (cfg->asid, iova); -SMMUTLBEntry *entry =g_hash_table_lookup (bs->iotlb, &key); +uint8_t tg = (tt->granule_sz -10 ) /2 ; +int level = tt->starting_level; +SMMUTLBEntry *entry =NULL ; + +while (level <=3 ) {hw/arm/smmu-common.c static gboolean smmu_iotlb_key_equal(gconstpointer v1, gconstpointer v2) · line 70
Quoted 10 lines
hw/arm/smmu-common.c +10
-0+uint64_t subpage_size =1ULL <<level_shift (level, tt->granule_sz); +uint64_t mask = subpage_size -1 ; +uint64_t key; + + key =smmu_get_iotlb_key (cfg->asid, iova & ~mask, tg, level); + entry =g_hash_table_lookup (bs->iotlb, &key); +if (entry) { +break ; + } + level++;Rather than looping around doing multiple hash table lookups like this, why not just avoid including the tg and level in the key equality test?
Quoted 9 lines
If I understand the range-based-invalidation feature correctly, the only time we care about the TG/LVL is if we're processing an invalidate-range command that specifies them. But in that case there should never be multiple entries in the
bs->iotlbwith the same iova, so we can just check whether the entry matches the requested TG/LVL once we've pulled it out of the hash table. (Or we could architecturally validly just blow it away regardless of requested TG/LVL -- they are only hints, not required-to-match.)
This change could have been done independently on the RIL feature. As we now put block entries in the IOTLB , when we look for an iova translation, the IOVA can be mapped using different block sizes or using page entries. So we start looking at blocks of the bigger size (entry level) downto the page, for instance 4TB/512MB/64KB. We cannot know which block and size the address belongs to. I do not know if we can make any hypothesis on whether the driver is forbidden to invalidate an address that is not the starting address of an initial mapping.
Not a justification but an info, this is implemented the same way on x86 (except they don't have variable TG), see vtd_lookup_iotlb in hw/i386/intel_iommu.c
Thanks
Eric
Quoted 4 lines
thanks
-- PMM
Re: [PATCH RESEND 6/9] hw/arm/smmu-common: Manage IOTLB block entries
Hi Peter,
Quoted 75 lines · Auger Eric
Hi Peter,
Quoted 53 lines · Peter Maydell
Quoted 37 lines · Eric Auger
At the moment each entry in the IOTLB corresponds to a page sized mapping (4K, 16K or 64K), even if the page belongs to a mapped block. In case of block mapping this unefficiently consume IOTLB entries.
Change the value of the entry so that it reflects the actual mapping it belongs to (block or page start address and size).
Also the level/tg of the entry is encoded in the key. In subsequent patches we will enable range invalidation. This latter is able to provide the level/tg of the entry.
Signed-off-by Eric Auger <[email protected]>hw/arm/smmu-common.c +11
-5-uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova) +uint64_t smmu_get_iotlb_key (uint16_t asid,uint64_t iova, +uint8_t tg,uint8_t level) { -return iova >>12 | (uint64_t )(asid) <<SMMU_IOTLB_ASID_SHIFT ; +return iova >>12 | (uint64_t )(asid) <<SMMU_IOTLB_ASID_SHIFT | + (uint64_t )(level) <<SMMU_IOTLB_LEVEL_SHIFT | + (uint64_t )(tg) <<SMMU_IOTLB_TG_SHIFT ; }SMMUTLBEntry *smmu_iotlb_lookup (SMMUState *bs,SMMUTransCfg *cfg, -hwaddr iova) +SMMUTransTableInfo *tt,hwaddr iova) { -uint64_t key =smmu_get_iotlb_key (cfg->asid, iova); -SMMUTLBEntry *entry =g_hash_table_lookup (bs->iotlb, &key); +uint8_t tg = (tt->granule_sz -10 ) /2 ; +int level = tt->starting_level; +SMMUTLBEntry *entry =NULL ; + +while (level <=3 ) {hw/arm/smmu-common.c static gboolean smmu_iotlb_key_equal(gconstpointer v1, gconstpointer v2) · line 70
Quoted 10 lines
hw/arm/smmu-common.c +10
-0+uint64_t subpage_size =1ULL <<level_shift (level, tt->granule_sz); +uint64_t mask = subpage_size -1 ; +uint64_t key; + + key =smmu_get_iotlb_key (cfg->asid, iova & ~mask, tg, level); + entry =g_hash_table_lookup (bs->iotlb, &key); +if (entry) { +break ; + } + level++;Rather than looping around doing multiple hash table lookups like this, why not just avoid including the tg and level in the key equality test?
Quoted 9 lines
If I understand the range-based-invalidation feature correctly, the only time we care about the TG/LVL is if we're processing an invalidate-range command that specifies them. But in that case there should never be multiple entries in the
bs->iotlbwith the same iova, so we can just check whether the entry matches the requested TG/LVL once we've pulled it out of the hash table. (Or we could architecturally validly just blow it away regardless of requested TG/LVL -- they are only hints, not required-to-match.)This change could have been done independently on the RIL feature. As we now put block entries in the IOTLB , when we look for an iova translation, the IOVA can be mapped using different block sizes or using page entries. So we start looking at blocks of the bigger size (entry level) downto the page, for instance 4TB/512MB/64KB. We cannot know which block and size the address belongs to. I do not know if we can make any hypothesis on whether the driver is forbidden to invalidate an address that is not the starting address of an initial mapping.
Quoted 3 lines
Not a justification but an info, this is implemented the same way on x86 (except they don't have variable TG), see
vtd_lookup_iotlbinhw/i386/intel_iommu.c
Does that make more sense overall? I will respin shortly.
Thanks
Eric
Quoted 10 lines
Thanks
Eric
Quoted 4 lines
thanks
-- PMM
Re: [PATCH RESEND 6/9] hw/arm/smmu-common: Manage IOTLB block entries
In reply to Auger EricQuoted 15 lines · Auger Eric
Quoted 4 lines · Peter Maydell
Rather than looping around doing multiple hash table lookups like this, why not just avoid including the tg and level in the key equality test?
Quoted 9 lines
If I understand the range-based-invalidation feature correctly, the only time we care about the TG/LVL is if we're processing an invalidate-range command that specifies them. But in that case there should never be multiple entries in the
bs->iotlbwith the same iova, so we can just check whether the entry matches the requested TG/LVL once we've pulled it out of the hash table. (Or we could architecturally validly just blow it away regardless of requested TG/LVL -- they are only hints, not required-to-match.)
Quoted 6 lines
This change could have been done independently on the RIL feature. As we now put block entries in the IOTLB , when we look for an iova translation, the IOVA can be mapped using different block sizes or using page entries. So we start looking at blocks of the bigger size (entry level) downto the page, for instance 4TB/512MB/64KB. We cannot know which block and size the address belongs to.
Yes, but we wouldn't need to care which TG and LVL the address belongs to if we didn't put them into
the key, would we? I'm probably missing something here, but just because the hardware might want to use the hints in the invalidation-command about TG and LVL doesn't inherently mean QEMU is most efficient if it cares about the hints.
thanks
-- PMM
Re: [PATCH RESEND 6/9] hw/arm/smmu-common: Manage IOTLB block entries
Hi Peter,
Quoted 23 lines · Peter Maydell
Quoted 15 lines · Auger Eric
Quoted 4 lines · Peter Maydell
Rather than looping around doing multiple hash table lookups like this, why not just avoid including the tg and level in the key equality test?
Quoted 9 lines
If I understand the range-based-invalidation feature correctly, the only time we care about the TG/LVL is if we're processing an invalidate-range command that specifies them. But in that case there should never be multiple entries in the
bs->iotlbwith the same iova, so we can just check whether the entry matches the requested TG/LVL once we've pulled it out of the hash table. (Or we could architecturally validly just blow it away regardless of requested TG/LVL -- they are only hints, not required-to-match.)Quoted 6 lines
This change could have been done independently on the RIL feature. As we now put block entries in the IOTLB , when we look for an iova translation, the IOVA can be mapped using different block sizes or using page entries. So we start looking at blocks of the bigger size (entry level) downto the page, for instance 4TB/512MB/64KB. We cannot know which block and size the address belongs to.
Quoted 6 lines
Yes, but we wouldn't need to care which TG and LVL the address belongs to if we didn't put them into
the key, would we? I'm probably missing something here, but just because the hardware might want to use the hints in the invalidation-command about TG and LVL doesn't inherently mean QEMU is most efficient if it cares about the hints.
OK I think I understand your point now. It is not necessary to put TG/LVL in the key as log as they are in the entry. I will look at this implementation ...
Thanks
Eric
Quoted 4 lines
thanks
-- PMM
Re: [PATCH RESEND 6/9] hw/arm/smmu-common: Manage IOTLB block entries
Hi Peter,
Quoted 33 lines · Auger Eric
Hi Peter,
Quoted 23 lines · Peter Maydell
Quoted 15 lines · Auger Eric
Quoted 4 lines · Peter Maydell
Rather than looping around doing multiple hash table lookups like this, why not just avoid including the tg and level in the key equality test?
Quoted 9 lines
If I understand the range-based-invalidation feature correctly, the only time we care about the TG/LVL is if we're processing an invalidate-range command that specifies them. But in that case there should never be multiple entries in the
bs->iotlbwith the same iova, so we can just check whether the entry matches the requested TG/LVL once we've pulled it out of the hash table. (Or we could architecturally validly just blow it away regardless of requested TG/LVL -- they are only hints, not required-to-match.)Quoted 6 lines
This change could have been done independently on the RIL feature. As we now put block entries in the IOTLB , when we look for an iova translation, the IOVA can be mapped using different block sizes or using page entries. So we start looking at blocks of the bigger size (entry level) downto the page, for instance 4TB/512MB/64KB. We cannot know which block and size the address belongs to.
Quoted 6 lines
Yes, but we wouldn't need to care which TG and LVL the address belongs to if we didn't put them into
the key, would we? I'm probably missing something here, but just because the hardware might want to use the hints in the invalidation-command about TG and LVL doesn't inherently mean QEMU is most efficient if it cares about the hints.
Quoted 3 lines
OK I think I understand your point now. It is not necessary to put TG/LVL in the key as log as they are in the entry. I will look at this implementation ...
Looking further at the implementation, if we don't encode the LVL in the key (but just encode the block addr), on invalidation, we are not able to remove the associated entry in one shot, using g_hash_table_remove(). We are obliged to use g_hash_table_foreach_remove and sort out the entries according to the invalidation parameters.
Putting the TG and LVL in the key allows to do this in one short if num_pages == 1. See [8/9] hw/arm/smmuv3: Get prepared for range invalidation, smmu_iotlb_inv_iova() implementation.
So my understanding is it is obviously feasible to get rid of TG/LVL in the key but may be less optimal when range invalidation gets used by the guest (most invalidations having num_pages == 1)
Thanks
Eric
Quoted 10 lines
Thanks
Eric
Quoted 4 lines
thanks
-- PMM
PATCH RESEND 7/9 hw/arm/smmuv3: Introduce smmuv3_s1_range_inval() helper
In reply to Eric AugerLet's introduce an helper for S1 IOVA range invalidation. This will be used for NH_VA and NH_VAA commands. It decodes the same fields, trace, calls the UNMAP notifiers and invalidate the corresponding IOTLB entries.
At the moment, we do not support 3.2 range invalidation yet. So it reduces to a single IOVA invalidation.
--- hw/arm/smmuv3.c | 36 +++++++++++++++++------------------- hw/arm/trace-events | 3 +-- 2 files changed, 18 insertions(+), 21 deletions(-)
hw/arm/smmuv3.c +17 -19
@@ -840,6 +840,22 @@ static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova) } } +static void smmuv3_s1_range_inval (SMMUState *s,Cmd *cmd) +{ +dma_addr_t addr =CMD_ADDR (cmd); +uint8_t type =CMD_TYPE (cmd); +uint16_t vmid =CMD_VMID (cmd); +bool leaf =CMD_LEAF (cmd); +int asid =-1 ; + +if (type ==SMMU_CMD_TLBI_NH_VA ) { + asid =CMD_ASID (cmd); + } +trace_smmuv3_s1_range_inval (vmid, asid, addr, leaf); +smmuv3_inv_notifiers_iova (s, asid, addr); +smmu_iotlb_inv_iova (s, asid, addr); +} +static int smmuv3_cmdq_consume (SMMUv3State *s) {SMMUState *bs =ARM_SMMU (s);
@@ -970,27 +986,9 @@ static int smmuv3_cmdq_consume(SMMUv3State *s)smmu_iotlb_inv_all (bs);break ;case SMMU_CMD_TLBI_NH_VAA : - { -dma_addr_t addr =CMD_ADDR (&cmd); -uint16_t vmid =CMD_VMID (&cmd); - -trace_smmuv3_cmdq_tlbi_nh_vaa (vmid, addr); -smmuv3_inv_notifiers_iova (bs,-1 , addr); -smmu_iotlb_inv_iova (bs,-1 , addr); -break ; - }case SMMU_CMD_TLBI_NH_VA : - { -uint16_t asid =CMD_ASID (&cmd); -uint16_t vmid =CMD_VMID (&cmd); -dma_addr_t addr =CMD_ADDR (&cmd); -bool leaf =CMD_LEAF (&cmd); - -trace_smmuv3_cmdq_tlbi_nh_va (vmid, asid, addr, leaf); -smmuv3_inv_notifiers_iova (bs, asid, addr); -smmu_iotlb_inv_iova (bs, asid, addr); +smmuv3_s1_range_inval (bs, &cmd);break ; - }case SMMU_CMD_TLBI_EL3_ALL :case SMMU_CMD_TLBI_EL3_VA :case SMMU_CMD_TLBI_EL2_ALL :
hw/arm/trace-events +1 -2
@@ -45,8 +45,7 @@ smmuv3_cmdq_cfgi_ste_range(int start, int end) "start=0x%d - end=0x%d" smmuv3_cmdq_cfgi_cd(uint32_t sid) "streamid = %d" smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache HIT for sid %d (hits=%d, misses=%d, hit rate=%d)" smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache MISS for sid %d (hits=%d, misses=%d, hit rate=%d)" -smmuv3_cmdq_tlbi_nh_va(int vmid, int asid, uint64_t addr, bool leaf) "vmid =%d asid =%d addr=0x%"PRIx64" leaf=%d" -smmuv3_cmdq_tlbi_nh_vaa(int vmid, uint64_t addr) "vmid =%d addr=0x%"PRIx64 +smmuv3_s1_range_inval(int vmid, int asid, uint64_t addr, bool leaf) "vmid =%d asid =%d addr=0x%"PRIx64" leaf=%d" smmuv3_cmdq_tlbi_nh(void) "" smmuv3_cmdq_tlbi_nh_asid(uint16_t asid) "asid=%d" smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid %d"
Signature · 1 line
2.20.1
Re: [PATCH RESEND 7/9] hw/arm/smmuv3: Introduce smmuv3_s1_range_inval() helper
Quoted 20 lines · Eric Auger
Let's introduce an helper for S1 IOVA range invalidation. This will be used for
NH_VAandNH_VAAcommands. It decodes the same fields, trace, calls the UNMAP notifiers and invalidate the corresponding IOTLB entries.At the moment, we do not support 3.2 range invalidation yet. So it reduces to a single IOVA invalidation.
Signed-off-by Eric Auger <[email protected]>--- hw/arm/smmuv3.c | 36 +++++++++++++++++------------------- hw/arm/trace-events | 3 +-- 2 files changed, 18 insertions(+), 21 deletions(-)hw/arm/smmuv3.c +0
-0
hw/arm/smmuv3.c static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova) · line 848
Quoted 9 lines
hw/arm/smmuv3.c +6
-0@@ -840,6 +840,22 @@ static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova) } } +static void smmuv3_s1_range_inval (SMMUState *s,Cmd *cmd) +{ +dma_addr_t addr =CMD_ADDR (cmd); +uint8_t type =CMD_TYPE (cmd); +uint16_t vmid =CMD_VMID (cmd); +bool leaf =CMD_LEAF (cmd);
We used to only read the leaf bit for CMD_TLBI_NH_VA, but now we read it also for CMD_TLBI_NH_VAA. That's OK because:
* the leaf bit really does exist on both commands
* the only thing we do with it is print it in the trace message, so this isn't fixing a bug
But you could mention it in the commit message.
Otherwise
thanks
-- PMM
PATCH RESEND 8/9 hw/arm/smmuv3: Get prepared for range invalidation
In reply to Eric AugerEnhance the smmu_iotlb_inv_iova() helper with range invalidation. This uses the new fields passed in the NH_VA and NH_VAA commands:
the size of the range, the level and the granule.
As NH_VA and NH_VAA both use those fields, their decoding and handling is factorized in a new smmuv3_s1_range_inval() helper.
--- hw/arm/smmuv3-internal.h | 4 +++ include/hw/arm/smmu-common.h | 3 +- hw/arm/smmu-common.c | 28 ++++++++++++---- hw/arm/smmuv3.c | 64 +++++++++++++++++++++++------------- hw/arm/trace-events | 4 +-- 5 files changed, 71 insertions(+), 32 deletions(-)
hw/arm/smmuv3-internal.h +4 -0
@@ -298,6 +298,8 @@ enum { /* Command completion notification */ };#define CMD_TYPE (x) extract32((x)->word[0], 0 , 8) +#define CMD_NUM (x) extract32((x)->word[0], 12 , 5) +#define CMD_SCALE (x) extract32((x)->word[0], 20 , 5)#define CMD_SSEC (x) extract32((x)->word[0], 10, 1)#define CMD_SSV (x) extract32((x)->word[0], 11, 1)#define CMD_RESUME_AC (x) extract32((x)->word[0], 12, 1)
@@ -310,6 +312,8 @@ enum { /* Command completion notification */#define CMD_RESUME_STAG (x) extract32((x)->word[2], 0 , 16)#define CMD_RESP (x) extract32((x)->word[2], 11, 2)#define CMD_LEAF (x) extract32((x)->word[2], 0 , 1) +#define CMD_TTL (x) extract32((x)->word[2], 8 , 2) +#define CMD_TG (x) extract32((x)->word[2], 10, 2)#define CMD_STE_RANGE (x) extract32((x)->word[2], 0 , 5)#define CMD_ADDR (x) ({ \ uint64_t high = (uint64_t)(x)->word[3]; \
include/hw/arm/smmu-common.h +2 -1
@@ -164,7 +164,8 @@ uint64_t smmu_get_iotlb_key(uint16_t asid, uint64_t iova, uint8_t tg, uint8_t level);void smmu_iotlb_inv_all (SMMUState *s);void smmu_iotlb_inv_asid (SMMUState *s,uint16_t asid); -void smmu_iotlb_inv_iova (SMMUState *s,int asid,dma_addr_t iova); +void smmu_iotlb_inv_iova (SMMUState *s,int asid,dma_addr_t iova, +uint8_t tg,uint64_t num_pages,uint8_t ttl);/* Unmap the range of all the notifiers registered to any IOMMU mr */ void smmu_inv_notifiers_all (SMMUState *s);
hw/arm/smmu-common.c +22 -6
@@ -125,18 +125,34 @@ static gboolean smmu_hash_remove_by_asid_iova(gpointer key, gpointer value,if (info->asid >=0 ) {return (info->asid ==SMMU_IOTLB_ASID (*iotlb_key)) && - ((info->iova & ~entry->addr_mask) == entry->iova); + (((entry->iova & ~info->mask) == info->iova) || + ((info->iova & ~entry->addr_mask) == entry->iova)); }else { -return (info->iova & ~entry->addr_mask) == entry->iova; +return (((entry->iova & ~info->mask) == info->iova) || + ((info->iova & ~entry->addr_mask) == entry->iova)); } } -inline void smmu_iotlb_inv_iova (SMMUState *s,int asid,dma_addr_t iova) +inline void +smmu_iotlb_inv_iova (SMMUState *s,int asid,dma_addr_t iova, +uint8_t tg,uint64_t num_pages,uint8_t ttl) { -SMMUIOTLBPageInvInfo info = {.asid = asid, .iova = iova}; +if (ttl && (num_pages ==1 )) { +uint64_t key =smmu_get_iotlb_key (asid, iova, tg, ttl); -trace_smmu_iotlb_inv_iova (asid, iova); -g_hash_table_foreach_remove (s->iotlb, smmu_hash_remove_by_asid_iova, &info); +g_hash_table_remove (s->iotlb, &key); + }else { +/* if tg is not set we use 4KB range invalidation */ +uint8_t granule = tg ? tg *2 +10 :12 ; + +SMMUIOTLBPageInvInfo info = { + .asid = asid, .iova = iova, + .mask = (num_pages *1 << granule) -1 }; + +g_hash_table_foreach_remove (s->iotlb, + smmu_hash_remove_by_asid_iova, + &info); + } }inline void smmu_iotlb_inv_asid (SMMUState *s,uint16_t asid)
hw/arm/smmuv3.c +41 -23
111 lines not on this page · raw message
hw/arm/trace-events +2 -2
@@ -45,11 +45,11 @@ smmuv3_cmdq_cfgi_ste_range(int start, int end) "start=0x%d - end=0x%d" smmuv3_cmdq_cfgi_cd(uint32_t sid) "streamid = %d" smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache HIT for sid %d (hits=%d, misses=%d, hit rate=%d)" smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, uint32_t perc) "Config cache MISS for sid %d (hits=%d, misses=%d, hit rate=%d)" -smmuv3_s1_range_inval(int vmid, int asid, uint64_t addr, bool leaf) "vmid =%d asid =%d addr=0x%"PRIx64" leaf=%d" +smmuv3_s1_range_inval(int vmid, int asid, uint64_t addr, uint8_t tg, uint64_t num_pages, uint8_t ttl, bool leaf) "vmid =%d asid =%d addr=0x%"PRIx64" tg=%d num_pages=0x%"PRIx64" ttl=%d leaf=%d" smmuv3_cmdq_tlbi_nh(void) "" smmuv3_cmdq_tlbi_nh_asid(uint16_t asid) "asid=%d" smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid %d" smmuv3_notify_flag_add(const char *iommu) "ADD SMMUNotifier node for iommu mr=%s" smmuv3_notify_flag_del(const char *iommu) "DEL SMMUNotifier node for iommu mr=%s" -smmuv3_inv_notifiers_iova(const char *name, uint16_t asid, uint64_t iova) "iommu mr=%s asid=%d iova=0x%"PRIx64 +smmuv3_inv_notifiers_iova(const char *name, uint16_t asid, uint64_t iova, uint8_t tg, uint64_t num_pages) "iommu mr=%s asid=%d iova=0x%"PRIx64" tg=%d num_pages=0x%"PRIx64
Signature · 1 line
2.20.1
Re: [PATCH RESEND 8/9] hw/arm/smmuv3: Get prepared for range invalidation
Quoted 9 lines · Eric Auger
Enhance the
smmu_iotlb_inv_iova()helper with range invalidation. This uses the new fields passed in theNH_VAandNH_VAAcommands:
the size of the range, the level and the granule.As
NH_VAandNH_VAAboth use those fields, their decoding and handling is factorized in a newsmmuv3_s1_range_inval()helper.Signed-off-by Eric Auger <[email protected]>
Haven't looked too closely at this patch, but see my remarks on patch 6 about keys and lookups.
thanks
-- PMM
PATCH RESEND 9/9 hw/arm/smmuv3: Advertise SMMUv3.2 range invalidation
In reply to Eric AugerExpose the RIL bit so that the guest driver uses range invalidation.
--- hw/arm/smmuv3-internal.h | 1 + hw/arm/smmuv3.c | 2 ++ 2 files changed, 3 insertions(+)
hw/arm/smmuv3-internal.h +1 -0
@@ -54,6 +54,7 @@ REG32(IDR1, 0x4) REG32(IDR2, 0x8) REG32(IDR3, 0xc) +FIELD (IDR3 ,RIL ,10 ,1 ); REG32(IDR4, 0x10) REG32(IDR5, 0x14)FIELD (IDR5 ,OAS ,0 ,3 );
hw/arm/smmuv3.c +2 -0
@@ -254,6 +254,8 @@ static void smmuv3_init_regs(SMMUv3State *s) s->idr[1 ] =FIELD_DP32 (s->idr[1 ],IDR1 ,EVENTQS ,SMMU_EVENTQS ); s->idr[1 ] =FIELD_DP32 (s->idr[1 ],IDR1 ,CMDQS ,SMMU_CMDQS ); + s->idr[3 ] =FIELD_DP32 (s->idr[3 ],IDR3 ,RIL ,1 ); +/* 4K and 64K granule support */ s->idr[5 ] =FIELD_DP32 (s->idr[5 ],IDR5 ,GRAN4K ,1 ); s->idr[5 ] =FIELD_DP32 (s->idr[5 ],IDR5 ,GRAN64K ,1 );
Signature · 1 line
2.20.1
Re: [PATCH RESEND 9/9] hw/arm/smmuv3: Advertise SMMUv3.2 range invalidation
Quoted 9 lines · Eric Auger
Expose the RIL bit so that the guest driver uses range invalidation.
Signed-off-by Eric Auger <[email protected]>--- hw/arm/smmuv3-internal.h | 1 + hw/arm/smmuv3.c | 2 ++ 2 files changed, 3 insertions(+)
thanks
-- PMM