ARM SMMUv3 Emulation Support
v12 cover ARM SMMUv3 Emulation Support
This series implements the emulation code for ARM SMMUv3.
SMMUv3 gets instantiated by adding ",iommu=smmuv3" to the virt machine option.
Only stage 1 and AArch64 PTW are supported. [1 - 14] bring the core emulation code. [15, 16] bring optimizations and 17 brings the vhost integration.
Important Note:
I added the last 3 patches for consistency but I would be happy if [1 - 14] could land upstream first, without waiting for the last 3 patch additional review cycles.
Also note that for vhost integration testing I needed a fix on kernel side: [PATCH] vhost: Fix vhost_copy_to_user(). Otherwise vhost-net stalls are observed.
Best Regards
Eric
This series can be found at:
v12: https://github.com/eauger/qemu/tree/v2.12.0-SMMU-v12
Previous version at:
v11: https://github.com/eauger/qemu/tree/v2.12.0-rc2-SMMU-v11
History:
v11 -> v12:
core:
- leak fixes in kvm_arch_fixup_msi_route
- few changes in ptw to match get_phys_addr_lpae more closely
- some fixes in CD consistency checks
last 3 patches:
- remove mutex as BQL does the job
- change IOTLB key
v10 -> v11:
- one fix irq_ctrl_ack read
- addition of last 3 patches
v9 -> v10:
- see individual change logs
v8 -> v9:
- fix mingw compilation (qemu/log.h)
- put gpl v2 license on all files to respect initial license
- change proto of smmu_ptw* to clarify inputs/outputs and prepare for iotlb emulation
- fix hash table lookup
- cleanup access type handling during ptw
- cleanup reset infra (parent_reset)
- replace some inline functions by macros
- fix some CMD fields
- increment cmdq cons only after cmd execution
- replace some remaining error_report by qemu_log_mask
v7 -> v8:
Took into account Peter's comments:
- revisit queue data structures
- use registerfields.h and got rid of reg array
- use dma_memory_read for all descriptor fetches
- got rid of page table walk for an iova range and implemented standard page table walk for single IOVA
- revisit event data structure
- report events in many more situations and pass the event handle all along the decode and ptw phases
- fix gerror/gerron computations
- completely got rid of stage2 decoding
- use a machine option for instantiation
- get rid of VFIO integration
- get rid of VHOST integration (this will be added in a
- abort in case vhost/vfio notifiers get detected second step together with TLB emulation)
- Tested migration
- fixed TTBR index computation (issue reported by Tomasz)
v6 -> v7:
- DPDK testpmd now running on guest with 2 assigned VFs
- Changed the instantiation method: add the following option to the QEMU command line
-device smmu # for virtio/vhost use cases
-device smmu,caching-mode # for vfio use cases (based on [1])
- splitted the series into smaller patches to allow the review
- the VFIO integration based on "ltlbi-on-map" smmuv3 driver is isolated from the rest: last 2 patches, not for upstream. This is shipped for testing/bench until a better solution is found.
- Reworked permission flag checks and event generation
v5 -> v6:
- Rebase on 2.10 and IOMMUMemoryRegion
- add ACPI TLBI_ON_MAP support (VFIO integration also works in ACPI mode)
- fix block replay
- handle implementation defined SMMU_CMD_TLBI_NH_VA_AM cmd (goes along with TLBI_ON_MAP FW quirk)
- replay systematically unmap the whole range first
- smmuv3_map_hook does not unmap anymore and the unmap is done before the replay
- add and use smmuv3_context_device_invalidate instead of blindly replaying everything
v4 -> v5:
- initial_level now part of SMMUTransCfg
- smmu_page_walk_64 takes into account the max input size
- implement sys->iommu_ops.replay and sys->iommu_ops.notify_flag_changed
- smmuv3_translate: bug fix: don't walk on bypass
- smmu_update_qreg: fix PROD index update
- I did not yet address Peter's comments as the code is not mature enough to be split into sub patches.
v3 -> v4 [Eric]:
- page table walk rewritten to allow scan of the page table within a range of IOVA. This prepares for VFIO integration and replay.
- configuration parsing partially reworked.
- do not advertise unsupported/untested features: S2, S1 + S2, HYP, PRI, ATS, ..
- added ACPI table generation
- migrated to dynamic traces
- mingw compilation fix
v2 -> v3 [Eric]:
- rebased on 2.9
- mostly code and patch reorganization to ease the review process
- optional patches removed. They may be handled separately. I am currently working on ACPI enablement.
- optional instantiation of the smmu in mach-virt
- removed [2/9] (fdt functions) since not mandated
- start splitting main patch into base and derived object
- no new function feature added
v1 -> v2 [Prem]:
- Adopted review comments from Eric Auger
- Make SMMU_DPRINTF to internally call qemu_log (since translation requests are too many, we need control on the type of log we want)
- SMMUTransCfg modified to suite simplicity
- Change RegInfo to uint64 register array
- Code cleanup
- Test cleanups
- Reshuffled patches
v0 -> v1 [Prem]:
- As per SMMUv3 spec 16.0 (only is_ste_consistant() is noticeable)
- Reworked register access/update logic
- Factored out translation code for
- single point bug fix
- sharing/removal in future
- (optional) Unit tests added, with PCI test device
- S1 with 4k/64k, S1+S2 with 4k/64k
- (S1 or S2) only can be verified by Linux 4.7 driver
- (optional) Priliminary ACPI support
v0 [Prem]:
- Implements SMMUv3 spec 11.0
- Supported for PCIe devices,
- Command Queue and Event Queue supported
- LPAE only, S1 is supported and Tested, S2 not tested
- BE mode Translation not supported
- IRQ support (legacy, no MSI)
- Tested with DPDK and e1000
Eric Auger (14):
hw/arm/smmu-common: smmu base device and datatypes hw/arm/smmu-common: IOMMU memory region and address space setup hw/arm/smmu-common: VMSAv8-64 page table walk hw/arm/smmuv3: Wired IRQ and GERROR helpers
hw/arm/smmuv3: Queue helpers
hw/arm/smmuv3: Implement MMIO write operations hw/arm/smmuv3: Event queue recording helper
hw/arm/smmuv3: Implement translate callback
hw/arm/smmuv3: Abort on vfio or vhost case
target/arm/kvm: Translate the MSI doorbell in kvm_arch_fixup_msi_route hw/arm/virt: Introduce the iommu option
hw/arm/smmuv3: Cache/invalidate config data
hw/arm/smmuv3: IOTLB emulation
hw/arm/smmuv3: Add notifications on invalidation
Prem Mallappa (3):
hw/arm/smmuv3: Skeleton
hw/arm/virt: Add SMMUv3 to the virt board hw/arm/virt-acpi-build: Add smmuv3 node in IORT table
default-configs/aarch64-softmmu.mak | 1 + hw/arm/Makefile.objs | 1 + hw/arm/smmu-common.c | 487 ++++++++++++ hw/arm/smmu-internal.h | 99 +++ hw/arm/smmuv3-internal.h | 621 +++++++++++++++ hw/arm/smmuv3.c | 1497 +++++++++++++++++++++++++++++++++++ hw/arm/trace-events | 58 ++ hw/arm/virt-acpi-build.c | 55 +- hw/arm/virt.c | 100 ++- include/hw/acpi/acpi-defs.h | 15 + include/hw/arm/smmu-common.h | 169 ++++ include/hw/arm/smmuv3.h | 87 ++ include/hw/arm/virt.h | 10 + target/arm/kvm.c | 38 +- target/arm/trace-events | 3 + 15 files changed, 3232 insertions(+), 9 deletions(-) create mode 100644 hw/arm/smmu-common.c create mode 100644 hw/arm/smmu-internal.h create mode 100644 hw/arm/smmuv3-internal.h create mode 100644 hw/arm/smmuv3.c create mode 100644 include/hw/arm/smmu-common.h create mode 100644 include/hw/arm/smmuv3.h
Signature · 1 line
2.5.5
PATCH v12 1/17 hw/arm/smmu-common: smmu base device and datatypes
The patch introduces the smmu base device and class for the ARM smmu. Devices for specific versions will be derived from this base device.
We also introduce some important datatypes.
---
v10 -> v11:
- remove hash tables for configs and iotlb which will be introduced later on
v9 -> v10:
- invoke the parent_realize
- use device_class_set_parent_realize
- use "" to include headers
- fix indent in SMMUTransCfg
- rename smmu_as_by_busptr into smmu_pcibus_by_busptr and smmu_as_by_bus_num into smmu_pcibus_by_bus_num
- mrtypename becomes const
- add Peter's R-b
v8 -> v9:
- remove page walk callback type from this patch (vhost related)
- add a new hash table for caching configuration data
- add reset function
- add asid
v7 -> v8:
- add bus_num property
- add primary-bus property
- add realize and remove instance_init
- rename TYPE and related macros to match naming convention using for GIC
- add SMMUPageTableWalkEventInfo
- tt[2] in translation config
v3 -> v4:
- added smmu_find_as_from_bus_num
- SMMU_PCI_BUS_MAX and SMMU_PCI_DEVFN_MAX in smmu-common header
- new fields in SMMUState:
- iommu_ops, smmu_as_by_busptr, smmu_as_by_bus_num
- add aa64[] field in SMMUTransCfg
v3:
- moved the base code in a separate patch to ease the review.
- clearer separation between base class and smmuv3 class
- translate_* only implemented as class methods
Conflicts:
default-configs/aarch64-softmmu.mak
Conflicts:
hw/arm/Makefile.objs
--- default-configs/aarch64-softmmu.mak | 1 + hw/arm/Makefile.objs | 1 + hw/arm/smmu-common.c | 81 ++++++++++++++++++++++++ include/hw/arm/smmu-common.h | 123 ++++++++++++++++++++++++++++++++++++ 4 files changed, 206 insertions(+) create mode 100644 hw/arm/smmu-common.c create mode 100644 include/hw/arm/smmu-common.h
default-configs/aarch64-softmmu.mak +1 -0
@@ -8,3 +8,4 @@ CONFIG_DDC=y CONFIG_DPCD=y CONFIG_XLNX_ZYNQMP=y CONFIG_XLNX_ZYNQMP_ARM=y +CONFIG_ARM_SMMUV3=y
hw/arm/Makefile.objs +1 -0
@@ -35,3 +35,4 @@ obj-$(CONFIG_MPS2) += mps2-tz.o obj-$(CONFIG_MSF2) += msf2-soc.o msf2-som.o obj-$(CONFIG_IOTKIT) += iotkit.o obj-$(CONFIG_FSL_IMX7) += fsl-imx7.o mcimx7d-sabre.o +obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o
hw/arm/smmu-common.c new file +81 -0
81 lines not on this page · raw message
include/hw/arm/smmu-common.h new file +123 -0
123 lines not on this page · raw message
Signature · 1 line
2.5.5
PATCH v12 2/17 hw/arm/smmu-common: IOMMU memory region and address space setup
In reply to Eric AugerWe set up the infrastructure to enumerate all the PCI devices attached to the SMMU and create an associated IOMMU memory region and address space.
Those info are stored in SMMUDevice objects. The devices are grouped according to the PCIBus they belong to. A hash table indexed by the PCIBus pointer is used. Also an array indexed by the bus number allows to find the list of SMMUDevices.
---
v11 -> v12:
- use PCI_BUILD_BDF in smmu_get_sid and add Peter's R-b
v9 -> v10:
- comment functions added to the header
- g_free(name)
- renamed smmu_find_as_from_bus_num into smmu_find_smmu_pcibus
- add a comment about lazy init in smmu_find_smmu_pcibus
- add a trace event when creating the smmu iommu mr
v8 -> v9:
- fix key value for lookup
v7 -> v8:
- introduce SMMU_MAX_VA_BITS
- use PCI bus handle as a key
- do not clear s->smmu_as_by_bus_num
- use g_new0 instead of g_malloc0
- use primary_bus field
--- hw/arm/smmu-common.c | 69 ++++++++++++++++++++++++++++++++++++++++++++ hw/arm/trace-events | 3 ++ include/hw/arm/smmu-common.h | 8 +++++ 3 files changed, 80 insertions(+)
hw/arm/smmu-common.c +69 -0
@@ -28,8 +28,69 @@#include "qemu/error-report.h" #include "hw/arm/smmu-common.h" +/** +* The bus number is used for lookup when SID based invalidation occurs. +* In that case we lazily populate the SMMUPciBus array from the bus hash +* table. At the time the SMMUPciBus is created (smmu_find_add_as), the bus +* numbers may not be always initialized yet. +*/ +SMMUPciBus *smmu_find_smmu_pcibus (SMMUState *s,uint8_t bus_num) +{ +SMMUPciBus *smmu_pci_bus = s->smmu_pcibus_by_bus_num[bus_num]; + +if (!smmu_pci_bus) { +GHashTableIter iter; + +g_hash_table_iter_init (&iter, s->smmu_pcibus_by_busptr); +while (g_hash_table_iter_next (&iter,NULL , (void **)&smmu_pci_bus)) { +if (pci_bus_num (smmu_pci_bus->bus) == bus_num) { + s->smmu_pcibus_by_bus_num[bus_num] = smmu_pci_bus; +return smmu_pci_bus; + } + } + } +return smmu_pci_bus; +} + +static AddressSpace *smmu_find_add_as (PCIBus *bus,void *opaque,int devfn) +{ +SMMUState *s = opaque; +SMMUPciBus *sbus =g_hash_table_lookup (s->smmu_pcibus_by_busptr, bus); +SMMUDevice *sdev; + +if (!sbus) { + sbus =g_malloc0 (sizeof (SMMUPciBus) + +sizeof (SMMUDevice *) *SMMU_PCI_DEVFN_MAX ); + sbus->bus = bus; +g_hash_table_insert (s->smmu_pcibus_by_busptr, bus, sbus); + } + + sdev = sbus->pbdev[devfn]; +if (!sdev) { +char *name =g_strdup_printf ("%s-%d-%d" , + s->mrtypename, +pci_bus_num (bus), devfn); + sdev = sbus->pbdev[devfn] =g_new0 (SMMUDevice,1 ); + + sdev->smmu = s; + sdev->bus = bus; + sdev->devfn = devfn; + +memory_region_init_iommu (&sdev->iommu,sizeof (sdev->iommu), + s->mrtypename, +OBJECT (s), name,1ULL <<SMMU_MAX_VA_BITS ); +address_space_init (&sdev->as, +MEMORY_REGION (&sdev->iommu), name); +trace_smmu_add_mr (name); +g_free (name); + } + +return &sdev->as; +} +static void smmu_base_realize (DeviceState *dev,Error **errp) { +SMMUState *s =ARM_SMMU (dev);SMMUBaseClass *sbc =ARM_SMMU_GET_CLASS (dev);Error *local_err =NULL ;
@@ -38,6 +99,14 @@ static void smmu_base_realize(DeviceState *dev, Error **errp)error_propagate (errp, local_err);return ; } + + s->smmu_pcibus_by_busptr =g_hash_table_new (NULL ,NULL ); + +if (s->primary_bus) { +pci_setup_iommu (s->primary_bus, smmu_find_add_as, s); + }else { +error_setg (errp,"SMMU is not attached to any PCI bus!" ); + } }static void smmu_base_reset (DeviceState *dev)
hw/arm/trace-events +3 -0
@@ -2,3 +2,6 @@ # hw/arm/virt-acpi-build.c virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." + +# hw/arm/smmu-common.c +smmu_add_mr(const char *name) "%s"
include/hw/arm/smmu-common.h +8 -0
@@ -120,4 +120,12 @@ typedef struct {#define ARM_SMMU_GET_CLASS (obj) \ OBJECT_GET_CLASS(SMMUBaseClass, (obj), TYPE_ARM_SMMU) +/* Return the SMMUPciBus handle associated to a PCI bus number */ +SMMUPciBus *smmu_find_smmu_pcibus (SMMUState *s,uint8_t bus_num); + +/* Return the stream ID of an SMMU device */ +static inline uint16_t smmu_get_sid (SMMUDevice *sdev) +{ +return PCI_BUILD_BDF (pci_bus_num (sdev->bus), sdev->devfn); +}#endif /* HW_ARM_SMMU_COMMON */
Signature · 1 line
2.5.5
PATCH v12 3/17 hw/arm/smmu-common: VMSAv8-64 page table walk
In reply to Eric AugerThis patch implements the page table walk for VMSAv8-64.
---
v11 -> v12:
- typo fixes and added Peter's R-b
- closer match with get_phys_addr_lpae()
- take into account the input size when computing the offset
v9 -> v10:
- Add 64b single-copy atomicity comment related to PTE fetch
- remove checks in get_block_pte_address and use formulae to compute block address offset
- remove check_perm
- fix select_tt
- in trace-events use PRIx64 instead of 0x%lx
- commented the fact we cannot discriminate between user and priviledged transactions
v8 -> v9:
- remove guest error log on PTE fetch fault
- rename trace functions
- fix smmu_page_walk_level_res_invalid_pte last arg
- fix PTE_ADDRESS
- turn functions into macros
- make sure to return the actual pte access permission into tlbe->perm
- change proto of smmu_ptw*
v7 -> v8:
- rework get_pte
- use LOG_LEVEL_ERROR
- remove error checking in get_block_pte_address
- page table walk simplified (no VFIO replay anymore)
- handle PTW error events
- use dma_memory_read
v6 -> v7:
- fix wrong error handling in walk_page_table
- check perm in smmu_translate
v5 -> v6:
- use IOMMUMemoryRegion
- remove initial_lookup_level()
- fix block replay
v4 -> v5:
- add initial level in translation config
- implement block pte
- rename must_translate into nofail
- introduce call_entry_hook
- small changes to dynamic traces
- smmu_page_walk code moved from smmuv3.c to this file
- remove smmu_translate*
v3 -> v4:
- reworked page table walk to prepare for VFIO integration (capability to scan a range of IOVA). Same function is used for translate for a single iova. This is largely inspired from intel_iommu.c
- as the translate function was not straightforward to me, I tried to stick more closely to the VMSA spec.
- remove support of nested stage (kernel driver does not support it anyway)
- use error_report and trace events
- add aa64[] field in SMMUTransCfg
--- hw/arm/smmu-common.c | 222 +++++++++++++++++++++++++++++++++++++++++++ hw/arm/smmu-internal.h | 99 +++++++++++++++++++ hw/arm/trace-events | 9 +- include/hw/arm/smmu-common.h | 14 +++ 4 files changed, 343 insertions(+), 1 deletion(-) create mode 100644 hw/arm/smmu-internal.h
hw/arm/smmu-common.c +222 -0
@@ -27,6 +27,228 @@#include "qemu/error-report.h" #include "hw/arm/smmu-common.h" +#include "smmu-internal.h" + +/* VMSAv8-64 Translation */ + +/** +* get_pte - Get the content of a page table entry located at +* @base_addr[@index] +*/ +static int get_pte (dma_addr_t baseaddr,uint32_t index,uint64_t *pte, +SMMUPTWEventInfo *info) +{ +int ret; +dma_addr_t addr = baseaddr + index *sizeof (*pte); + +/* TODO: guarantee 64-bit single-copy atomicity */ + ret =dma_memory_read (&address_space_memory, addr, + (uint8_t *)pte,sizeof (*pte)); + +if (ret !=MEMTX_OK ) { + info->type =SMMU_PTW_ERR_WALK_EABT ; + info->addr = addr; +return -EINVAL ; + } +trace_smmu_get_pte (baseaddr, index, addr, *pte); +return 0 ; +} + +/* VMSAv8-64 Translation Table Format Descriptor Decoding */ + +/** +* get_page_pte_address - returns the L3 descriptor output address, +* ie. the page frame +* ARM ARM spec: Figure D4-17 VMSAv8-64 level 3 descriptor format +*/ +static inline hwaddr get_page_pte_address (uint64_t pte,int granule_sz) +{ +return PTE_ADDRESS (pte, granule_sz); +} + +/** +* get_table_pte_address - return table descriptor output address, +* ie. address of next level table +* ARM ARM Figure D4-16 VMSAv8-64 level0, level1, and level 2 descriptor formats +*/ +static inline hwaddr get_table_pte_address (uint64_t pte,int granule_sz) +{ +return PTE_ADDRESS (pte, granule_sz); +} + +/** +* get_block_pte_address - return block descriptor output address and block size +* ARM ARM Figure D4-16 VMSAv8-64 level0, level1, and level 2 descriptor formats +*/ +static inline hwaddr get_block_pte_address (uint64_t pte,int level, +int granule_sz,uint64_t *bsz) +{ +int n = (granule_sz -3 ) * (4 - level) +3 ; + + *bsz =1 << n; +return PTE_ADDRESS (pte, n); +} + +SMMUTransTableInfo *select_tt (SMMUTransCfg *cfg,dma_addr_t iova) +{ +bool tbi =extract64 (iova,55 ,1 ) ?TBI1 (cfg->tbi) :TBI0 (cfg->tbi); +uint8_t tbi_byte = tbi *8 ; + +if (cfg->tt[0 ].tsz && + !extract64 (iova,64 - cfg->tt[0 ].tsz, cfg->tt[0 ].tsz - tbi_byte)) { +/* there is a ttbr0 region and we are in it (high bits all zero) */ +return &cfg->tt[0 ]; + }else if (cfg->tt[1 ].tsz && + !extract64 (iova,64 - cfg->tt[1 ].tsz, cfg->tt[1 ].tsz - tbi_byte)) { +/* there is a ttbr1 region and we are in it (high bits all one) */ +return &cfg->tt[1 ]; + }else if (!cfg->tt[0 ].tsz) { +/* ttbr0 region is "everything not in the ttbr1 region" */ +return &cfg->tt[0 ]; + }else if (!cfg->tt[1 ].tsz) { +/* ttbr1 region is "everything not in the ttbr0 region" */ +return &cfg->tt[1 ]; + } +/* in the gap between the two regions, this is a Translation fault */ +return NULL ; +} + +/** +* smmu_ptw_64 - VMSAv8-64 Walk of the page tables for a given IOVA +* @cfg: translation config +* @iova: iova to translate +* @perm: access type +* @tlbe: IOMMUTLBEntry (out) +* @info: handle to an error info +* +* Return 0 on success, < 0 on error. In case of error, @info is filled +* and tlbe->perm is set to IOMMU_NONE. +* Upon success, @tlbe is filled with translated_addr and entry +* permission rights. +*/ +static int smmu_ptw_64 (SMMUTransCfg *cfg, +dma_addr_t iova,IOMMUAccessFlags perm, +IOMMUTLBEntry *tlbe,SMMUPTWEventInfo *info) +{ +dma_addr_t baseaddr, indexmask; +int stage = cfg->stage; +SMMUTransTableInfo *tt =select_tt (cfg, iova); +uint8_t level, granule_sz, inputsize, stride; + +if (!tt || tt->disabled) { + info->type =SMMU_PTW_ERR_TRANSLATION ; + goto error; + } + + granule_sz = tt->granule_sz; + stride = granule_sz -3 ; + inputsize =64 - tt->tsz; + level =4 - (inputsize -4 ) / stride; + indexmask = (1ULL << (inputsize - (stride * (4 - level)))) -1 ; + baseaddr =extract64 (tt->ttb,0 ,48 ); + baseaddr &= ~indexmask; + + tlbe->iova = iova; + tlbe->addr_mask = (1 << granule_sz) -1 ; + +while (level <=3 ) { +uint64_t subpage_size =1ULL <<level_shift (level, granule_sz); +uint64_t mask = subpage_size -1 ; +uint32_t offset =iova_level_offset (iova, inputsize, level, granule_sz); +uint64_t pte; +dma_addr_t pte_addr = baseaddr + offset *sizeof (pte); +uint8_t ap; + +if (get_pte (baseaddr, offset, &pte, info)) { + goto error; + } +trace_smmu_ptw_level (level, iova, subpage_size, + baseaddr, offset, pte); + +if (is_invalid_pte (pte) ||is_reserved_pte (pte, level)) { +trace_smmu_ptw_invalid_pte (stage, level, baseaddr, + pte_addr, offset, pte); + info->type =SMMU_PTW_ERR_TRANSLATION ; + goto error; + } + +if (is_page_pte (pte, level)) { +uint64_t gpa =get_page_pte_address (pte, granule_sz); + + ap =PTE_AP (pte); +if (is_permission_fault (ap, perm)) { + info->type =SMMU_PTW_ERR_PERMISSION ; + goto error; + } + + tlbe->translated_addr = gpa + (iova & mask); + tlbe->perm =PTE_AP_TO_PERM (ap); +trace_smmu_ptw_page_pte (stage, level, iova, + baseaddr, pte_addr, pte, gpa); +return 0 ; + } +if (is_block_pte (pte, level)) { +uint64_t block_size; +hwaddr gpa =get_block_pte_address (pte, level, granule_sz, + &block_size); + + ap =PTE_AP (pte); +if (is_permission_fault (ap, perm)) { + info->type =SMMU_PTW_ERR_PERMISSION ; + goto error; + } + +trace_smmu_ptw_block_pte (stage, level, baseaddr, + pte_addr, pte, iova, gpa, + block_size >>20 ); + + tlbe->translated_addr = gpa + (iova & mask); + tlbe->perm =PTE_AP_TO_PERM (ap); +return 0 ; + } + +/* table pte */ + ap =PTE_APTABLE (pte); + +if (is_permission_fault (ap, perm)) { + info->type =SMMU_PTW_ERR_PERMISSION ; + goto error; + } + baseaddr =get_table_pte_address (pte, granule_sz); + level++; + } + + info->type =SMMU_PTW_ERR_TRANSLATION ; + +error: + tlbe->perm =IOMMU_NONE ; +return -EINVAL ; +} + +/** +* smmu_ptw - Walk the page tables for an IOVA, according to @cfg +* +* @cfg: translation configuration +* @iova: iova to translate +* @perm: tentative access type +* @tlbe: returned entry +* @info: ptw event handle +* +* return 0 on success +*/ +inline int smmu_ptw (SMMUTransCfg *cfg,dma_addr_t iova,IOMMUAccessFlags perm, +IOMMUTLBEntry *tlbe,SMMUPTWEventInfo *info) +{ +if (!cfg->aa64) { +/* +* This code path is not entered as we check this while decoding +* the configuration data in the derived SMMU model. +*/ +g_assert_not_reached (); + } + +return smmu_ptw_64 (cfg, iova, perm, tlbe, info); +}/** * The bus number is used for lookup when SID based invalidation occurs.
hw/arm/smmu-internal.h new file +99 -0
99 lines not on this page · raw message
hw/arm/trace-events +8 -1
@@ -4,4 +4,11 @@ virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." # hw/arm/smmu-common.c -smmu_add_mr(const char *name) "%s" +smmu_add_mr(const char *name) "%s" +smmu_page_walk(int stage, uint64_t baseaddr, int first_level, uint64_t start, uint64_t end) "stage=%d, baseaddr=0x%"PRIx64", first level=%d, start=0x%"PRIx64", end=0x%"PRIx64 +smmu_lookup_table(int level, uint64_t baseaddr, int granule_sz, uint64_t start, uint64_t end, int flags, uint64_t subpage_size) "level=%d baseaddr=0x%"PRIx64" granule=%d, start=0x%"PRIx64" end=0x%"PRIx64" flags=%d subpage_size=0x%"PRIx64 +smmu_ptw_level(int level, uint64_t iova, size_t subpage_size, uint64_t baseaddr, uint32_t offset, uint64_t pte) "level=%d iova=0x%"PRIx64" subpage_sz=0x%lx baseaddr=0x%"PRIx64" offset=%d => pte=0x%"PRIx64 +smmu_ptw_invalid_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint32_t offset, uint64_t pte) "stage=%d level=%d base@=0x%"PRIx64" pte@=0x%"PRIx64" offset=%d pte=0x%"PRIx64 +smmu_ptw_page_pte(int stage, int level, uint64_t iova, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) "stage=%d level=%d iova=0x%"PRIx64" base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" page address = 0x%"PRIx64 +smmu_ptw_block_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t iova, uint64_t gpa, int bsize_mb) "stage=%d level=%d base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" iova=0x%"PRIx64" block address = 0x%"PRIx64" block size = %d MiB" +smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) "baseaddr=0x%"PRIx64" index=0x%x, pteaddr=0x%"PRIx64", pte=0x%"PRIx64
include/hw/arm/smmu-common.h +14 -0
@@ -128,4 +128,18 @@ static inline uint16_t smmu_get_sid(SMMUDevice *sdev) {return PCI_BUILD_BDF (pci_bus_num (sdev->bus), sdev->devfn); } + +/** +* smmu_ptw - Perform the page table walk for a given iova / access flags +* pair, according to @cfg translation config +*/ +int smmu_ptw (SMMUTransCfg *cfg,dma_addr_t iova,IOMMUAccessFlags perm, +IOMMUTLBEntry *tlbe,SMMUPTWEventInfo *info); + +/** +* select_tt - compute which translation table shall be used according to +* the input iova and translation config and return the TT specific info +*/ +SMMUTransTableInfo *select_tt (SMMUTransCfg *cfg,dma_addr_t iova); +#endif /* HW_ARM_SMMU_COMMON */
Signature · 1 line
2.5.5
PATCH v12 4/17 hw/arm/smmuv3: Skeleton
In reply to Eric AugerFrom: Prem Mallappa <[email protected]>
This patch implements a skeleton for the smmuv3 device. Datatypes and register definitions are introduced. The MMIO region, the interrupts and the queue are initialized.
Only the MMIO read operation is implemented here.
---
v11 -> v12:
- remove smmu_read64. Added Peter's R-b.
v10 -> v11:
- remove irq_ctrl_ack and return irq_ctrl on A_IRQ_CTRL_ACK read
v9 -> v10:
- s/hwaddr/uint64_t in trace-events
- add comments
- s->mrtypename = TYPE_SMMUV3_IOMMU_MEMORY_REGION
- removed iidr and idr from VMState
- use VMSTATE_STRUCT for the queues
- use qemu_log_mask(LOG_UNIMP,*) for unimplemented regs
- added SMMU_CMDQS, SMMU_EVENTQS
- use ops with attributes
- split readl/readll
- put id_regs in an array
- removed smmu_read64
- removed SMMU_FEATURE_2LVL_STE and NB_REGS
- RAZ when read access at unexpected address
v8 -> v9:
- add #include "qemu/log.h"
- add parent_reset
v7 -> v8:
- remove __smmu_data structs
- revisit struct SMMUQueue
- do not advertise stage 2 support anymore
- use the register definition API and get rid of REG array
- get read of queue structs
v6 -> v7:
- split into several patches
v5 -> v6:
- Use IOMMUMemoryregion
- regs become uint32_t and fix 64b MMIO access (.impl)
- trace_smmuv3_write/read_mmio take the size param
v4 -> v5:
- change smmuv3_translate proto (IOMMUAccessFlags flag)
- has_stagex replaced by is_ste_stagex
- smmu_cfg_populate removed
- added smmuv3_decode_config and reworked error management
- remwork the naming of IOMMU mrs
- fix SMMU_CMDQ_CONS offset
v3 -> v4
- smmu_irq_update
- fix hash key allocation
- set smmu_iommu_ops
- set SMMU_REG_CR0,
- smmuv3_translate: ret.perm not set in bypass mode
- use trace events
- renamed STM2U64 into L1STD_L2PTR and STMSPAN into L1STD_SPAN
- rework smmu_find_ste
- fix tg2granule in TT0/0b10 corresponds to 16kB
v2 -> v3:
- move creation of include/hw/arm/smmuv3.h to this patch to fix compil issue
- compilation allowed
- fix sbus allocation in smmu_init_pci_iommu
- restructure code into headers
- misc cleanups
Conflicts:
hw/arm/Makefile.objs
--- hw/arm/Makefile.objs | 2 +- hw/arm/smmuv3-internal.h | 142 ++++++++++++++++++ hw/arm/smmuv3.c | 366 +++++++++++++++++++++++++++++++++++++++++++++++ hw/arm/trace-events | 3 + include/hw/arm/smmuv3.h | 87 +++++++++++ 5 files changed, 599 insertions(+), 1 deletion(-) create mode 100644 hw/arm/smmuv3-internal.h create mode 100644 hw/arm/smmuv3.c create mode 100644 include/hw/arm/smmuv3.h
hw/arm/Makefile.objs +1 -1
@@ -35,4 +35,4 @@ obj-$(CONFIG_MPS2) += mps2-tz.o obj-$(CONFIG_MSF2) += msf2-soc.o msf2-som.o obj-$(CONFIG_IOTKIT) += iotkit.o obj-$(CONFIG_FSL_IMX7) += fsl-imx7.o mcimx7d-sabre.o -obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o +obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o smmuv3.o
hw/arm/smmuv3-internal.h new file +142 -0
142 lines not on this page · raw message
hw/arm/smmuv3.c new file +366 -0
366 lines not on this page · raw message
hw/arm/trace-events +3 -0
@@ -12,3 +12,6 @@ smmu_ptw_invalid_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, smmu_ptw_page_pte(int stage, int level, uint64_t iova, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) "stage=%d level=%d iova=0x%"PRIx64" base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" page address = 0x%"PRIx64 smmu_ptw_block_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t iova, uint64_t gpa, int bsize_mb) "stage=%d level=%d base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" iova=0x%"PRIx64" block address = 0x%"PRIx64" block size = %d MiB" smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) "baseaddr=0x%"PRIx64" index=0x%x, pteaddr=0x%"PRIx64", pte=0x%"PRIx64 + +#hw/arm/smmuv3.c +smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)"
include/hw/arm/smmuv3.h new file +87 -0
87 lines not on this page · raw message
Signature · 1 line
2.5.5
PATCH v12 5/17 hw/arm/smmuv3: Wired IRQ and GERROR helpers
In reply to Eric AugerWe introduce some helpers to handle wired IRQs and especially GERROR interrupt. SMMU writes GERROR register on GERROR event and SW acks GERROR interrupts by setting GERRORn.
The Wired interrupts are edge sensitive hence the pulse usage.
---
v11 -> v12:
- added Peter's R-b
v9 - v10:
- s/hwaddr/uint64_t in trace-events
- use qemu_log_mask LOG_UNIMP on PRI IRQ
- add a comment saying smmuv3_trigger_irq and smmuv3_write_gerrorn will become static later on
- write gerrorn without filtering (ie. even if the guest toggles non active IRQs)
- pulse if at least one new IRQ type
- smmuv3_eventq_irq_enabled and smmuv3_gerror_irq_enabled become static inline
v7 -> v8:
- remove SMMU_PENDING_GERRORS macro
- properly toggle gerror
- properly sanitize gerrorn write
--- hw/arm/smmuv3-internal.h | 14 +++++++++++ hw/arm/smmuv3.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++++ hw/arm/trace-events | 3 +++ 3 files changed, 81 insertions(+)
hw/arm/smmuv3-internal.h +14 -0
@@ -139,4 +139,18 @@ static inline uint32_t smmuv3_idreg(int regoffset) return smmuv3_ids[regoffset / 4]; } +static inline bool smmuv3_eventq_irq_enabled (SMMUv3State *s) +{ +return FIELD_EX32 (s->irq_ctrl,IRQ_CTRL ,EVENTQ_IRQEN ); +} + +static inline bool smmuv3_gerror_irq_enabled (SMMUv3State *s) +{ +return FIELD_EX32 (s->irq_ctrl,IRQ_CTRL ,GERROR_IRQEN ); +} + +/* public until callers get introduced */ +void smmuv3_trigger_irq (SMMUv3State *s,SMMUIrq irq,uint32_t gerror_mask); +void smmuv3_write_gerrorn (SMMUv3State *s,uint32_t gerrorn); +#endif
hw/arm/smmuv3.c +64 -0
70 lines not on this page · raw message
hw/arm/trace-events +3 -0
@@ -15,3 +15,6 @@ smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) "base #hw/arm/smmuv3.c smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)" +smmuv3_trigger_irq(int irq) "irq=%d" +smmuv3_write_gerror(uint32_t toggled, uint32_t gerror) "toggled=0x%x, new GERROR=0x%x" +smmuv3_write_gerrorn(uint32_t acked, uint32_t gerrorn) "acked=0x%x, new GERRORN=0x%x"
Signature · 1 line
2.5.5
PATCH v12 6/17 hw/arm/smmuv3: Queue helpers
In reply to Eric AugerWe introduce helpers to read/write into the command and event circular queues.
smmuv3_write_eventq and smmuv3_cmq_consume will become static in subsequent patches.
Invalidation commands are not yet dealt with. We do not cache data that need to be invalidated. This will change with vhost integration.
---
v11 -> v12:
- added a comment related to queue_cons_incr deposit usage
- check type value in smmu_cmd_string
- added Peter's R-b
v9 -> v10:
- simplified macros
- s/BASE/Q_BASE
- use log2size field
- static inline functions replacing some macros
- simplified queue_prod_incr/queue_cons_incr and use deposit32
- trace for cmdq_consume failure
v8 -> v9:
- fix CMD_SSID & CMD_ADDR + some renamings
- do cons increment after the execution of the command
- add Q_INCONSISTENT()
v7 -> v8
- use address_space_rw
- helpers inspired from spec
--- hw/arm/smmuv3-internal.h | 163 +++++++++++++++++++++++++++++++++++++++++++++++ hw/arm/smmuv3.c | 136 +++++++++++++++++++++++++++++++++++++++ hw/arm/trace-events | 5 ++ 3 files changed, 304 insertions(+)
hw/arm/smmuv3-internal.h +163 -0
@@ -153,4 +153,167 @@ static inline bool smmuv3_gerror_irq_enabled(SMMUv3State *s)void smmuv3_trigger_irq (SMMUv3State *s,SMMUIrq irq,uint32_t gerror_mask);void smmuv3_write_gerrorn (SMMUv3State *s,uint32_t gerrorn); +/* Queue Handling */ + +#define Q_BASE (q) ((q)->base & SMMU_BASE_ADDR_MASK) +#define WRAP_MASK (q) (1 << (q)->log2size) +#define INDEX_MASK (q) (((1 << (q)->log2size)) - 1) +#define WRAP_INDEX_MASK (q) ((1 << ((q)->log2size + 1)) - 1) + +#define Q_CONS (q) ((q)->cons & INDEX_MASK(q)) +#define Q_PROD (q) ((q)->prod & INDEX_MASK(q)) + +#define Q_CONS_ENTRY (q) (Q_BASE(q) + (q)->entry_size * Q_CONS(q)) +#define Q_PROD_ENTRY (q) (Q_BASE(q) + (q)->entry_size * Q_PROD(q)) + +#define Q_CONS_WRAP (q) (((q)->cons & WRAP_MASK(q)) >> (q)->log2size) +#define Q_PROD_WRAP (q) (((q)->prod & WRAP_MASK(q)) >> (q)->log2size) + +static inline bool smmuv3_q_full (SMMUQueue *q) +{ +return ((q->cons ^ q->prod) &WRAP_INDEX_MASK (q)) ==WRAP_MASK (q); +} + +static inline bool smmuv3_q_empty (SMMUQueue *q) +{ +return (q->cons &WRAP_INDEX_MASK (q)) == (q->prod &WRAP_INDEX_MASK (q)); +} + +static inline void queue_prod_incr (SMMUQueue *q) +{ + q->prod = (q->prod +1 ) &WRAP_INDEX_MASK (q); +} + +static inline void queue_cons_incr (SMMUQueue *q) +{ +/* +* We have to use deposit for the CONS registers to preserve +* the ERR field in the high bits. +*/ + q->cons =deposit32 (q->cons,0 , q->log2size +1 , q->cons +1 ); +} + +static inline bool smmuv3_cmdq_enabled (SMMUv3State *s) +{ +return FIELD_EX32 (s->cr[0 ],CR0 ,CMDQEN ); +} + +static inline bool smmuv3_eventq_enabled (SMMUv3State *s) +{ +return FIELD_EX32 (s->cr[0 ],CR0 ,EVENTQEN ); +} + +static inline void smmu_write_cmdq_err (SMMUv3State *s,uint32_t err_type) +{ + s->cmdq.cons =FIELD_DP32 (s->cmdq.cons,CMDQ_CONS ,ERR , err_type); +} + +void smmuv3_write_eventq (SMMUv3State *s,Evt *evt); + +/* Commands */ + +typedef enum SMMUCommandType { +SMMU_CMD_NONE =0x00 , +SMMU_CMD_PREFETCH_CONFIG , +SMMU_CMD_PREFETCH_ADDR , +SMMU_CMD_CFGI_STE , +SMMU_CMD_CFGI_STE_RANGE , +SMMU_CMD_CFGI_CD , +SMMU_CMD_CFGI_CD_ALL , +SMMU_CMD_CFGI_ALL , +SMMU_CMD_TLBI_NH_ALL =0x10 , +SMMU_CMD_TLBI_NH_ASID , +SMMU_CMD_TLBI_NH_VA , +SMMU_CMD_TLBI_NH_VAA , +SMMU_CMD_TLBI_EL3_ALL =0x18 , +SMMU_CMD_TLBI_EL3_VA =0x1a , +SMMU_CMD_TLBI_EL2_ALL =0x20 , +SMMU_CMD_TLBI_EL2_ASID , +SMMU_CMD_TLBI_EL2_VA , +SMMU_CMD_TLBI_EL2_VAA , +SMMU_CMD_TLBI_S12_VMALL =0x28 , +SMMU_CMD_TLBI_S2_IPA =0x2a , +SMMU_CMD_TLBI_NSNH_ALL =0x30 , +SMMU_CMD_ATC_INV =0x40 , +SMMU_CMD_PRI_RESP , +SMMU_CMD_RESUME =0x44 , +SMMU_CMD_STALL_TERM , +SMMU_CMD_SYNC , +}SMMUCommandType ; + +static const char *cmd_stringify[] = { + [SMMU_CMD_PREFETCH_CONFIG ] ="SMMU_CMD_PREFETCH_CONFIG" , + [SMMU_CMD_PREFETCH_ADDR ] ="SMMU_CMD_PREFETCH_ADDR" , + [SMMU_CMD_CFGI_STE ] ="SMMU_CMD_CFGI_STE" , + [SMMU_CMD_CFGI_STE_RANGE ] ="SMMU_CMD_CFGI_STE_RANGE" , + [SMMU_CMD_CFGI_CD ] ="SMMU_CMD_CFGI_CD" , + [SMMU_CMD_CFGI_CD_ALL ] ="SMMU_CMD_CFGI_CD_ALL" , + [SMMU_CMD_CFGI_ALL ] ="SMMU_CMD_CFGI_ALL" , + [SMMU_CMD_TLBI_NH_ALL ] ="SMMU_CMD_TLBI_NH_ALL" , + [SMMU_CMD_TLBI_NH_ASID ] ="SMMU_CMD_TLBI_NH_ASID" , + [SMMU_CMD_TLBI_NH_VA ] ="SMMU_CMD_TLBI_NH_VA" , + [SMMU_CMD_TLBI_NH_VAA ] ="SMMU_CMD_TLBI_NH_VAA" , + [SMMU_CMD_TLBI_EL3_ALL ] ="SMMU_CMD_TLBI_EL3_ALL" , + [SMMU_CMD_TLBI_EL3_VA ] ="SMMU_CMD_TLBI_EL3_VA" , + [SMMU_CMD_TLBI_EL2_ALL ] ="SMMU_CMD_TLBI_EL2_ALL" , + [SMMU_CMD_TLBI_EL2_ASID ] ="SMMU_CMD_TLBI_EL2_ASID" , + [SMMU_CMD_TLBI_EL2_VA ] ="SMMU_CMD_TLBI_EL2_VA" , + [SMMU_CMD_TLBI_EL2_VAA ] ="SMMU_CMD_TLBI_EL2_VAA" , + [SMMU_CMD_TLBI_S12_VMALL ] ="SMMU_CMD_TLBI_S12_VMALL" , + [SMMU_CMD_TLBI_S2_IPA ] ="SMMU_CMD_TLBI_S2_IPA" , + [SMMU_CMD_TLBI_NSNH_ALL ] ="SMMU_CMD_TLBI_NSNH_ALL" , + [SMMU_CMD_ATC_INV ] ="SMMU_CMD_ATC_INV" , + [SMMU_CMD_PRI_RESP ] ="SMMU_CMD_PRI_RESP" , + [SMMU_CMD_RESUME ] ="SMMU_CMD_RESUME" , + [SMMU_CMD_STALL_TERM ] ="SMMU_CMD_STALL_TERM" , + [SMMU_CMD_SYNC ] ="SMMU_CMD_SYNC" , +}; + +static inline const char *smmu_cmd_string (SMMUCommandType type) +{ +if (type >SMMU_CMD_NONE && type <ARRAY_SIZE (cmd_stringify)) { +return cmd_stringify[type] ? cmd_stringify[type] :"UNKNOWN" ; + }else { +return "INVALID" ; + } +} + +/* CMDQ fields */ + +typedef enum { +SMMU_CERROR_NONE =0 , +SMMU_CERROR_ILL , +SMMU_CERROR_ABT , +SMMU_CERROR_ATC_INV_SYNC , +}SMMUCmdError ; + +enum {/* Command completion notification */ +CMD_SYNC_SIG_NONE , +CMD_SYNC_SIG_IRQ , +CMD_SYNC_SIG_SEV , +}; + +#define CMD_TYPE (x) extract32((x)->word[0], 0 , 8) +#define CMD_SSEC (x) extract32((x)->word[0], 10, 1) +#define CMD_SSV (x) extract32((x)->word[0], 11, 1) +#define CMD_RESUME_AC (x) extract32((x)->word[0], 12, 1) +#define CMD_RESUME_AB (x) extract32((x)->word[0], 13, 1) +#define CMD_SYNC_CS (x) extract32((x)->word[0], 12, 2) +#define CMD_SSID (x) extract32((x)->word[0], 12, 20) +#define CMD_SID (x) ((x)->word[1]) +#define CMD_VMID (x) extract32((x)->word[1], 0 , 16) +#define CMD_ASID (x) extract32((x)->word[1], 16, 16) +#define CMD_RESUME_STAG (x) extract32((x)->word[2], 0 , 16) +#define CMD_RESP (x) extract32((x)->word[2], 11, 2) +#define CMD_LEAF (x) extract32((x)->word[2], 0 , 1) +#define CMD_STE_RANGE (x) extract32((x)->word[2], 0 , 5) +#define CMD_ADDR (x) ({ \ + uint64_t high = (uint64_t)(x)->word[3]; \ + uint64_t low = extract32((x)->word[2], 12, 20); \ + uint64_t addr = high << 32 | (low << 12); \ + addr; \ + }) + +int smmuv3_cmdq_consume (SMMUv3State *s); +#endif
hw/arm/smmuv3.c +136 -0
148 lines not on this page · raw message
hw/arm/trace-events +5 -0
@@ -18,3 +18,8 @@ smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: smmuv3_trigger_irq(int irq) "irq=%d" smmuv3_write_gerror(uint32_t toggled, uint32_t gerror) "toggled=0x%x, new GERROR=0x%x" smmuv3_write_gerrorn(uint32_t acked, uint32_t gerrorn) "acked=0x%x, new GERRORN=0x%x" +smmuv3_unhandled_cmd(uint32_t type) "Unhandled command type=%d" +smmuv3_cmdq_consume(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "prod=%d cons=%d prod.wrap=%d cons.wrap=%d" +smmuv3_cmdq_opcode(const char *opcode) "<--- %s" +smmuv3_cmdq_consume_out(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "prod:%d, cons:%d, prod_wrap:%d, cons_wrap:%d " +smmuv3_cmdq_consume_error(const char *cmd_name, uint8_t cmd_error) "Error on %s command execution: %d"
Signature · 1 line
2.5.5
PATCH v12 7/17 hw/arm/smmuv3: Implement MMIO write operations
In reply to Eric AugerNow we have relevant helpers for queue and irq management, let's implement MMIO write operations.
---
v11 -> v12:
- s/value/data in smmu_write_mmio and friends
- added Peter's R-b
v9 -> v10:
- s/hwaddr/uint64_t in trace-events
- added SMMU_FEATURE_2LVL_STE in this patch
- removed smmu_write64 and created writel/writell infra
- store capped log2size
- mask CR0 reserved bits
v7 -> v8:
- precise in the commit message invalidation commands are not yet treated.
- use new queue helpers
- do not decode unhandled commands at this stage
--- hw/arm/smmuv3-internal.h | 8 +-- hw/arm/smmuv3.c | 170 +++++++++++++++++++++++++++++++++++++++++++++-- hw/arm/trace-events | 6 ++ 3 files changed, 174 insertions(+), 10 deletions(-)
hw/arm/smmuv3-internal.h +3 -5
@@ -61,6 +61,8 @@ REG32(CR0, 0x20) FIELD(CR0, EVENTQEN, 2, 1) FIELD(CR0, CMDQEN, 3, 1) +#define SMMU_CR0_RESERVED 0xFFFFFC20 + REG32(CR0ACK, 0x24) REG32(CR1, 0x28)REG32 (CR2 ,0x2c )
@@ -149,10 +151,6 @@ static inline bool smmuv3_gerror_irq_enabled(SMMUv3State *s)return FIELD_EX32 (s->irq_ctrl,IRQ_CTRL ,GERROR_IRQEN ); } -/* public until callers get introduced */ -void smmuv3_trigger_irq (SMMUv3State *s,SMMUIrq irq,uint32_t gerror_mask); -void smmuv3_write_gerrorn (SMMUv3State *s,uint32_t gerrorn); -/* Queue Handling */ #define Q_BASE (q) ((q)->base & SMMU_BASE_ADDR_MASK)
@@ -314,6 +312,6 @@ enum { /* Command completion notification */ addr; \ }) -int smmuv3_cmdq_consume (SMMUv3State *s); +#define SMMU_FEATURE_2LVL_STE (1 << 0)#endif
hw/arm/smmuv3.c +165 -5
197 lines not on this page · raw message
hw/arm/trace-events +6 -0
@@ -23,3 +23,9 @@ smmuv3_cmdq_consume(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t con smmuv3_cmdq_opcode(const char *opcode) "<--- %s" smmuv3_cmdq_consume_out(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "prod:%d, cons:%d, prod_wrap:%d, cons_wrap:%d " smmuv3_cmdq_consume_error(const char *cmd_name, uint8_t cmd_error) "Error on %s command execution: %d" +smmuv3_update(bool is_empty, uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "q empty:%d prod:%d cons:%d p.wrap:%d p.cons:%d" +smmuv3_update_check_cmd(int error) "cmdq not enabled or error :0x%x" +smmuv3_write_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)" +smmuv3_write_mmio_idr(uint64_t addr, uint64_t val) "write to RO/Unimpl reg 0x%lx val64:0x%lx" +smmuv3_write_mmio_evtq_cons_bef_clear(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "Before clearing interrupt prod:0x%x cons:0x%x prod.w:%d cons.w:%d" +smmuv3_write_mmio_evtq_cons_after_clear(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "after clearing interrupt prod:0x%x cons:0x%x prod.w:%d cons.w:%d"
Signature · 1 line
2.5.5
PATCH v12 8/17 hw/arm/smmuv3: Event queue recording helper
In reply to Eric AugerLet's introduce a helper function aiming at recording an event in the event queue.
---
v11 -> v12:
- use do { ... } while (0)
- check cmd parameter in smmu_event_string
- camelcase struct names
v9 -> v10:
- rework SMMU_EVENT_STRING
- trigger a GERROR EVENTQ_ABT_ERR in case of eventq write failure
v8 -> v9:
- add SMMU_EVENT_STRING
v7 -> v8:
- use dma_addr_t instead of hwaddr in smmuv3_record_event()
- introduce struct SMMUEventInfo
- add event_stringify + helpers for all fields
--- hw/arm/smmuv3-internal.h | 148 ++++++++++++++++++++++++++++++++++++++++++++++- hw/arm/smmuv3.c | 108 ++++++++++++++++++++++++++++++++-- hw/arm/trace-events | 1 + 3 files changed, 249 insertions(+), 8 deletions(-)
hw/arm/smmuv3-internal.h +146 -2
@@ -206,8 +206,6 @@ static inline void smmu_write_cmdq_err(SMMUv3State *s, uint32_t err_type) s->cmdq.cons =FIELD_DP32 (s->cmdq.cons,CMDQ_CONS ,ERR , err_type); } -void smmuv3_write_eventq (SMMUv3State *s,Evt *evt); -/* Commands */ typedef enum SMMUCommandType {
@@ -314,4 +312,150 @@ enum { /* Command completion notification */#define SMMU_FEATURE_2LVL_STE (1 << 0) +/* Events */ + +typedef enum SMMUEventType { +SMMU_EVT_OK =0x00 , +SMMU_EVT_F_UUT , +SMMU_EVT_C_BAD_STREAMID , +SMMU_EVT_F_STE_FETCH , +SMMU_EVT_C_BAD_STE , +SMMU_EVT_F_BAD_ATS_TREQ , +SMMU_EVT_F_STREAM_DISABLED , +SMMU_EVT_F_TRANS_FORBIDDEN , +SMMU_EVT_C_BAD_SUBSTREAMID , +SMMU_EVT_F_CD_FETCH , +SMMU_EVT_C_BAD_CD , +SMMU_EVT_F_WALK_EABT , +SMMU_EVT_F_TRANSLATION =0x10 , +SMMU_EVT_F_ADDR_SIZE , +SMMU_EVT_F_ACCESS , +SMMU_EVT_F_PERMISSION , +SMMU_EVT_F_TLB_CONFLICT =0x20 , +SMMU_EVT_F_CFG_CONFLICT , +SMMU_EVT_E_PAGE_REQ =0x24 , +}SMMUEventType ; + +static const char *event_stringify[] = { + [SMMU_EVT_OK ] ="SMMU_EVT_OK" , + [SMMU_EVT_F_UUT ] ="SMMU_EVT_F_UUT" , + [SMMU_EVT_C_BAD_STREAMID ] ="SMMU_EVT_C_BAD_STREAMID" , + [SMMU_EVT_F_STE_FETCH ] ="SMMU_EVT_F_STE_FETCH" , + [SMMU_EVT_C_BAD_STE ] ="SMMU_EVT_C_BAD_STE" , + [SMMU_EVT_F_BAD_ATS_TREQ ] ="SMMU_EVT_F_BAD_ATS_TREQ" , + [SMMU_EVT_F_STREAM_DISABLED ] ="SMMU_EVT_F_STREAM_DISABLED" , + [SMMU_EVT_F_TRANS_FORBIDDEN ] ="SMMU_EVT_F_TRANS_FORBIDDEN" , + [SMMU_EVT_C_BAD_SUBSTREAMID ] ="SMMU_EVT_C_BAD_SUBSTREAMID" , + [SMMU_EVT_F_CD_FETCH ] ="SMMU_EVT_F_CD_FETCH" , + [SMMU_EVT_C_BAD_CD ] ="SMMU_EVT_C_BAD_CD" , + [SMMU_EVT_F_WALK_EABT ] ="SMMU_EVT_F_WALK_EABT" , + [SMMU_EVT_F_TRANSLATION ] ="SMMU_EVT_F_TRANSLATION" , + [SMMU_EVT_F_ADDR_SIZE ] ="SMMU_EVT_F_ADDR_SIZE" , + [SMMU_EVT_F_ACCESS ] ="SMMU_EVT_F_ACCESS" , + [SMMU_EVT_F_PERMISSION ] ="SMMU_EVT_F_PERMISSION" , + [SMMU_EVT_F_TLB_CONFLICT ] ="SMMU_EVT_F_TLB_CONFLICT" , + [SMMU_EVT_F_CFG_CONFLICT ] ="SMMU_EVT_F_CFG_CONFLICT" , + [SMMU_EVT_E_PAGE_REQ ] ="SMMU_EVT_E_PAGE_REQ" , +}; + +static inline const char *smmu_event_string (SMMUEventType type) +{ +if (type <ARRAY_SIZE (event_stringify)) { +return event_stringify[type] ? event_stringify[type] :"UNKNOWN" ; + }else { +return "INVALID" ; + } +} + +/* Encode an event record */ +typedef struct SMMUEventInfo { +SMMUEventType type; +uint32_t sid; +bool recorded; +bool record_trans_faults; +union { +struct { +uint32_t ssid; +bool ssv; +dma_addr_t addr; +bool rnw; +bool pnu; +bool ind; + } f_uut; +struct SSIDInfo { +uint32_t ssid; +bool ssv; + } c_bad_streamid; +struct SSIDAddrInfo { +uint32_t ssid; +bool ssv; +dma_addr_t addr; + } f_ste_fetch; +struct SSIDInfo c_bad_ste; +struct { +dma_addr_t addr; +bool rnw; + } f_transl_forbidden; +struct { +uint32_t ssid; + } c_bad_substream; +struct SSIDAddrInfo f_cd_fetch; +struct SSIDInfo c_bad_cd; +struct FullInfo { +bool stall; +uint16_t stag; +uint32_t ssid; +bool ssv; +bool s2; +dma_addr_t addr; +bool rnw; +bool pnu; +bool ind; +uint8_t class; +dma_addr_t addr2; + } f_walk_eabt; +struct FullInfo f_translation; +struct FullInfo f_addr_size; +struct FullInfo f_access; +struct FullInfo f_permission; +struct SSIDInfo f_cfg_conflict; +/** +* not supported yet: +* F_BAD_ATS_TREQ +* F_BAD_ATS_TREQ +* F_TLB_CONFLICT +* E_PAGE_REQUEST +* IMPDEF_EVENTn +*/ + } u; +}SMMUEventInfo ; + +/* EVTQ fields */ + +#define EVT_Q_OVERFLOW (1 << 31) + +#define EVT_SET_TYPE (x, v) deposit32((x)->word[0], 0 , 8 , v) +#define EVT_SET_SSV (x, v) deposit32((x)->word[0], 11, 1 , v) +#define EVT_SET_SSID (x, v) deposit32((x)->word[0], 12, 20, v) +#define EVT_SET_SID (x, v) ((x)->word[1] = v) +#define EVT_SET_STAG (x, v) deposit32((x)->word[2], 0 , 16, v) +#define EVT_SET_STALL (x, v) deposit32((x)->word[2], 31, 1 , v) +#define EVT_SET_PNU (x, v) deposit32((x)->word[3], 1 , 1 , v) +#define EVT_SET_IND (x, v) deposit32((x)->word[3], 2 , 1 , v) +#define EVT_SET_RNW (x, v) deposit32((x)->word[3], 3 , 1 , v) +#define EVT_SET_S2 (x, v) deposit32((x)->word[3], 7 , 1 , v) +#define EVT_SET_CLASS (x, v) deposit32((x)->word[3], 8 , 2 , v) +#define EVT_SET_ADDR (x, addr) \ + do { \ + (x)->word[5] = (uint32_t)(addr >> 32); \ + (x)->word[4] = (uint32_t)(addr & 0xffffffff); \ + } while (0) +#define EVT_SET_ADDR2 (x, addr) \ + do { \ + deposit32((x)->word[7], 3, 29, addr >> 16); \ + deposit32((x)->word[7], 0, 16, addr & 0xffff);\ + } while (0) + +void smmuv3_record_event (SMMUv3State *s,SMMUEventInfo *event); +#endif
hw/arm/smmuv3.c +102 -6
125 lines not on this page · raw message
hw/arm/trace-events +1 -0
@@ -29,3 +29,4 @@ smmuv3_write_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: smmuv3_write_mmio_idr(uint64_t addr, uint64_t val) "write to RO/Unimpl reg 0x%lx val64:0x%lx" smmuv3_write_mmio_evtq_cons_bef_clear(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "Before clearing interrupt prod:0x%x cons:0x%x prod.w:%d cons.w:%d" smmuv3_write_mmio_evtq_cons_after_clear(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "after clearing interrupt prod:0x%x cons:0x%x prod.w:%d cons.w:%d" +smmuv3_record_event(const char *type, uint32_t sid) "%s sid=%d"
Signature · 1 line
2.5.5
PATCH v12 9/17 hw/arm/smmuv3: Implement translate callback
In reply to Eric AugerThis patch implements the IOMMU Memory Region translate() callback. Most of the code relates to the translation configuration decoding and check (STE, CD).
---
v11 -> v12:
- check fixes in decode_cd
- in decode_ste, first check the STE validity
- initial_level computed in ptw
v10 -> v11:
- in case of error, print ret instead of the event.type which may be set to OK if recording is off. This was misleading.
- trace point at the output of translate, only in case of sucess
v9 -> v10:
- s/hwaddr/uint64_t in trace-events
- use qemu_log_mask(LOG_UNIMP, in some situations where assert was done
- report bad_ste if stage2 is found in STE
- check TTB does not exceed IPS
- fix oas2bits, CD_TTB
- tg2granule handles illegal cases
- convert L1STD_L2PTR macro into an inline function
- added S1STALLD, CD_A, CD_S, CD_HA, CD_HD checks
v8 -> v9:
- use SMMU_EVENT_STRING macro
- get rid of last erro_report's
- decode asid
- handle config abort before ptw
- add 64-bit single-copy atomic comment
v7 -> v8:
- use address_space_rw
- s/Ste/STE, s/Cd/CD
- use dma_memory_read
- remove everything related to stage 2
- collect data for both TTx
- renamings
- pass the event handle all along the config decoding path
- decode tbi, ars
--- hw/arm/smmuv3-internal.h | 160 +++++++++++++++++++++ hw/arm/smmuv3.c | 358 +++++++++++++++++++++++++++++++++++++++++++++++ hw/arm/trace-events | 9 ++ 3 files changed, 527 insertions(+)
hw/arm/smmuv3-internal.h +160 -0
@@ -458,4 +458,164 @@ typedef struct SMMUEventInfo {void smmuv3_record_event (SMMUv3State *s,SMMUEventInfo *event); +/* Configuration Data */ + +/* STE Level 1 Descriptor */ +typedef struct STEDesc { +uint32_t word[2 ]; +}STEDesc ; + +/* CD Level 1 Descriptor */ +typedef struct CDDesc { +uint32_t word[2 ]; +}CDDesc ; + +/* Stream Table Entry(STE) */ +typedef struct STE { +uint32_t word[16 ]; +}STE ; + +/* Context Descriptor(CD) */ +typedef struct CD { +uint32_t word[16 ]; +}CD ; + +/* STE fields */ + +#define STE_VALID (x) extract32((x)->word[0], 0, 1) + +#define STE_CONFIG (x) extract32((x)->word[0], 1, 3) +#define STE_CFG_S1_ENABLED (config) (config & 0x1) +#define STE_CFG_S2_ENABLED (config) (config & 0x2) +#define STE_CFG_ABORT (config) (!(config & 0x4)) +#define STE_CFG_BYPASS (config) (config == 0x4) + +#define STE_S1FMT (x) extract32((x)->word[0], 4 , 2) +#define STE_S1CDMAX (x) extract32((x)->word[1], 27, 5) +#define STE_S1STALLD (x) extract32((x)->word[2], 27, 1) +#define STE_EATS (x) extract32((x)->word[2], 28, 2) +#define STE_STRW (x) extract32((x)->word[2], 30, 2) +#define STE_S2VMID (x) extract32((x)->word[4], 0 , 16) +#define STE_S2T0SZ (x) extract32((x)->word[5], 0 , 6) +#define STE_S2SL0 (x) extract32((x)->word[5], 6 , 2) +#define STE_S2TG (x) extract32((x)->word[5], 14, 2) +#define STE_S2PS (x) extract32((x)->word[5], 16, 3) +#define STE_S2AA64 (x) extract32((x)->word[5], 19, 1) +#define STE_S2HD (x) extract32((x)->word[5], 24, 1) +#define STE_S2HA (x) extract32((x)->word[5], 25, 1) +#define STE_S2S (x) extract32((x)->word[5], 26, 1) +#define STE_CTXPTR (x) \ + ({ \ + unsigned long addr; \ + addr = (uint64_t)extract32((x)->word[1], 0, 16) << 32; \ + addr |= (uint64_t)((x)->word[0] & 0xffffffc0); \ + addr; \ + }) + +#define STE_S2TTB (x) \ + ({ \ + unsigned long addr; \ + addr = (uint64_t)extract32((x)->word[7], 0, 16) << 32; \ + addr |= (uint64_t)((x)->word[6] & 0xfffffff0); \ + addr; \ + }) + +static inline int oas2bits (int oas_field) +{ +switch (oas_field) { +case 0 : +return 32 ; +case 1 : +return 36 ; +case 2 : +return 40 ; +case 3 : +return 42 ; +case 4 : +return 44 ; +case 5 : +return 48 ; + } +return -1 ; +} + +static inline int pa_range (STE *ste) +{ +int oas_field =MIN (STE_S2PS (ste),SMMU_IDR5_OAS ); + +if (!STE_S2AA64 (ste)) { +return 40 ; + } + +return oas2bits (oas_field); +} + +#define MAX_PA (ste) ((1 << pa_range(ste)) - 1) + +/* CD fields */ + +#define CD_VALID (x) extract32((x)->word[0], 30, 1) +#define CD_ASID (x) extract32((x)->word[1], 16, 16) +#define CD_TTB (x, sel) \ + ({ \ + uint64_t hi, lo; \ + hi = extract32((x)->word[(sel) * 2 + 3], 0, 19); \ + hi <<= 32; \ + lo = (x)->word[(sel) * 2 + 2] & ~0xfULL; \ + hi | lo; \ + }) + +#define CD_TSZ (x, sel) extract32((x)->word[0], (16 * (sel)) + 0, 6) +#define CD_TG (x, sel) extract32((x)->word[0], (16 * (sel)) + 6, 2) +#define CD_EPD (x, sel) extract32((x)->word[0], (16 * (sel)) + 14, 1) +#define CD_ENDI (x) extract32((x)->word[0], 15, 1) +#define CD_IPS (x) extract32((x)->word[1], 0 , 3) +#define CD_TBI (x) extract32((x)->word[1], 6 , 2) +#define CD_HD (x) extract32((x)->word[1], 10 , 1) +#define CD_HA (x) extract32((x)->word[1], 11 , 1) +#define CD_S (x) extract32((x)->word[1], 12, 1) +#define CD_R (x) extract32((x)->word[1], 13, 1) +#define CD_A (x) extract32((x)->word[1], 14, 1) +#define CD_AARCH64 (x) extract32((x)->word[1], 9 , 1) + +#define CDM_VALID (x) ((x)->word[0] & 0x1) + +static inline int is_cd_valid (SMMUv3State *s,STE *ste,CD *cd) +{ +return CD_VALID (cd); +} + +/** +* tg2granule - Decodes the CD translation granule size field according +* to the ttbr in use +* @bits: TG0/1 fields +* @ttbr: ttbr index in use +*/ +static inline int tg2granule (int bits,int ttbr) +{ +switch (bits) { +case 0 : +return ttbr ?0 :12 ; +case 1 : +return ttbr ?14 :16 ; +case 2 : +return ttbr ?12 :14 ; +case 3 : +return ttbr ?16 :0 ; +default : +return 0 ; + } +} + +static inline uint64_t l1std_l2ptr (STEDesc *desc) +{ +uint64_t hi, lo; + + hi = desc->word[1 ]; + lo = desc->word[0 ] & ~0x1fULL ; +return hi <<32 | lo; +} + +#define L1STD_SPAN (stm) (extract32((stm)->word[0], 0, 4)) +#endif
hw/arm/smmuv3.c +358 -0
370 lines not on this page · raw message
hw/arm/trace-events +9 -0
@@ -30,3 +30,12 @@ smmuv3_write_mmio_idr(uint64_t addr, uint64_t val) "write to RO/Unimpl reg 0x%lx smmuv3_write_mmio_evtq_cons_bef_clear(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "Before clearing interrupt prod:0x%x cons:0x%x prod.w:%d cons.w:%d" smmuv3_write_mmio_evtq_cons_after_clear(uint32_t prod, uint32_t cons, uint8_t prod_wrap, uint8_t cons_wrap) "after clearing interrupt prod:0x%x cons:0x%x prod.w:%d cons.w:%d" smmuv3_record_event(const char *type, uint32_t sid) "%s sid=%d" +smmuv3_find_ste(uint16_t sid, uint32_t features, uint16_t sid_split) "SID:0x%x features:0x%x, sid_split:0x%x" +smmuv3_find_ste_2lvl(uint64_t strtab_base, uint64_t l1ptr, int l1_ste_offset, uint64_t l2ptr, int l2_ste_offset, int max_l2_ste) "strtab_base:0x%lx l1ptr:0x%"PRIx64" l1_off:0x%x, l2ptr:0x%"PRIx64" l2_off:0x%x max_l2_ste:%d" +smmuv3_get_ste(uint64_t addr) "STE addr: 0x%"PRIx64 +smmuv3_translate_bypass(const char *n, uint16_t sid, uint64_t addr, bool is_write) "%s sid=%d bypass iova:0x%"PRIx64" is_write=%d" +smmuv3_translate_in(uint16_t sid, int pci_bus_num, uint64_t strtab_base) "SID:0x%x bus:%d strtab_base:0x%"PRIx64 +smmuv3_get_cd(uint64_t addr) "CD addr: 0x%"PRIx64 +smmuv3_translate(const char *n, uint16_t sid, uint64_t iova, uint64_t translated, int perm) "%s sid=%d iova=0x%"PRIx64" translated=0x%"PRIx64" perm=0x%x" +smmuv3_decode_cd(uint32_t oas) "oas=%d" +smmuv3_decode_cd_tt(int i, uint32_t tsz, uint64_t ttb, uint32_t granule_sz) "TT[%d]:tsz:%d ttb:0x%"PRIx64" granule_sz:%d"
Signature · 1 line
2.5.5
PATCH v12 10/17 hw/arm/smmuv3: Abort on vfio or vhost case
In reply to Eric AugerAt the moment, the SMMUv3 does not support notification on TLB invalidation. So let's log an error as soon as such notifier gets enabled.
---
v11 -> v12:
- Added Peter's R-b
--- hw/arm/smmuv3.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
hw/arm/smmuv3.c +11 -0
@@ -1147,12 +1147,23 @@ static void smmuv3_class_init(ObjectClass *klass, void *data) dc->realize = smmu_realize; } +static void smmuv3_notify_flag_changed (IOMMUMemoryRegion *iommu, +IOMMUNotifierFlag old, +IOMMUNotifierFlag new) +{ +if (old ==IOMMU_NOTIFIER_NONE ) { +warn_report ("SMMUV3 does not support vhost/vfio integration yet: " +"devices of those types will not function properly" ); + } +} +static void smmuv3_iommu_memory_region_class_init (ObjectClass *klass,void *data) {IOMMUMemoryRegionClass *imrc =IOMMU_MEMORY_REGION_CLASS (klass); imrc->translate = smmuv3_translate; + imrc->notify_flag_changed = smmuv3_notify_flag_changed; }static const TypeInfo smmuv3_type_info = {
Signature · 1 line
2.5.5
PATCH v12 11/17 target/arm/kvm: Translate the MSI doorbell in kvm_arch_fixup_msi_route
In reply to Eric AugerIn case the MSI is translated by an IOMMU we need to fixup the MSI route with the translated address.
---
v11 -> v12:
- fix rcu_read_lock and mr ref leaks
- check mrs.mr
v9 -> v10:
- use address_space_translate
v5 -> v6:
- use IOMMUMemoryRegionClass API
--- target/arm/kvm.c | 38 +++++++++++++++++++++++++++++++++++++- target/arm/trace-events | 3 +++ 2 files changed, 40 insertions(+), 1 deletion(-)
target/arm/kvm.c +37 -1
@@ -20,8 +20,10 @@#include "sysemu/kvm.h" #include "kvm_arm.h" #include "cpu.h" +#include "trace.h" #include "internals.h" #include "hw/arm/arm.h" +#include "hw/pci/pci.h" #include "exec/memattrs.h" #include "exec/address-spaces.h" #include "hw/boards.h"
@@ -649,7 +651,41 @@ int kvm_arm_vgic_probe(void)int kvm_arch_fixup_msi_route (struct kvm_irq_routing_entry *route,uint64_t address,uint32_t data,PCIDevice *dev) { -return 0 ; +AddressSpace *as =pci_device_iommu_address_space (dev); +hwaddr xlat, len, doorbell_gpa; +MemoryRegionSection mrs; +MemoryRegion *mr; +int ret =1 ; + +if (as == &address_space_memory) { +return 0 ; + } + +/* MSI doorbell address is translated by an IOMMU */ + +rcu_read_lock (); + mr =address_space_translate (as, address, &xlat, &len, true); +if (!mr) { + goto unlock; + } + mrs =memory_region_find (mr, xlat,1 ); +if (!mrs.mr) { + goto unlock; + } + + doorbell_gpa = mrs.offset_within_address_space; +memory_region_unref (mrs.mr); + + route->u.msi.address_lo = doorbell_gpa; + route->u.msi.address_hi = doorbell_gpa >>32 ; + +trace_kvm_arm_fixup_msi_route (address, doorbell_gpa); + + ret =0 ; + +unlock: +rcu_read_unlock (); +return ret; } int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
target/arm/trace-events +3 -0
@@ -8,3 +8,6 @@ arm_gt_tval_write(int timer, uint64_t value) "gt_tval_write: timer %d value 0x%" arm_gt_ctl_write(int timer, uint64_t value) "gt_ctl_write: timer %d value 0x%" PRIx64 arm_gt_imask_toggle(int timer, int irqstate) "gt_ctl_write: timer %d IMASK toggle, new irqstate %d" arm_gt_cntvoff_write(uint64_t value) "gt_cntvoff_write: value 0x%" PRIx64 + +# target/arm/kvm.c +kvm_arm_fixup_msi_route(uint64_t iova, uint64_t gpa) "MSI iova = 0x%"PRIx64" is translated into 0x%"PRIx64
Signature · 1 line
2.5.5
PATCH v12 12/17 hw/arm/virt: Add SMMUv3 to the virt board
In reply to Eric AugerFrom: Prem Mallappa <[email protected]>
Add code to instantiate an smmuv3 in virt machine. A new iommu integer member is introduced in VirtMachineState to store the type of the iommu in use.
---
v9 -> v10:
- add VirtIOMMUType
- add Peter's R-b
v7 -> v8:
- integer iommu member
- add primary-bus property
v4 -> v5:
- add dma-coherent property
v2 -> v3:
- vbi was removed. Use vms instead
- migrate to new smmu binding format (iommu-map)
- don't use appendprop anymore
- add vms->smmu and guard instantiation with this latter
- interrupts type changed to edge
--- hw/arm/virt.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++++++- include/hw/arm/virt.h | 10 ++++++++ 2 files changed, 73 insertions(+), 1 deletion(-)
hw/arm/virt.c +63 -1
@@ -58,6 +58,7 @@#include "hw/smbios/smbios.h" #include "qapi/visitor.h" #include "standard-headers/linux/input.h" +#include "hw/arm/smmuv3.h" #define DEFINE_VIRT_MACHINE_LATEST (major, minor, latest) \ static void virt_##major##_##minor##_class_init(ObjectClass *oc, \
@@ -141,6 +142,7 @@ static const MemMapEntry a15memmap[] = { [VIRT_FW_CFG] = { 0x09020000, 0x00000018 }, [VIRT_GPIO] = { 0x09030000, 0x00001000 }, [VIRT_SECURE_UART] = { 0x09040000, 0x00001000 }, + [VIRT_SMMU] = { 0x09050000, 0x00020000 }, [VIRT_MMIO] = { 0x0a000000, 0x00000200 }, /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */ [VIRT_PLATFORM_BUS] = { 0x0c000000, 0x02000000 },
@@ -161,6 +163,7 @@ static const int a15irqmap[] = { [VIRT_SECURE_UART] = 8, [VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */ [VIRT_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */ + [VIRT_SMMU] = 74, /* ...to 74 + NUM_SMMU_IRQS - 1 */ [VIRT_PLATFORM_BUS] = 112, /* ...to 112 + PLATFORM_BUS_NUM_IRQS -1 */ };
@@ -942,7 +945,57 @@ static void create_pcie_irq_map(const VirtMachineState *vms, 0x7 /* PCI irq */); } -static void create_pcie (const VirtMachineState *vms,qemu_irq *pic) +static void create_smmu (const VirtMachineState *vms,qemu_irq *pic, +PCIBus *bus) +{ +char *node; +const char compat[] ="arm,smmu-v3" ; +int irq = vms->irqmap[VIRT_SMMU ]; +int i; +hwaddr base = vms->memmap[VIRT_SMMU ].base; +hwaddr size = vms->memmap[VIRT_SMMU ].size; +const char irq_names[] ="eventq\0priq\0cmdq-sync\0gerror" ; +DeviceState *dev; + +if (vms->iommu !=VIRT_IOMMU_SMMUV3 || !vms->iommu_phandle) { +return ; + } + + dev =qdev_create (NULL ,"arm-smmuv3" ); + +object_property_set_link (OBJECT (dev),OBJECT (bus),"primary-bus" , + &error_abort); +qdev_init_nofail (dev); +sysbus_mmio_map (SYS_BUS_DEVICE (dev),0 , base); +for (i =0 ; i <NUM_SMMU_IRQS ; i++) { +sysbus_connect_irq (SYS_BUS_DEVICE (dev), i, pic[irq + i]); + } + + node =g_strdup_printf ("/smmuv3@%" PRIx64, base); +qemu_fdt_add_subnode (vms->fdt, node); +qemu_fdt_setprop (vms->fdt, node,"compatible" , compat,sizeof (compat)); +qemu_fdt_setprop_sized_cells (vms->fdt, node,"reg" ,2 , base,2 , size); + +qemu_fdt_setprop_cells (vms->fdt, node,"interrupts" , +GIC_FDT_IRQ_TYPE_SPI , irq ,GIC_FDT_IRQ_FLAGS_EDGE_LO_HI , +GIC_FDT_IRQ_TYPE_SPI , irq +1 ,GIC_FDT_IRQ_FLAGS_EDGE_LO_HI , +GIC_FDT_IRQ_TYPE_SPI , irq +2 ,GIC_FDT_IRQ_FLAGS_EDGE_LO_HI , +GIC_FDT_IRQ_TYPE_SPI , irq +3 ,GIC_FDT_IRQ_FLAGS_EDGE_LO_HI ); + +qemu_fdt_setprop (vms->fdt, node,"interrupt-names" , irq_names, +sizeof (irq_names)); + +qemu_fdt_setprop_cell (vms->fdt, node,"clocks" , vms->clock_phandle); +qemu_fdt_setprop_string (vms->fdt, node,"clock-names" ,"apb_pclk" ); +qemu_fdt_setprop (vms->fdt, node,"dma-coherent" ,NULL ,0 ); + +qemu_fdt_setprop_cell (vms->fdt, node,"#iommu-cells" ,1 ); + +qemu_fdt_setprop_cell (vms->fdt, node,"phandle" , vms->iommu_phandle); +g_free (node); +} + +static void create_pcie (VirtMachineState *vms,qemu_irq *pic) {hwaddr base_mmio = vms->memmap[VIRT_PCIE_MMIO ].base;hwaddr size_mmio = vms->memmap[VIRT_PCIE_MMIO ].size;
@@ -1055,6 +1108,15 @@ static void create_pcie(const VirtMachineState *vms, qemu_irq *pic)qemu_fdt_setprop_cell (vms->fdt, nodename,"#interrupt-cells" ,1 );create_pcie_irq_map (vms, vms->gic_phandle, irq, nodename); +if (vms->iommu) { + vms->iommu_phandle =qemu_fdt_alloc_phandle (vms->fdt); + +create_smmu (vms, pic, pci->bus); + +qemu_fdt_setprop_cells (vms->fdt, nodename,"iommu-map" , +0x0 , vms->iommu_phandle,0x0 ,0x10000 ); + } +g_free (nodename); }
include/hw/arm/virt.h +10 -0
@@ -38,6 +38,7 @@#define NUM_GICV2M_SPIS 64#define NUM_VIRTIO_TRANSPORTS 32 +#define NUM_SMMU_IRQS 4#define ARCH_GICV3_MAINT_IRQ 9
@@ -59,6 +60,7 @@ enum {VIRT_GIC_V2M ,VIRT_GIC_ITS ,VIRT_GIC_REDIST , +VIRT_SMMU ,VIRT_UART ,VIRT_MMIO , VIRT_RTC,
@@ -74,6 +76,12 @@ enum { VIRT_SECURE_MEM, }; +typedef enum VirtIOMMUType { +VIRT_IOMMU_NONE , +VIRT_IOMMU_SMMUV3 , +VIRT_IOMMU_VIRTIO , +}VirtIOMMUType ; + typedef struct MemMapEntry { hwaddr base; hwaddr size;
@@ -97,6 +105,7 @@ typedef struct {bool its;bool virt;int32_t gic_version; +VirtIOMMUType iommu;struct arm_boot_info bootinfo;const MemMapEntry *memmap;const int *irqmap;
@@ -106,6 +115,7 @@ typedef struct {uint32_t clock_phandle;uint32_t gic_phandle;uint32_t msi_phandle; +uint32_t iommu_phandle;int psci_conduit; } VirtMachineState;
Signature · 1 line
2.5.5
PATCH v12 13/17 hw/arm/virt-acpi-build: Add smmuv3 node in IORT table
In reply to Eric AugerFrom: Prem Mallappa <[email protected]>
This patch builds the smmuv3 node in the ACPI IORT table.
The RID space of the root complex, which spans 0x0-0x10000 maps to streamid space 0x0-0x10000 in smmuv3, which in turn maps to deviceid space 0x0-0x10000 in the ITS group.
The guest must feature the IOMMU probe deferral series
(https://lkml.org/lkml/2017/4/10/214) which fixes streamid multiple lookup. This bug is not related to the SMMU emulation.
---
v9 -> v10:
- replaced vms->iommu tests by vms->iommu == VIRT_IOMMU_SMMUV3
- Added Shannon's R-b
v2 -> v3:
- integrate into the existing IORT table made up of ITS, RC nodes
- take into account vms->smmu
- match linux actbl2.h acpi_iort_smmu_v3 field names
--- hw/arm/virt-acpi-build.c | 55 +++++++++++++++++++++++++++++++++++++++------ include/hw/acpi/acpi-defs.h | 15 +++++++++++++ 2 files changed, 63 insertions(+), 7 deletions(-)
hw/arm/virt-acpi-build.c +48 -7
@@ -393,19 +393,26 @@ build_rsdp(GArray *rsdp_table, BIOSLinker *linker, unsigned xsdt_tbl_offset) }static void -build_iort (GArray *table_data,BIOSLinker *linker) +build_iort (GArray *table_data,BIOSLinker *linker,VirtMachineState *vms) { -int iort_start = table_data->len; +int nb_nodes, iort_start = table_data->len;AcpiIortIdMapping *idmap;AcpiIortItsGroup *its;AcpiIortTable *iort; -size_t node_size, iort_length; +AcpiIortSmmu3 *smmu; +size_t node_size, iort_length, smmu_offset =0 ;AcpiIortRC *rc; iort =acpi_data_push (table_data,sizeof (*iort)); +if (vms->iommu ==VIRT_IOMMU_SMMUV3 ) { + nb_nodes =3 ;/* RC, ITS, SMMUv3 */ + }else { + nb_nodes =2 ;/* RC, ITS */ + } + iort_length =sizeof (*iort); - iort->node_count =cpu_to_le32 (2 );/* RC and ITS nodes */ + iort->node_count =cpu_to_le32 (nb_nodes); iort->node_offset =cpu_to_le32 (sizeof (*iort));/* ITS group node */
@@ -418,6 +425,34 @@ build_iort(GArray *table_data, BIOSLinker *linker) its->its_count =cpu_to_le32 (1 ); its->identifiers[0 ] =0 ;/* MADT translation_id */ +if (vms->iommu ==VIRT_IOMMU_SMMUV3 ) { +int irq = vms->irqmap[VIRT_SMMU ]; + +/* SMMUv3 node */ + smmu_offset = iort->node_offset + node_size; + node_size =sizeof (*smmu) +sizeof (*idmap); + iort_length += node_size; + smmu =acpi_data_push (table_data, node_size); + + smmu->type =ACPI_IORT_NODE_SMMU_V3 ; + smmu->length =cpu_to_le16 (node_size); + smmu->mapping_count =cpu_to_le32 (1 ); + smmu->mapping_offset =cpu_to_le32 (sizeof (*smmu)); + smmu->base_address =cpu_to_le64 (vms->memmap[VIRT_SMMU ].base); + smmu->event_gsiv =cpu_to_le32 (irq); + smmu->pri_gsiv =cpu_to_le32 (irq +1 ); + smmu->gerr_gsiv =cpu_to_le32 (irq +2 ); + smmu->sync_gsiv =cpu_to_le32 (irq +3 ); + +/* Identity RID mapping covering the whole input RID range */ + idmap = &smmu->id_mapping_array[0 ]; + idmap->input_base =0 ; + idmap->id_count =cpu_to_le32 (0xFFFF ); + idmap->output_base =0 ; +/* output IORT node is the ITS group node (the first node) */ + idmap->output_reference =cpu_to_le32 (iort->node_offset); + } +/* Root Complex Node */ node_size =sizeof (*rc) +sizeof (*idmap); iort_length += node_size;
@@ -438,8 +473,14 @@ build_iort(GArray *table_data, BIOSLinker *linker) idmap->input_base =0 ; idmap->id_count =cpu_to_le32 (0xFFFF ); idmap->output_base =0 ; -/* output IORT node is the ITS group node (the first node) */ - idmap->output_reference =cpu_to_le32 (iort->node_offset); + +if (vms->iommu ==VIRT_IOMMU_SMMUV3 ) { +/* output IORT node is the smmuv3 node */ + idmap->output_reference =cpu_to_le32 (smmu_offset); + }else { +/* output IORT node is the ITS group node (the first node) */ + idmap->output_reference =cpu_to_le32 (iort->node_offset); + } iort->length =cpu_to_le32 (iort_length);
@@ -777,7 +818,7 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables)if (its_class_name () && !vmc->no_its) {acpi_add_table (table_offsets, tables_blob); -build_iort (tables_blob, tables->linker); +build_iort (tables_blob, tables->linker, vms); }/* XSDT is pointed to by RSDP */
include/hw/acpi/acpi-defs.h +15 -0
@@ -628,6 +628,21 @@ struct AcpiIortItsGroup { }QEMU_PACKED ;typedef struct AcpiIortItsGroup AcpiIortItsGroup ; +struct AcpiIortSmmu3 { +ACPI_IORT_NODE_HEADER_DEF + uint64_t base_address; +uint32_t flags; +uint32_t reserved2; +uint64_t vatos_address; +uint32_t model; +uint32_t event_gsiv; +uint32_t pri_gsiv; +uint32_t gerr_gsiv; +uint32_t sync_gsiv; +AcpiIortIdMapping id_mapping_array[0 ]; +}QEMU_PACKED ; +typedef struct AcpiIortSmmu3 AcpiIortSmmu3 ; +struct AcpiIortRC {ACPI_IORT_NODE_HEADER_DEF AcpiIortMemoryAccess memory_properties;
Signature · 1 line
2.5.5
PATCH v12 14/17 hw/arm/virt: Introduce the iommu option
In reply to Eric AugerARM virt machine now exposes a new "iommu" option.
The SMMUv3 IOMMU is instantiated using -machine virt,iommu=smmuv3.
---
v11 -> v12:
- change the property description
- add Peter's R-b
v9 -> v10:
- remove no_iommu
v7 -> v8:
- Revert to machine option, now dubbed "iommu", preparing for virtio instantiation.
v5 -> v6: machine 2_11
--- hw/arm/virt.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
hw/arm/virt.c +36 -0
@@ -1560,6 +1560,34 @@ static void virt_set_gic_version(Object *obj, const char *value, Error **errp) } } +static char *virt_get_iommu (Object *obj,Error **errp) +{ +VirtMachineState *vms =VIRT_MACHINE (obj); + +switch (vms->iommu) { +case VIRT_IOMMU_NONE : +return g_strdup ("none" ); +case VIRT_IOMMU_SMMUV3 : +return g_strdup ("smmuv3" ); +default : +g_assert_not_reached (); + } +} + +static void virt_set_iommu (Object *obj,const char *value,Error **errp) +{ +VirtMachineState *vms =VIRT_MACHINE (obj); + +if (!strcmp (value,"smmuv3" )) { + vms->iommu =VIRT_IOMMU_SMMUV3 ; + }else if (!strcmp (value,"none" )) { + vms->iommu =VIRT_IOMMU_NONE ; + }else { +error_setg (errp,"Invalid iommu value" ); +error_append_hint (errp,"Valid values are none, smmuv3.\n" ); + } +} +static CpuInstanceProperties virt_cpu_index_to_props (MachineState *ms,unsigned cpu_index) {
@@ -1692,6 +1720,14 @@ static void virt_2_12_instance_init(Object *obj) NULL); } +/* Default disallows iommu instantiation */ + vms->iommu =VIRT_IOMMU_NONE ; +object_property_add_str (obj,"iommu" , virt_get_iommu, virt_set_iommu,NULL ); +object_property_set_description (obj,"iommu" , +"Set the IOMMU type. " +"Valid values are none and smmuv3" , +NULL ); + vms->memmap = a15memmap; vms->irqmap = a15irqmap; }
Signature · 1 line
2.5.5
PATCH v12 15/17 hw/arm/smmuv3: Cache/invalidate config data
In reply to Eric AugerLet's cache config data to avoid fetching and parsing STE/CD structures on each translation. We invalidate them on data structure invalidation commands.
---
v11 -> v12:
- only insert the new config if decode_cfg succeeds
- use smmu_get_sid for trace_* and store hits/misses in the SMMUDevice
- s/smmuv3_put_config/smmuv3_flush_config
- document smmuv3_get_config
- removing the mutex as BQL does the job
--- hw/arm/smmu-common.c | 26 ++++++++- hw/arm/smmuv3.c | 130 +++++++++++++++++++++++++++++++++++++++++-- hw/arm/trace-events | 6 ++ include/hw/arm/smmu-common.h | 5 ++ 4 files changed, 159 insertions(+), 8 deletions(-)
hw/arm/smmu-common.c +24 -2
@@ -297,6 +297,8 @@ static AddressSpace *smmu_find_add_as(PCIBus *bus, void *opaque, int devfn) sdev->smmu = s; sdev->bus = bus; sdev->devfn = devfn; + sdev->cfg_cache_misses =0 ; + sdev->cfg_cache_hits =0 ; memory_region_init_iommu(&sdev->iommu, sizeof(sdev->iommu), s->mrtypename,
@@ -310,6 +312,24 @@ static AddressSpace *smmu_find_add_as(PCIBus *bus, void *opaque, int devfn)return &sdev->as; } +IOMMUMemoryRegion *smmu_iommu_mr (SMMUState *s,uint32_t sid) +{ +uint8_t bus_n, devfn; +SMMUPciBus *smmu_bus; +SMMUDevice *smmu; + + bus_n =PCI_BUS_NUM (sid); + smmu_bus =smmu_find_smmu_pcibus (s, bus_n); +if (smmu_bus) { + devfn = sid &0x7 ; + smmu = smmu_bus->pbdev[devfn]; +if (smmu) { +return &smmu->iommu; + } + } +return NULL ; +} +static void smmu_base_realize (DeviceState *dev,Error **errp) {SMMUState *s =ARM_SMMU (dev);
@@ -321,7 +341,7 @@ static void smmu_base_realize(DeviceState *dev, Error **errp)error_propagate (errp, local_err);return ; } - + s->configs =g_hash_table_new_full (NULL ,NULL ,NULL , g_free); s->smmu_pcibus_by_busptr =g_hash_table_new (NULL ,NULL );if (s->primary_bus) {
@@ -333,7 +353,9 @@ static void smmu_base_realize(DeviceState *dev, Error **errp)static void smmu_base_reset (DeviceState *dev) { -/* will be filled later on */ +SMMUState *s =ARM_SMMU (dev); + +g_hash_table_remove_all (s->configs); }static Property smmu_dev_properties[] = {
hw/arm/smmuv3.c +124 -6
176 lines not on this page · raw message
hw/arm/trace-events +6 -0
@@ -39,3 +39,9 @@ smmuv3_get_cd(uint64_t addr) "CD addr: 0x%"PRIx64 smmuv3_translate(const char *n, uint16_t sid, uint64_t iova, uint64_t translated, int perm) "%s sid=%d iova=0x%"PRIx64" translated=0x%"PRIx64" perm=0x%x" smmuv3_decode_cd(uint32_t oas) "oas=%d" smmuv3_decode_cd_tt(int i, uint32_t tsz, uint64_t ttb, uint32_t granule_sz) "TT[%d]:tsz:%d ttb:0x%"PRIx64" granule_sz:%d" +smmuv3_cmdq_cfgi_ste(int streamid) " |_ streamid =%d" +smmuv3_cmdq_cfgi_ste_range(int start, int end) " |_ start=0x%d - end=0x%d" +smmuv3_cmdq_cfgi_cd(uint32_t sid) " |_ streamid = %d" +smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, float perc) "Config cache HIT for sid %d (hits=%d, misses=%d, hit rate=%.1f)" +smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, float perc) "Config cache MISS for sid %d (hits=%d, misses=%d, hit rate=%.1f)" +smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid %d"
include/hw/arm/smmu-common.h +5 -0
@@ -75,6 +75,8 @@ typedef struct SMMUDevice {int devfn;IOMMUMemoryRegion iommu;AddressSpace as; +uint32_t cfg_cache_hits; +uint32_t cfg_cache_misses; } SMMUDevice; typedef struct SMMUNotifierNode {
@@ -142,4 +144,7 @@ int smmu_ptw(SMMUTransCfg *cfg, dma_addr_t iova, IOMMUAccessFlags perm,*/ SMMUTransTableInfo *select_tt (SMMUTransCfg *cfg,dma_addr_t iova); +/* Return the iommu mr associated to @sid, or NULL if none */ +IOMMUMemoryRegion *smmu_iommu_mr (SMMUState *s,uint32_t sid); +#endif /* HW_ARM_SMMU_COMMON */
Signature · 1 line
2.5.5
PATCH v12 16/17 hw/arm/smmuv3: IOTLB emulation
In reply to Eric AugerWe emulate a TLB cache of size SMMU_IOTLB_MAX_SIZE=256. It is implemented as a hash table whose key is a combination of the 16b asid and 48b IOVA.
Entries are invalidated on TLB invalidation commands, either globally, or per asid, or per asid/iova.
---
v11 -> v12:
- Add new trace point when smmu is bypassed
- s/iotlb_miss/iotlb_misses, s/iotlb_hit/iotlb_hits
- use SMMUIOTLBKey as a key
Credit to Tomasz Nowicki who did the first implementation of this IOTLB implementation, inspired of intel_iommu implementation.
--- hw/arm/smmu-common.c | 59 ++++++++++++++++++++++++ hw/arm/smmuv3.c | 105 ++++++++++++++++++++++++++++++++++++++++--- hw/arm/trace-events | 10 +++++ include/hw/arm/smmu-common.h | 13 ++++++ 4 files changed, 181 insertions(+), 6 deletions(-)
hw/arm/smmu-common.c +59 -0
@@ -24,11 +24,43 @@#include "qom/cpu.h" #include "hw/qdev-properties.h" #include "qapi/error.h" +#include "qemu/jhash.h" #include "qemu/error-report.h" #include "hw/arm/smmu-common.h" #include "smmu-internal.h" +/* IOTLB Management */ + +inline void smmu_iotlb_inv_all (SMMUState *s) +{ +trace_smmu_iotlb_inv_all (); +g_hash_table_remove_all (s->iotlb); +} + +static gboolean smmu_hash_remove_by_asid (gpointer key,gpointer value, +gpointer user_data) +{ +uint16_t asid = *(uint16_t *)user_data; +SMMUIOTLBKey *iotlb_key = (SMMUIOTLBKey *)key; + +return iotlb_key->asid == asid; +} + +inline void smmu_iotlb_inv_iova (SMMUState *s,uint16_t asid,dma_addr_t iova) +{ +SMMUIOTLBKey key = {.asid = asid, .iova = iova}; + +trace_smmu_iotlb_inv_iova (asid, iova); +g_hash_table_remove (s->iotlb, &key); +} + +inline void smmu_iotlb_inv_asid (SMMUState *s,uint16_t asid) +{ +trace_smmu_iotlb_inv_asid (asid); +g_hash_table_foreach_remove (s->iotlb, smmu_hash_remove_by_asid, &asid); +} +/* VMSAv8-64 Translation */ /**
@@ -330,6 +362,30 @@ IOMMUMemoryRegion *smmu_iommu_mr(SMMUState *s, uint32_t sid)return NULL ; } +static guint smmu_iotlb_key_hash (gconstpointer v) +{ +SMMUIOTLBKey *key = (SMMUIOTLBKey *)v; +uint32_t a, b, c; + + a = b = c =JHASH_INITVAL +sizeof (*key); + a += key->asid; + b +=extract64 (key->iova,0 ,32 ); + c +=extract64 (key->iova,32 ,32 ); + +__jhash_mix (a, b, c); +__jhash_final (a, b, c); + +return c; +} + +static gboolean smmu_iotlb_key_equal (gconstpointer v1,gconstpointer v2) +{ +SMMUIOTLBKey *k1 = (SMMUIOTLBKey *)v1; +SMMUIOTLBKey *k2 = (SMMUIOTLBKey *)v2; + +return (k1->asid == k2->asid) && (k1->iova == k2->iova); +} +static void smmu_base_realize (DeviceState *dev,Error **errp) {SMMUState *s =ARM_SMMU (dev);
@@ -342,6 +398,8 @@ static void smmu_base_realize(DeviceState *dev, Error **errp)return ; } s->configs =g_hash_table_new_full (NULL ,NULL ,NULL , g_free); + s->iotlb =g_hash_table_new_full (smmu_iotlb_key_hash, smmu_iotlb_key_equal, + g_free, g_free); s->smmu_pcibus_by_busptr =g_hash_table_new (NULL ,NULL );if (s->primary_bus) {
@@ -356,6 +414,7 @@ static void smmu_base_reset(DeviceState *dev)SMMUState *s =ARM_SMMU (dev);g_hash_table_remove_all (s->configs); +g_hash_table_remove_all (s->iotlb); }static Property smmu_dev_properties[] = {
hw/arm/smmuv3.c +99 -6
154 lines not on this page · raw message
hw/arm/trace-events +10 -0
@@ -12,6 +12,11 @@ smmu_ptw_invalid_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, smmu_ptw_page_pte(int stage, int level, uint64_t iova, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) "stage=%d level=%d iova=0x%"PRIx64" base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" page address = 0x%"PRIx64 smmu_ptw_block_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t iova, uint64_t gpa, int bsize_mb) "stage=%d level=%d base@=0x%"PRIx64" pte@=0x%"PRIx64" pte=0x%"PRIx64" iova=0x%"PRIx64" block address = 0x%"PRIx64" block size = %d MiB" smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) "baseaddr=0x%"PRIx64" index=0x%x, pteaddr=0x%"PRIx64", pte=0x%"PRIx64 +smmu_iotlb_cache_hit(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, float p) "IOTLB cache HIT asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%.1f" +smmu_iotlb_cache_miss(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, float p) "IOTLB cache MISS asid=%d addr=0x%"PRIx64" hit=%d miss=%d hit rate=%.1f" +smmu_iotlb_inv_all(void) "IOTLB invalidate all" +smmu_iotlb_inv_asid(uint16_t asid) "IOTLB invalidate asid=%d" +smmu_iotlb_inv_iova(uint16_t asid, uint64_t addr) "IOTLB invalidate asid=%d addr=0x%"PRIx64 #hw/arm/smmuv3.c smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)"
@@ -37,10 +42,15 @@ smmuv3_translate_bypass(const char *n, uint16_t sid, uint64_t addr, bool is_writ smmuv3_translate_in(uint16_t sid, int pci_bus_num, uint64_t strtab_base) "SID:0x%x bus:%d strtab_base:0x%"PRIx64 smmuv3_get_cd(uint64_t addr) "CD addr: 0x%"PRIx64 smmuv3_translate(const char *n, uint16_t sid, uint64_t iova, uint64_t translated, int perm) "%s sid=%d iova=0x%"PRIx64" translated=0x%"PRIx64" perm=0x%x" +smmuv3_translate_pt(const char *n, uint16_t sid, uint64_t iova, int perm) "%s sid=%d iova=0x%"PRIx64" perm=0x%x PASSTHROUGH" smmuv3_decode_cd(uint32_t oas) "oas=%d" smmuv3_decode_cd_tt(int i, uint32_t tsz, uint64_t ttb, uint32_t granule_sz) "TT[%d]:tsz:%d ttb:0x%"PRIx64" granule_sz:%d" smmuv3_cmdq_cfgi_ste(int streamid) " |_ streamid =%d" smmuv3_cmdq_cfgi_ste_range(int start, int end) " |_ start=0x%d - end=0x%d" +smmuv3_cmdq_tlbi_nh_va(int vmid, int asid, uint64_t addr, bool leaf) " |_ vmid =%d asid =%d addr=0x%"PRIx64" leaf=%d" +smmuv3_cmdq_tlbi_nh_vaa(int vmid, uint64_t addr) " |_ vmid =%d addr=0x%"PRIx64 +smmuv3_cmdq_tlbi_nh(void) "" +smmuv3_cmdq_tlbi_nh_asid(uint16_t asid) "asid=%d" smmuv3_cmdq_cfgi_cd(uint32_t sid) " |_ streamid = %d" smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, float perc) "Config cache HIT for sid %d (hits=%d, misses=%d, hit rate=%.1f)" smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, float perc) "Config cache MISS for sid %d (hits=%d, misses=%d, hit rate=%.1f)"
include/hw/arm/smmu-common.h +13 -0
@@ -67,6 +67,8 @@ typedef struct SMMUTransCfg {uint8_t tbi;/* Top Byte Ignore */ uint16_t asid;SMMUTransTableInfo tt[2 ]; +uint32_t iotlb_hits;/* counts IOTLB hits for this asid */ +uint32_t iotlb_misses;/* counts IOTLB misses for this asid */ } SMMUTransCfg; typedef struct SMMUDevice {
@@ -89,6 +91,11 @@ typedef struct SMMUPciBus {SMMUDevice *pbdev[0 ];/* Parent array is sparse, so dynamically alloc */ } SMMUPciBus; +typedef struct SMMUIOTLBKey { +uint64_t iova; +uint16_t asid; +}SMMUIOTLBKey ; + typedef struct SMMUState { /* <private> */ SysBusDevice dev;
@@ -147,4 +154,10 @@ SMMUTransTableInfo *select_tt(SMMUTransCfg *cfg, dma_addr_t iova);/* Return the iommu mr associated to @sid, or NULL if none */ IOMMUMemoryRegion *smmu_iommu_mr (SMMUState *s,uint32_t sid); +#define SMMU_IOTLB_MAX_SIZE 256 + +void smmu_iotlb_inv_all (SMMUState *s); +void smmu_iotlb_inv_asid (SMMUState *s,uint16_t asid); +void smmu_iotlb_inv_iova (SMMUState *s,uint16_t asid,dma_addr_t iova); +#endif /* HW_ARM_SMMU_COMMON */
Signature · 1 line
2.5.5
PATCH v12 17/17 hw/arm/smmuv3: Add notifications on invalidation
In reply to Eric AugerOn TLB invalidation commands, let's call registered IOMMU notifiers. Those can only be UNMAP notifiers. SMMUv3 does not support notification on MAP (VFIO).
This patch allows vhost use case where IOTLB API is notified on each guest IOTLB invalidation.
--- hw/arm/smmu-common.c | 34 +++++++++++++++ hw/arm/smmuv3.c | 99 +++++++++++++++++++++++++++++++++++++++++++- hw/arm/trace-events | 5 +++ include/hw/arm/smmu-common.h | 6 +++ 4 files changed, 142 insertions(+), 2 deletions(-)
hw/arm/smmu-common.c +34 -0
@@ -386,6 +386,40 @@ static gboolean smmu_iotlb_key_equal(gconstpointer v1, gconstpointer v2)return (k1->asid == k2->asid) && (k1->iova == k2->iova); } +/* Unmap the whole notifier's range */ +static void smmu_unmap_notifier_range (IOMMUNotifier *n) +{ +IOMMUTLBEntry entry; + + entry.target_as = &address_space_memory; + entry.iova = n->start; + entry.perm =IOMMU_NONE ; + entry.addr_mask = n->end - n->start; + +memory_region_notify_one (n, &entry); +} + +/* Unmap all notifiers attached to @mr */ +inline void smmu_inv_notifiers_mr (IOMMUMemoryRegion *mr) +{ +IOMMUNotifier *n; + +trace_smmu_inv_notifiers_mr (mr->parent_obj.name); + IOMMU_NOTIFIER_FOREACH(n, mr) { +smmu_unmap_notifier_range (n); + } +} + +/* Unmap all notifiers of all mr's */ +void smmu_inv_notifiers_all (SMMUState *s) +{ +SMMUNotifierNode *node; + + QLIST_FOREACH(node, &s->notifiers_list, next) { +smmu_inv_notifiers_mr (&node->sdev->iommu); + } +} +static void smmu_base_realize (DeviceState *dev,Error **errp) {SMMUState *s =ARM_SMMU (dev);
hw/arm/smmuv3.c +97 -2
136 lines not on this page · raw message
hw/arm/trace-events +5 -0
@@ -17,6 +17,7 @@ smmu_iotlb_cache_miss(uint16_t asid, uint64_t addr, uint32_t hit, uint32_t miss, smmu_iotlb_inv_all(void) "IOTLB invalidate all" smmu_iotlb_inv_asid(uint16_t asid) "IOTLB invalidate asid=%d" smmu_iotlb_inv_iova(uint16_t asid, uint64_t addr) "IOTLB invalidate asid=%d addr=0x%"PRIx64 +smmu_inv_notifiers_mr(const char *name) "iommu mr=%s" #hw/arm/smmuv3.c smmuv3_read_mmio(uint64_t addr, uint64_t val, unsigned size, uint32_t r) "addr: 0x%"PRIx64" val:0x%"PRIx64" size: 0x%x(%d)"
@@ -55,3 +56,7 @@ smmuv3_cmdq_cfgi_cd(uint32_t sid) " |_ streamid = %d" smmuv3_config_cache_hit(uint32_t sid, uint32_t hits, uint32_t misses, float perc) "Config cache HIT for sid %d (hits=%d, misses=%d, hit rate=%.1f)" smmuv3_config_cache_miss(uint32_t sid, uint32_t hits, uint32_t misses, float perc) "Config cache MISS for sid %d (hits=%d, misses=%d, hit rate=%.1f)" smmuv3_config_cache_inv(uint32_t sid) "Config cache INV for sid %d" +smmuv3_notify_flag_add(const char *iommu) "ADD SMMUNotifier node for iommu mr=%s" +smmuv3_notify_flag_del(const char *iommu) "DEL SMMUNotifier node for iommu mr=%s" +smmuv3_inv_notifiers_iova(const char *name, uint16_t asid, uint64_t iova) "iommu mr=%s asid=%d iova=0x%"PRIx64 +
include/hw/arm/smmu-common.h +6 -0
@@ -160,4 +160,10 @@ void smmu_iotlb_inv_all(SMMUState *s);void smmu_iotlb_inv_asid (SMMUState *s,uint16_t asid);void smmu_iotlb_inv_iova (SMMUState *s,uint16_t asid,dma_addr_t iova); +/* Unmap the range of all the notifiers registered to any IOMMU mr */ +void smmu_inv_notifiers_all (SMMUState *s); + +/* Unmap the range of all the notifiers registered to @mr */ +void smmu_inv_notifiers_mr (IOMMUMemoryRegion *mr); +#endif /* HW_ARM_SMMU_COMMON */
Signature · 1 line
2.5.5
Quoted 5 lines · Eric Auger
This series implements the emulation code for ARM SMMUv3.
SMMUv3 gets instantiated by adding ",iommu=smmuv3" to the virt machine option.
Quoted 8 lines
Only stage 1 and AArch64 PTW are supported. [1 - 14] bring the core emulation code. [15, 16] bring optimizations and 17 brings the vhost integration.
Important Note:
I added the last 3 patches for consistency but I would be happy if [1 - 14] could land upstream first, without waiting for the last 3 patch additional review cycles.
I have applied 1-14 to target-arm.next; haven't looked at 15-17, so I suggest you repost those as their own series once 1-14 are in master.
thanks
-- PMM
Hi Peter,
Quoted 10 lines · Peter Maydell
Quoted 9 lines · Eric Auger
This series implements the emulation code for ARM SMMUv3.
SMMUv3 gets instantiated by adding ",iommu=smmuv3" to the virt machine option.
Only stage 1 and AArch64 PTW are supported. [1 - 14] bring the core emulation code. [15, 16] bring optimizations and 17 brings the vhost integration.
Quoted 8 lines
Quoted 4 lines
Important Note:
I added the last 3 patches for consistency but I would be happy if [1 - 14] could land upstream first, without waiting for the last 3 patch additional review cycles.I have applied 1-14 to target-arm.next; haven't looked at 15-17, so I suggest you repost those as their own series once 1-14 are in master.
Sure.
Thanks
Eric
Quoted 4 lines
thanks
-- PMM