System Cache support for GPU and required SMMU support
Subsystems (6)
v3 cover System Cache support for GPU and required SMMU support
Some hardware variants contain a system cache or the last level cache(llc). This cache is typically a large block which is shared by multiple clients on the SOC. GPU uses the system cache to cache both the GPU data buffers(like textures) as well the SMMU pagetables. This helps with improved render performance as well as lower power consumption by reducing the bus traffic to the system memory.
The system cache architecture allows the cache to be split into slices which then be used by multiple SOC clients. This patch series is an effort to enable and use two of those slices perallocated for the GPU, one for the GPU data buffers and another for the GPU SMMU hardware pagetables.
Patch 1 adds a init_context_bank implementation hook to set SCTLR.HUPCF. Patch 2,3,6,7 adds system cache support in SMMU and GPU driver. Patch 4 and 5 are minor cleanups for arm-smmu impl.
Changes in v3:
* Fix domain attribute setting to before iommu_attach_device()
* Fix few code style and checkpatch warnings
* Rebase on top of Jordan's latest split pagetables and per-instance pagetables support [1][2]
Changes in v2:
* Addressed review comments and rebased on top of Jordan's split pagetables series
[1] https://lore.kernel.org/patchwork/cover/1264446/
[2] https://lore.kernel.org/patchwork/cover/1264460/
Jordan Crouse (1):
iommu/arm-smmu: Add a init_context_bank implementation hook
Sai Prakash Ranjan (4):
iommu/io-pgtable-arm: Add support to use system cache iommu/arm-smmu: Add domain attribute for system cache
iommu: arm-smmu-impl: Remove unwanted extra blank lines
iommu: arm-smmu-impl: Convert to use of_match_node() for qcom impl
Sharat Masetty (2):
drm/msm: rearrange the gpu_rmw() function drm/msm/a6xx: Add support for using system cache(LLC)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 82 +++++++++++++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 3 + drivers/gpu/drm/msm/adreno/adreno_gpu.c | 23 ++++++- drivers/gpu/drm/msm/msm_drv.c | 8 +++ drivers/gpu/drm/msm/msm_drv.h | 1 + drivers/gpu/drm/msm/msm_gpu.h | 5 +- drivers/gpu/drm/msm/msm_iommu.c | 3 + drivers/gpu/drm/msm/msm_mmu.h | 4 ++ drivers/iommu/arm-smmu-impl.c | 13 ++-- drivers/iommu/arm-smmu-qcom.c | 13 ++++ drivers/iommu/arm-smmu.c | 46 +++++++++----- drivers/iommu/arm-smmu.h | 13 ++++ drivers/iommu/io-pgtable-arm.c | 7 ++- include/linux/io-pgtable.h | 4 ++ include/linux/iommu.h | 1 + 15 files changed, 198 insertions(+), 28 deletions(-)
Signature · 7 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v3 1/7 iommu/arm-smmu: Add a init_context_bank implementation hook
From: Jordan Crouse <[email protected]>
Add a new implementation hook to allow the implementation specific code to tweek the context bank configuration just before it gets written. The first user will be the Adreno GPU implementation to turn on SCTLR.HUPCF to ensure that a page fault doesn't terminating pending transactions. Doing so could hang the GPU if one of the terminated transactions is a CP read.
This depends on the arm-smmu adreno SMMU implementation [1].
[1] https://lore.kernel.org/patchwork/patch/1264452/
--- drivers/iommu/arm-smmu-qcom.c | 13 +++++++++++++ drivers/iommu/arm-smmu.c | 29 +++++++++++++---------------- drivers/iommu/arm-smmu.h | 12 ++++++++++++ 3 files changed, 38 insertions(+), 16 deletions(-)
drivers/iommu/arm-smmu-qcom.c +13 -0
@@ -17,6 +17,18 @@ static bool qcom_adreno_smmu_is_gpu_device(struct arm_smmu_domain *smmu_domain)return of_device_is_compatible (smmu_domain->dev->of_node,"qcom,adreno" ); } +static void qcom_adreno_smmu_init_context_bank (struct arm_smmu_domain *smmu_domain, +struct arm_smmu_cb *cb) +{ +/* +* On the GPU device we want to process subsequent transactions after a +* fault to keep the GPU from hanging +*/ + +if (qcom_adreno_smmu_is_gpu_device (smmu_domain)) + cb->sctlr |=ARM_SMMU_SCTLR_HUPCF ; +} +static int qcom_adreno_smmu_init_context (struct arm_smmu_domain *smmu_domain,struct io_pgtable_cfg *pgtbl_cfg) {
@@ -92,6 +104,7 @@ static const struct arm_smmu_impl qcom_adreno_smmu_impl = { .init_context = qcom_adreno_smmu_init_context, .def_domain_type = qcom_smmu_def_domain_type, .reset = qcom_smmu500_reset, + .init_context_bank = qcom_adreno_smmu_init_context_bank, };
drivers/iommu/arm-smmu.c +13 -16
@@ -86,14 +86,6 @@ struct arm_smmu_smr {bool valid; }; -struct arm_smmu_cb { -u64 ttbr[2 ]; -u32 tcr[2 ]; -u32 mair[2 ]; -struct arm_smmu_cfg *cfg; -atomic_t aux; -}; -struct arm_smmu_master_cfg {struct arm_smmu_device *smmu;s16 smendx[];
@@ -580,6 +572,18 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain, cb->mair[1 ] = pgtbl_cfg->arm_lpae_s1_cfg.mair >>32 ; } } + + cb->sctlr =ARM_SMMU_SCTLR_CFIE |ARM_SMMU_SCTLR_CFRE |ARM_SMMU_SCTLR_AFE | +ARM_SMMU_SCTLR_TRE |ARM_SMMU_SCTLR_M ; + +if (stage1) + cb->sctlr |=ARM_SMMU_SCTLR_S1_ASIDPNE ; +if (IS_ENABLED (CONFIG_CPU_BIG_ENDIAN )) + cb->sctlr |=ARM_SMMU_SCTLR_E ; + +/* Give the implementation a chance to adjust the configuration */ +if (smmu_domain->smmu->impl && smmu_domain->smmu->impl->init_context_bank) + smmu_domain->smmu->impl->init_context_bank (smmu_domain, cb); }static void arm_smmu_write_context_bank (struct arm_smmu_device *smmu,int idx)
@@ -658,14 +662,7 @@ static void arm_smmu_write_context_bank(struct arm_smmu_device *smmu, int idx) }/* SCTLR */ - reg =ARM_SMMU_SCTLR_CFIE |ARM_SMMU_SCTLR_CFRE |ARM_SMMU_SCTLR_AFE | -ARM_SMMU_SCTLR_TRE |ARM_SMMU_SCTLR_M ; -if (stage1) - reg |=ARM_SMMU_SCTLR_S1_ASIDPNE ; -if (IS_ENABLED (CONFIG_CPU_BIG_ENDIAN )) - reg |=ARM_SMMU_SCTLR_E ; - -arm_smmu_cb_write (smmu, idx,ARM_SMMU_CB_SCTLR , reg); +arm_smmu_cb_write (smmu, idx,ARM_SMMU_CB_SCTLR , cb->sctlr); }/*
drivers/iommu/arm-smmu.h +12 -0
@@ -142,6 +142,7 @@ enum arm_smmu_cbar_type {#define ARM_SMMU_CB_SCTLR 0x0#define ARM_SMMU_SCTLR_S1_ASIDPNE BIT(12) +#define ARM_SMMU_SCTLR_HUPCF BIT(8)#define ARM_SMMU_SCTLR_CFCFG BIT(7)#define ARM_SMMU_SCTLR_CFIE BIT(6)#define ARM_SMMU_SCTLR_CFRE BIT(5)
@@ -349,6 +350,15 @@ struct arm_smmu_domain {bool aux; }; +struct arm_smmu_cb { +u64 ttbr[2 ]; +u32 tcr[2 ]; +u32 mair[2 ]; +u32 sctlr; +struct arm_smmu_cfg *cfg; +atomic_t aux; +}; +static inline u32 arm_smmu_lpae_tcr (struct io_pgtable_cfg *cfg) { u32 tcr = FIELD_PREP(ARM_SMMU_TCR_TG0, cfg->arm_lpae_s1_cfg.tcr.tg) |
@@ -403,6 +413,8 @@ struct arm_smmu_impl {void (*tlb_sync)(struct arm_smmu_device *smmu,int page,int sync,int status);int (*def_domain_type)(struct device *dev); +void (*init_context_bank)(struct arm_smmu_domain *smmu_domain, +struct arm_smmu_cb *cb); };static inline void __iomem *arm_smmu_page (struct arm_smmu_device *smmu,int n)
Signature · 7 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v3 2/7 iommu/io-pgtable-arm: Add support to use system cache
In reply to Sai Prakash RanjanAdd a quirk IO_PGTABLE_QUIRK_SYS_CACHE to override the attributes set in TCR for the page table walker when using system cache.
--- drivers/iommu/io-pgtable-arm.c | 7 ++++++- include/linux/io-pgtable.h | 4 ++++ 2 files changed, 10 insertions(+), 1 deletion(-)
drivers/iommu/io-pgtable-arm.c +6 -1
@@ -792,7 +792,8 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie)if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS |IO_PGTABLE_QUIRK_NON_STRICT | -IO_PGTABLE_QUIRK_ARM_TTBR1 )) +IO_PGTABLE_QUIRK_ARM_TTBR1 | +IO_PGTABLE_QUIRK_SYS_CACHE ))return NULL ; data =arm_lpae_alloc_pgtable (cfg);
@@ -804,6 +805,10 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) tcr->sh =ARM_LPAE_TCR_SH_IS ; tcr->irgn =ARM_LPAE_TCR_RGN_WBWA ; tcr->orgn =ARM_LPAE_TCR_RGN_WBWA ; + }else if (cfg->quirks &IO_PGTABLE_QUIRK_SYS_CACHE ) { + tcr->sh =ARM_LPAE_TCR_SH_OS ; + tcr->irgn =ARM_LPAE_TCR_RGN_NC ; + tcr->orgn =ARM_LPAE_TCR_RGN_WBWA ; }else { tcr->sh =ARM_LPAE_TCR_SH_OS ; tcr->irgn =ARM_LPAE_TCR_RGN_NC ;
include/linux/io-pgtable.h +4 -0
@@ -86,6 +86,9 @@ struct io_pgtable_cfg {* * IO_PGTABLE_QUIRK_ARM_TTBR1: (ARM LPAE format) Configure the table * for use in the upper half of a split address space. +* +* IO_PGTABLE_QUIRK_SYS_CACHE: Override the attributes set in TCR for +* the page table walker when using system cache. */ #define IO_PGTABLE_QUIRK_ARM_NS BIT(0)#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)
@@ -93,6 +96,7 @@ struct io_pgtable_cfg {#define IO_PGTABLE_QUIRK_ARM_MTK_EXT BIT(3)#define IO_PGTABLE_QUIRK_NON_STRICT BIT(4)#define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5) +#define IO_PGTABLE_QUIRK_SYS_CACHE BIT(6)unsigned long quirks;unsigned long pgsize_bitmap;unsigned int ias;
Signature · 7 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v3 3/7 iommu/arm-smmu: Add domain attribute for system cache
In reply to Sai Prakash RanjanAdd iommu domain attribute for using system cache aka last level cache by client drivers like GPU to set right attributes for caching the hardware pagetables into the system cache.
--- drivers/iommu/arm-smmu.c | 17 +++++++++++++++++ drivers/iommu/arm-smmu.h | 1 + include/linux/iommu.h | 1 + 3 files changed, 19 insertions(+)
drivers/iommu/arm-smmu.c +17 -0
@@ -897,6 +897,9 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, goto out_unlock; } +if (smmu_domain->sys_cache) + pgtbl_cfg.quirks |=IO_PGTABLE_QUIRK_SYS_CACHE ; +if (smmu_domain->non_strict) pgtbl_cfg.quirks |=IO_PGTABLE_QUIRK_NON_STRICT ;
@@ -1732,6 +1735,9 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain,case DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE : *(int *)data = smmu_domain->non_strict;return 0 ; +case DOMAIN_ATTR_SYS_CACHE : + *((int *)data) = smmu_domain->sys_cache; +return 0 ;default :return -ENODEV ; }
@@ -1763,6 +1769,17 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain,else smmu_domain->stage =ARM_SMMU_DOMAIN_S1 ;break ; +case DOMAIN_ATTR_SYS_CACHE : +if (smmu_domain->smmu) { + ret = -EPERM ; + goto out_unlock; + } + +if (*((int *)data)) + smmu_domain->sys_cache = true; +else + smmu_domain->sys_cache = false; +break ;default : ret = -ENODEV ; }
drivers/iommu/arm-smmu.h +1 -0
@@ -348,6 +348,7 @@ struct arm_smmu_domain {struct iommu_domain domain;struct device *dev;/* Device attached to this domain */ bool aux; +bool sys_cache; };struct arm_smmu_cb {
include/linux/iommu.h +1 -0
@@ -125,6 +125,7 @@ enum iommu_attr {DOMAIN_ATTR_NESTING ,/* two stages of translation */ DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE ,DOMAIN_ATTR_PGTABLE_CFG , +DOMAIN_ATTR_SYS_CACHE , DOMAIN_ATTR_MAX, };
Signature · 7 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v3 4/7 iommu: arm-smmu-impl: Remove unwanted extra blank lines
In reply to Sai Prakash RanjanThere are few places in arm-smmu-impl where there are extra blank lines, remove them and while at it fix the checkpatch warning for space required before the open parenthesis.
--- drivers/iommu/arm-smmu-impl.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
drivers/iommu/arm-smmu-impl.c +1 -5
@@ -9,10 +9,9 @@#include "arm-smmu.h" -static int arm_smmu_gr0_ns (int offset) { -switch (offset) { +switch (offset) {case ARM_SMMU_GR0_sCR0:case ARM_SMMU_GR0_sACR:case ARM_SMMU_GR0_sGFSR:
@@ -47,7 +46,6 @@ static const struct arm_smmu_impl calxeda_impl = { .write_reg = arm_smmu_write_ns, }; -struct cavium_smmu {struct arm_smmu_device smmu;u32 id_base;
@@ -103,7 +101,6 @@ static struct arm_smmu_device *cavium_smmu_impl_init(struct arm_smmu_device *smmreturn &cs->smmu; } -#define ARM_MMU500_ACTLR_CPRE (1 << 1)#define ARM_MMU500_ACR_CACHE_LOCK (1 << 26)
@@ -148,7 +145,6 @@ static const struct arm_smmu_impl arm_mmu500_impl = { .reset = arm_mmu500_reset, }; -struct arm_smmu_device *arm_smmu_impl_init (struct arm_smmu_device *smmu) {const struct device_node *np = smmu->dev->of_node;
Signature · 7 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v3 5/7 iommu: arm-smmu-impl: Convert to use of_match_node() for qcom impl
In reply to Sai Prakash RanjanUse of_match_node() to match qcom implementation instead of multiple of_device_compatible() calls for each qcom implementation.
--- drivers/iommu/arm-smmu-impl.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/iommu/arm-smmu-impl.c +7 -2
@@ -9,6 +9,12 @@#include "arm-smmu.h" +static const struct of_device_id __maybe_unused qcom_smmu_impl_of_match[] = { + { .compatible ="qcom,sc7180-smmu-500" }, + { .compatible ="qcom,sdm845-smmu-500" }, + { } +}; +static int arm_smmu_gr0_ns (int offset) {switch (offset) {
@@ -168,8 +174,7 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)if (of_property_read_bool (np,"calxeda,smmu-secure-config-access" )) smmu->impl = &calxeda_impl; -if (of_device_is_compatible (np,"qcom,sdm845-smmu-500" ) || -of_device_is_compatible (np,"qcom,sc7180-smmu-500" )) +if (of_match_node (qcom_smmu_impl_of_match, np))return qcom_smmu_impl_init (smmu);if (of_device_is_compatible (smmu->dev->of_node,"qcom,adreno-smmu" ))
Signature · 7 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v3 6/7 drm/msm: rearrange the gpu_rmw() function
In reply to Sai Prakash RanjanFrom: Sharat Masetty <[email protected]>
The register read-modify-write construct is generic enough that it can be used by other subsystems as needed, create a more generic rmw() function and have the gpu_rmw() use this new function.
--- drivers/gpu/drm/msm/msm_drv.c | 8 ++++++++ drivers/gpu/drm/msm/msm_drv.h | 1 + drivers/gpu/drm/msm/msm_gpu.h | 5 +---- 3 files changed, 10 insertions(+), 4 deletions(-)
drivers/gpu/drm/msm/msm_drv.c +8 -0
@@ -166,6 +166,14 @@ u32 msm_readl(const void __iomem *addr)return val; } +void msm_rmw (void __iomem *addr,u32 mask,u32 or) +{ +u32 val =msm_readl (addr); + + val &= ~mask; +msm_writel (val | or, addr); +} +struct msm_vblank_work {struct work_struct work;int crtc_id;
drivers/gpu/drm/msm/msm_drv.h +1 -0
@@ -417,6 +417,7 @@ void __iomem *msm_ioremap(struct platform_device *pdev, const char *name, const char *dbgname);void msm_writel (u32 data,void __iomem *addr);u32 msm_readl (const void __iomem *addr); +void msm_rmw (void __iomem *addr,u32 mask,u32 or);struct msm_gpu_submitqueue ;int msm_submitqueue_init (struct drm_device *drm,struct msm_file_private *ctx);
drivers/gpu/drm/msm/msm_gpu.h +1 -4
@@ -232,10 +232,7 @@ static inline u32 gpu_read(struct msm_gpu *gpu, u32 reg)static inline void gpu_rmw (struct msm_gpu *gpu,u32 reg,u32 mask,u32 or) { -uint32_t val =gpu_read (gpu, reg); - - val &= ~mask; -gpu_write (gpu, reg, val | or); +msm_rmw (gpu->mmio + (reg <<2 ), mask, or); }static inline u64 gpu_read64 (struct msm_gpu *gpu,u32 lo,u32 hi)
Signature · 7 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v3 7/7 drm/msm/a6xx: Add support for using system cache(LLC)
In reply to Sai Prakash RanjanFrom: Sharat Masetty <[email protected]>
The last level system cache can be partitioned to 32 different slices of which GPU has two slices preallocated. One slice is used for caching GPU buffers and the other slice is used for caching the GPU SMMU pagetables. This talks to the core system cache driver to acquire the slice handles, configure the SCID's to those slices and activates and deactivates the slices upon GPU power collapse and restore.
Some support from the IOMMU driver is also needed to make use of the system cache. IOMMU_SYS_CACHE_ONLY is a buffer protection flag which enables caching GPU data buffers in the system cache with memory attributes such as outer cacheable, read-allocate, write-allocate for buffers. The GPU then has the ability to override a few cacheability parameters which it does to override write-allocate to write-no-allocate as the GPU hardware does not benefit much from it.
Similarly DOMAIN_ATTR_SYS_CACHE is another domain level attribute used by the IOMMU driver to set the right attributes to cache the hardware pagetables into the system cache.
(sai: fix to set attr before device attach to IOMMU and rebase)
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 82 +++++++++++++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 3 + drivers/gpu/drm/msm/adreno/adreno_gpu.c | 23 ++++++- drivers/gpu/drm/msm/msm_iommu.c | 3 + drivers/gpu/drm/msm/msm_mmu.h | 4 ++ 5 files changed, 114 insertions(+), 1 deletion(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +82 -0
@@ -9,6 +9,8 @@#include "a6xx_gmu.xml.h" #include <linux/devfreq.h> +#include <linux/bitfield.h> +#include <linux/soc/qcom/llcc-qcom.h> #define GPU_PAS_ID 13
@@ -808,6 +810,79 @@ static const u32 a6xx_register_offsets[REG_ADRENO_REGISTER_MAX] = { REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_CNTL, REG_A6XX_CP_RB_CNTL), }; +static void a6xx_llc_rmw (struct a6xx_gpu *a6xx_gpu,u32 reg,u32 mask,u32 or) +{ +return msm_rmw (a6xx_gpu->llc_mmio + (reg <<2 ), mask, or); +} + +static void a6xx_llc_write (struct a6xx_gpu *a6xx_gpu,u32 reg,u32 value) +{ +return msm_writel (value, a6xx_gpu->llc_mmio + (reg <<2 )); +} + +static void a6xx_llc_deactivate (struct a6xx_gpu *a6xx_gpu) +{ +llcc_slice_deactivate (a6xx_gpu->llc_slice); +llcc_slice_deactivate (a6xx_gpu->htw_llc_slice); +} + +static void a6xx_llc_activate (struct a6xx_gpu *a6xx_gpu) +{ +u32 cntl1_regval =0 ; + +if (IS_ERR (a6xx_gpu->llc_mmio)) +return ; + +if (!llcc_slice_activate (a6xx_gpu->llc_slice)) { +u32 gpu_scid =llcc_get_slice_id (a6xx_gpu->llc_slice); + + gpu_scid &=0x1f ; + cntl1_regval = (gpu_scid <<0 ) | (gpu_scid <<5 ) | (gpu_scid <<10 ) | + (gpu_scid <<15 ) | (gpu_scid <<20 ); + } + +if (!llcc_slice_activate (a6xx_gpu->htw_llc_slice)) { +u32 gpuhtw_scid =llcc_get_slice_id (a6xx_gpu->htw_llc_slice); + + gpuhtw_scid &=0x1f ; + cntl1_regval |=FIELD_PREP (GENMASK (29 ,25 ), gpuhtw_scid); + } + +if (cntl1_regval) { +/* +* Program the slice IDs for the various GPU blocks and GPU MMU +* pagetables +*/ +a6xx_llc_write (a6xx_gpu,REG_A6XX_CX_MISC_SYSTEM_CACHE_CNTL_1 , cntl1_regval); + +/* +* Program cacheability overrides to not allocate cache lines on +* a write miss +*/ +a6xx_llc_rmw (a6xx_gpu,REG_A6XX_CX_MISC_SYSTEM_CACHE_CNTL_0 ,0xF ,0x03 ); + } +} + +static void a6xx_llc_slices_destroy (struct a6xx_gpu *a6xx_gpu) +{ +llcc_slice_putd (a6xx_gpu->llc_slice); +llcc_slice_putd (a6xx_gpu->htw_llc_slice); +} + +static void a6xx_llc_slices_init (struct platform_device *pdev, +struct a6xx_gpu *a6xx_gpu) +{ + a6xx_gpu->llc_mmio =msm_ioremap (pdev,"cx_mem" ,"gpu_cx" ); +if (IS_ERR (a6xx_gpu->llc_mmio)) +return ; + + a6xx_gpu->llc_slice =llcc_slice_getd (LLCC_GPU ); + a6xx_gpu->htw_llc_slice =llcc_slice_getd (LLCC_GPUHTW ); + +if (IS_ERR (a6xx_gpu->llc_slice) &&IS_ERR (a6xx_gpu->htw_llc_slice)) + a6xx_gpu->llc_mmio =ERR_PTR (-EINVAL ); +} +static int a6xx_pm_resume (struct msm_gpu *gpu) {struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);
@@ -822,6 +897,8 @@ static int a6xx_pm_resume(struct msm_gpu *gpu)msm_gpu_resume_devfreq (gpu); +a6xx_llc_activate (a6xx_gpu); +return 0 ; }
@@ -830,6 +907,8 @@ static int a6xx_pm_suspend(struct msm_gpu *gpu)struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); +a6xx_llc_deactivate (a6xx_gpu); +devfreq_suspend_device (gpu->devfreq.devfreq);return a6xx_gmu_stop (a6xx_gpu);
@@ -868,6 +947,7 @@ static void a6xx_destroy(struct msm_gpu *gpu)drm_gem_object_put_unlocked (a6xx_gpu->sqe_bo); } +a6xx_llc_slices_destroy (a6xx_gpu);a6xx_gmu_remove (a6xx_gpu);adreno_gpu_cleanup (adreno_gpu);
@@ -962,6 +1042,8 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) adreno_gpu->registers =NULL ; adreno_gpu->reg_offsets = a6xx_register_offsets; +a6xx_llc_slices_init (pdev, a6xx_gpu); + ret =adreno_gpu_init (dev, pdev, adreno_gpu, &funcs,1 );if (ret) {a6xx_destroy (&(a6xx_gpu->base.base));
drivers/gpu/drm/msm/adreno/a6xx_gpu.h +3 -0
@@ -21,6 +21,9 @@ struct a6xx_gpu {struct msm_ringbuffer *cur_ring;struct a6xx_gmu gmu; +void __iomem *llc_mmio; +void *llc_slice; +void *htw_llc_slice; };#define to_a6xx_gpu (x) container_of(x, struct a6xx_gpu, base)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +22 -1
@@ -190,10 +190,31 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu, struct platform_device *pdev) {struct iommu_domain *iommu =iommu_domain_alloc (&platform_bus_type); -struct msm_mmu *mmu =msm_iommu_new (&pdev->dev, iommu); +struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu); +struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu);struct msm_gem_address_space *aspace; +struct msm_mmu *mmu;u64 start, size; +/* +* This allows GPU to set the bus attributes required to use system +* cache on behalf of the iommu page table walker. +*/ +if (!IS_ERR (a6xx_gpu->htw_llc_slice)) { +int gpu_htw_llc =1 ; + +iommu_domain_set_attr (iommu,DOMAIN_ATTR_SYS_CACHE , &gpu_htw_llc); + } + + mmu =msm_iommu_new (&pdev->dev, iommu); +if (IS_ERR (mmu)) { +iommu_domain_free (iommu); +return ERR_CAST (mmu); + } + +if (!IS_ERR (a6xx_gpu->llc_slice)) + mmu->features |=MMU_FEATURE_USE_SYSTEM_CACHE ; +/* * Use the aperture start or SZ_16M, whichever is greater. This will * ensure that we align with the allocated pagetable range while still
drivers/gpu/drm/msm/msm_iommu.c +3 -0
@@ -218,6 +218,9 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, iova |=GENMASK_ULL (63 ,49 ); +if (mmu->features &MMU_FEATURE_USE_SYSTEM_CACHE ) + prot |=IOMMU_SYS_CACHE_ONLY ; + ret =iommu_map_sg (iommu->domain, iova, sgt->sgl, sgt->nents, prot);WARN_ON (!ret);
drivers/gpu/drm/msm/msm_mmu.h +4 -0
@@ -23,12 +23,16 @@ enum msm_mmu_type { MSM_MMU_IOMMU_PAGETABLE, }; +/* MMU features */ +#define MMU_FEATURE_USE_SYSTEM_CACHE BIT(0) +struct msm_mmu {const struct msm_mmu_funcs *funcs;struct device *dev;int (*handler)(void *arg,unsigned long iova,int flags);void *arg;enum msm_mmu_type type; +u32 features; }; static inline void msm_mmu_init(struct msm_mmu *mmu, struct device *dev,
Signature · 7 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Will Deacon
maintainerRe: [PATCHv3 7/7] drm/msm/a6xx: Add support for using system cache(LLC)
drivers/gpu/drm/msm/msm_iommu.c static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, · line 222
Quoted 10 lines · Sai Prakash Ranjan
drivers/gpu/drm/msm/msm_iommu.c +2
-0@@ -218,6 +218,9 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, iova |=GENMASK_ULL (63 ,49 ); +if (mmu->features &MMU_FEATURE_USE_SYSTEM_CACHE ) + prot |=IOMMU_SYS_CACHE_ONLY ;
Given that I think this is the only user of IOMMU_SYS_CACHE_ONLY, then it looks like it should actually be a property on the domain because we never need to configure it on a per-mapping basis within a domain, and therefore it shouldn't be exposed by the IOMMU API as a prot flag.
Do you agree?
Will
Re: [PATCHv3 7/7] drm/msm/a6xx: Add support for using system cache(LLC)
Hi Will,
Quoted 18 lines · Will Deacon
drivers/gpu/drm/msm/msm_iommu.c static int msm_iommu_map(struct msm_mmu *mmu, · line 223
Quoted 11 lines · Sai Prakash Ranjan
drivers/gpu/drm/msm/msm_iommu.c +2
-0@@ -218,6 +218,9 @@ static int msm_iommu_map(struct msm_mmu *mmu,uint64_t iova, iova |= GENMASK_ULL(63, 49); +if (mmu->features &MMU_FEATURE_USE_SYSTEM_CACHE ) + prot |=IOMMU_SYS_CACHE_ONLY ;Given that I think this is the only user of
IOMMU_SYS_CACHE_ONLY, then it looks like it should actually be a property on the domain because we never need to configure it on a per-mapping basis within a domain, and therefore it shouldn't be exposed by the IOMMU API as a prot flag.
Quoted 2 lines
Do you agree?
GPU being the only user is for now, but there are other clients which can use this. Plus how do we set the memory attributes if we do not expose this as prot flag?
Thanks,
Sai
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Will Deacon
maintainerRe: [PATCHv3 7/7] drm/msm/a6xx: Add support for using system cache(LLC)
Quoted 23 lines · Sai Prakash Ranjan
Quoted 22 lines · Will Deacon
drivers/gpu/drm/msm/msm_iommu.c static int msm_iommu_map(struct msm_mmu *mmu, · line 223
Quoted 12 lines · Sai Prakash Ranjan
drivers/gpu/drm/msm/msm_iommu.c +2
-0@@ -218,6 +218,9 @@ static int msm_iommu_map(struct msm_mmu *mmu,uint64_t iova, iova |= GENMASK_ULL(63, 49); +if (mmu->features &MMU_FEATURE_USE_SYSTEM_CACHE ) + prot |=IOMMU_SYS_CACHE_ONLY ;Given that I think this is the only user of
IOMMU_SYS_CACHE_ONLY, then it
looks like it should actually be a property on the domain because we never
need to configure it on a per-mapping basis within a domain, and therefore
it shouldn't be exposed by the IOMMU API as a prot flag.
Quoted 7 lines
Quoted 2 lines
Do you agree?
GPU being the only user is for now, but there are other clients which can use this.
Plus how do we set the memory attributes if we do not expose this as prot flag?
I just don't understand the need for it to be per-map operation. Put another way, if we extended the domain attribute to apply to cacheable mappings on the domain and not just the table walk, what would break?
Will
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCHv3 7/7] drm/msm/a6xx: Add support for using system cache(LLC)
In reply to Sai Prakash RanjanQuoted 26 lines · Sai Prakash Ranjan
Hi Will,
Quoted 22 lines · Will Deacon
drivers/gpu/drm/msm/msm_iommu.c static int msm_iommu_map(struct msm_mmu *mmu, · line 223
Quoted 12 lines · Sai Prakash Ranjan
drivers/gpu/drm/msm/msm_iommu.c +2
-0@@ -218,6 +218,9 @@ static int msm_iommu_map(struct msm_mmu *mmu,uint64_t iova, iova |= GENMASK_ULL(63, 49); +if (mmu->features &MMU_FEATURE_USE_SYSTEM_CACHE ) + prot |=IOMMU_SYS_CACHE_ONLY ;Given that I think this is the only user of
IOMMU_SYS_CACHE_ONLY, then it
looks like it should actually be a property on the domain because we never
need to configure it on a per-mapping basis within a domain, and therefore
it shouldn't be exposed by the IOMMU API as a prot flag.
Quoted 7 lines
Quoted 2 lines
Do you agree?
GPU being the only user is for now, but there are other clients which can use this.
Plus how do we set the memory attributes if we do not expose this as prot flag?
It does appear that the downstream kgsl driver sets this for basically all mappings.. well there is some conditional stuff around DOMAIN_ATTR_USE_LLC_NWA but it seems based on the property of the domain. (Jordan may know more about what that is about.) But looks like there are a lot of different paths into iommu_map in kgsl so I might have missed something.
Assuming there isn't some case where we specifically don't want to use the system cache for some mapping, I think it could be a domain attribute that sets an io_pgtable_cfg::quirks flag
BR,
-R
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCHv3 7/7] drm/msm/a6xx: Add support for using system cache(LLC)
Quoted 43 lines · Rob Clark
Quoted 26 lines · Sai Prakash Ranjan
Hi Will,
Quoted 22 lines · Will Deacon
drivers/gpu/drm/msm/msm_iommu.c static int msm_iommu_map(struct msm_mmu *mmu, · line 223
Quoted 12 lines · Sai Prakash Ranjan
drivers/gpu/drm/msm/msm_iommu.c +2
-0@@ -218,6 +218,9 @@ static int msm_iommu_map(struct msm_mmu *mmu,uint64_t iova, iova |= GENMASK_ULL(63, 49); +if (mmu->features &MMU_FEATURE_USE_SYSTEM_CACHE ) + prot |=IOMMU_SYS_CACHE_ONLY ;Given that I think this is the only user of
IOMMU_SYS_CACHE_ONLY, then it
looks like it should actually be a property on the domain because we never
need to configure it on a per-mapping basis within a domain, and therefore
it shouldn't be exposed by the IOMMU API as a prot flag.Quoted 7 lines
Quoted 2 lines
Do you agree?
GPU being the only user is for now, but there are other clients which can use this.
Plus how do we set the memory attributes if we do not expose this as prot flag?It does appear that the downstream kgsl driver sets this for basically all mappings.. well there is some conditional stuff around
DOMAIN_ATTR_USE_LLC_NWAbut it seems based on the property of the domain. (Jordan may know more about what that is about.) But looks like there are a lot of different paths intoiommu_mapin kgsl so I might have missed something.
Quoted 4 lines
Assuming there isn't some case where we specifically don't want to use the system cache for some mapping, I think it could be a domain attribute that sets an
io_pgtable_cfg::quirksflag
Ok then we are good to remove unused sys cache prot flag which Will has posted.
Thanks,
Sai
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCHv3 7/7] drm/msm/a6xx: Add support for using system cache(LLC)
In reply to Rob ClarkQuoted 36 lines · Rob Clark
Quoted 26 lines · Sai Prakash Ranjan
Hi Will,
Quoted 22 lines · Will Deacon
drivers/gpu/drm/msm/msm_iommu.c static int msm_iommu_map(struct msm_mmu *mmu, · line 223
Quoted 12 lines · Sai Prakash Ranjan
drivers/gpu/drm/msm/msm_iommu.c +2
-0@@ -218,6 +218,9 @@ static int msm_iommu_map(struct msm_mmu *mmu,uint64_t iova, iova |= GENMASK_ULL(63, 49); +if (mmu->features &MMU_FEATURE_USE_SYSTEM_CACHE ) + prot |=IOMMU_SYS_CACHE_ONLY ;Given that I think this is the only user of
IOMMU_SYS_CACHE_ONLY, then it
looks like it should actually be a property on the domain because we never
need to configure it on a per-mapping basis within a domain, and therefore
it shouldn't be exposed by the IOMMU API as a prot flag.Quoted 7 lines
Quoted 2 lines
Do you agree?
GPU being the only user is for now, but there are other clients which can use this.
Plus how do we set the memory attributes if we do not expose this as prot flag?
Quoted 6 lines
It does appear that the downstream kgsl driver sets this for basically all mappings.. well there is some conditional stuff around
DOMAIN_ATTR_USE_LLC_NWAbut it seems based on the property of the domain. (Jordan may know more about what that is about.) But looks like there are a lot of different paths intoiommu_mapin kgsl so I might have missed something.
Downstream does set it universally. There are some theoretical use cases where it might be beneficial to set it on a per-mapping basis with a bunch of hinting from userspace and nobody has tried to characterize this on real hardware so it is not clear to me if it is worth it.
I think a domain wide attribute works for now but if a compelling per-mapping use case does comes down the pipeline we need to have a backup in mind - possibly a prot flag to disable NWA?
Jordan
Quoted 6 lines
Assuming there isn't some case where we specifically don't want to use the system cache for some mapping, I think it could be a domain attribute that sets an
io_pgtable_cfg::quirksflagBR,
-R
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project