drm/msm/a6xx: LLCC related fix and cleanup
Subsystems (4)
cover drm/msm/a6xx: LLCC related fix and cleanup
Patch 1 is a fix to not set the attributes when CONFIG_QCOM_LLCC is disabled and Patch 2 is a cleanup to create an a6xx specific address space.
Sai Prakash Ranjan (2):
drm/msm: Add proper checks for GPU LLCC support drm/msm/a6xx: Create an A6XX GPU specific address space
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 48 +++++++++++++++++++++++-- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 23 +++++------- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 7 ++-- 3 files changed, 56 insertions(+), 22 deletions(-)
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
PATCH 1/2 drm/msm: Add proper checks for GPU LLCC support
Domain attribute setting for LLCC is guarded by !IS_ERR check which works fine only when CONFIG_QCOM_LLCC=y but when it is disabled, the LLCC apis return NULL and that is not handled by IS_ERR check. Due to this, domain attribute for LLCC will be set even on GPUs which do not support it and cause issues, so correct this by using IS_ERR_OR_NULL checks appropriately. Meanwhile also cleanup comment block and remove unwanted blank line.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1 -1
@@ -1117,7 +1117,7 @@ static void a6xx_llc_slices_init(struct platform_device *pdev, a6xx_gpu->llc_slice =llcc_slice_getd (LLCC_GPU ); a6xx_gpu->htw_llc_slice =llcc_slice_getd (LLCC_GPUHTW ); -if (IS_ERR (a6xx_gpu->llc_slice) &&IS_ERR (a6xx_gpu->htw_llc_slice)) +if (IS_ERR_OR_NULL (a6xx_gpu->llc_slice) &&IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) a6xx_gpu->llc_mmio =ERR_PTR (-EINVAL ); }
drivers/gpu/drm/msm/adreno/adreno_gpu.c +5 -5
@@ -200,15 +200,15 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu,if (!iommu)return NULL ; -if (adreno_is_a6xx (adreno_gpu)) {struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu);struct io_pgtable_domain_attr pgtbl_cfg; +/* -* This allows GPU to set the bus attributes required to use system -* cache on behalf of the iommu page table walker. -*/ -if (!IS_ERR (a6xx_gpu->htw_llc_slice)) { +* This allows GPU to set the bus attributes required to use system +* cache on behalf of the iommu page table walker. +*/ +if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) { pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ;iommu_domain_set_attr (iommu,DOMAIN_ATTR_IO_PGTABLE_CFG , &pgtbl_cfg); }
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
PATCH 2/2 drm/msm/a6xx: Create an A6XX GPU specific address space
In reply to Sai Prakash RanjanA6XX GPUs have support for last level cache(LLC) also known as system cache and need to set the bus attributes to use it. Currently we use a generic adreno iommu address space implementation which are also used by older GPU generations which do not have LLC and might introduce issues accidentally and is not clean in a way that anymore additions of GPUs supporting LLC would have to be guarded under ifdefs. So keep the generic code separate and make the address space creation A6XX specific. We also have a helper to set the llc attributes so that if the newer GPU generations do support them, we can use it instead of open coding domain attribute setting for each GPU.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 46 ++++++++++++++++++++++++- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 23 +++++-------- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 7 ++-- 3 files changed, 55 insertions(+), 21 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +45 -1
@@ -1239,6 +1239,50 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu)return (unsigned long )busy_time; } +static struct msm_gem_address_space * +a6xx_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) +{ +struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu); +struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); +struct iommu_domain *iommu; +struct msm_mmu *mmu; +struct msm_gem_address_space *aspace; +u64 start, size; + + iommu =iommu_domain_alloc (&platform_bus_type); +if (!iommu) +return NULL ; + +/* +* This allows GPU to set the bus attributes required to use system +* cache on behalf of the iommu page table walker. +*/ +if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) +adreno_set_llc_attributes (iommu); + + mmu =msm_iommu_new (&pdev->dev, iommu); +if (IS_ERR (mmu)) { +iommu_domain_free (iommu); +return ERR_CAST (mmu); + } + +/* +* Use the aperture start or SZ_16M, whichever is greater. This will +* ensure that we align with the allocated pagetable range while still +* allowing room in the lower 32 bits for GMEM and whatnot +*/ + start =max_t (u64,SZ_16M , iommu->geometry.aperture_start); + size = iommu->geometry.aperture_end - start +1 ; + + aspace =msm_gem_address_space_create (mmu,"gpu" , + start &GENMASK_ULL (48 ,0 ), size); + +if (IS_ERR (aspace) && !IS_ERR (mmu)) + mmu->funcs->destroy (mmu); + +return aspace; +} +static struct msm_gem_address_space *a6xx_create_private_address_space (struct msm_gpu *gpu) {
@@ -1285,7 +1329,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_state_get = a6xx_gpu_state_get, .gpu_state_put = a6xx_gpu_state_put, #endif - .create_address_space = adreno_iommu_create_address_space, + .create_address_space = a6xx_create_address_space, .create_private_address_space = a6xx_create_private_address_space, .get_rptr = a6xx_get_rptr, },
drivers/gpu/drm/msm/adreno/adreno_gpu.c +8 -15
@@ -186,11 +186,18 @@ int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid)return zap_shader_load_mdt (gpu, adreno_gpu->info->zapfw, pasid); } +void adreno_set_llc_attributes (struct iommu_domain *iommu) +{ +struct io_pgtable_domain_attr pgtbl_cfg; + + pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ; +iommu_domain_set_attr (iommu,DOMAIN_ATTR_IO_PGTABLE_CFG , &pgtbl_cfg); +} +struct msm_gem_address_space *adreno_iommu_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) { -struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);struct iommu_domain *iommu;struct msm_mmu *mmu;struct msm_gem_address_space *aspace;
@@ -200,20 +207,6 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu,if (!iommu)return NULL ; -if (adreno_is_a6xx (adreno_gpu)) { -struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); -struct io_pgtable_domain_attr pgtbl_cfg; - -/* -* This allows GPU to set the bus attributes required to use system -* cache on behalf of the iommu page table walker. -*/ -if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) { - pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ; -iommu_domain_set_attr (iommu,DOMAIN_ATTR_IO_PGTABLE_CFG , &pgtbl_cfg); - } - } - mmu =msm_iommu_new (&pdev->dev, iommu);if (IS_ERR (mmu)) {iommu_domain_free (iommu);
drivers/gpu/drm/msm/adreno/adreno_gpu.h +2 -5
@@ -212,11 +212,6 @@ static inline int adreno_is_a540(struct adreno_gpu *gpu)return gpu->revn ==540 ; } -static inline bool adreno_is_a6xx (struct adreno_gpu *gpu) -{ -return ((gpu->revn <700 && gpu->revn >599 )); -} -static inline int adreno_is_a618 (struct adreno_gpu *gpu) {return gpu->revn ==618 ;
@@ -278,6 +273,8 @@ struct msm_gem_address_space * adreno_iommu_create_address_space(struct msm_gpu *gpu, struct platform_device *pdev); +void adreno_set_llc_attributes (struct iommu_domain *iommu); +/* * For a5xx and a6xx targets load the zap shader that is used to pull the GPU * out of secure mode
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH 2/2] drm/msm/a6xx: Create an A6XX GPU specific address space
Il 11/01/21 13:04, Sai Prakash Ranjan ha scritto:
Quoted 4 lines
A6XX GPUs have support for last level cache(LLC) also known as system cache and need to set the bus attributes to use it. Currently we use a generic adreno iommu address space implementation which are also used by older GPU generations
Quoted 9 lines
which do not have LLC and might introduce issues accidentally and is not clean in a way that anymore additions of GPUs supporting LLC would have to be guarded under ifdefs. So keep the generic code separate and make the address space creation A6XX specific. We also have a helper to set the llc attributes so that if the newer GPU generations do support them, we can use it instead of open coding domain attribute setting for each GPU.
Hello!
Quoted 49 lines
Signed-off-by Sai Prakash Ranjan <[email protected]>--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 46 ++++++++++++++++++++++++- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 23 +++++-------- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 7 ++-- 3 files changed, 55 insertions(+), 21 deletions(-)drivers/gpu/drm/msm/adreno/a6xx_gpu.c +34
-0@@ -1239,6 +1239,50 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu)return (unsigned long )busy_time; } +static struct msm_gem_address_space * +a6xx_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) +{ +struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu); +struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); +struct iommu_domain *iommu; +struct msm_mmu *mmu; +struct msm_gem_address_space *aspace; +u64 start, size; + + iommu =iommu_domain_alloc (&platform_bus_type); +if (!iommu) +return NULL ; + +/* +* This allows GPU to set the bus attributes required to use system +* cache on behalf of the iommu page table walker. +*/ +if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) +adreno_set_llc_attributes (iommu); + + mmu =msm_iommu_new (&pdev->dev, iommu); +if (IS_ERR (mmu)) { +iommu_domain_free (iommu); +return ERR_CAST (mmu); + } + +/* +* Use the aperture start or SZ_16M, whichever is greater. This will +* ensure that we align with the allocated pagetable range while still +* allowing room in the lower 32 bits for GMEM and whatnot +*/ + start =max_t (u64,SZ_16M , iommu->geometry.aperture_start); + size = iommu->geometry.aperture_end - start +1 ;
drivers/gpu/drm/msm/adreno/a6xx_gpu.c static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu) · line 1285
Quoted 10 lines
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +10
-0+ + aspace =msm_gem_address_space_create (mmu,"gpu" , + start &GENMASK_ULL (48 ,0 ), size); + +if (IS_ERR (aspace) && !IS_ERR (mmu)) + mmu->funcs->destroy (mmu); + +return aspace; +} +
I get what you're trying to do - yes the intentions are good, however... you are effectively duplicating code 1:1, as this *is* the same as function adreno_iommu_create_address_space.
I don't see adding two lines to a function as a valid justification to duplicate all the rest: perhaps, you may want to find another way to do this;
Here's one of the many ideas, perhaps you could:
1. Introduce a "generic feature" to signal LLCC support (perhaps in struct adreno_info ?)
2. If LLCC is supported, and LLCC slices are initialized, set the LLCC attributes on the IOMMU. Of course this would mean passing the init state of the slices (maybe just a bool would be fine) back to the generic adreno_gpu.c
This, unless you tell me that the entire function is going to be a6xx specific, but that doesn't seem to be the case at all.
Concerns are that when an hypotetical Adreno A7XX comes and perhaps also uses the LLCC slices, this function will be duplicated yet another time.
Quoted 17 lines
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1
-1static struct msm_gem_address_space *a6xx_create_private_address_space (struct msm_gpu *gpu) {@@ -1285,7 +1329,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_state_get = a6xx_gpu_state_get, .gpu_state_put = a6xx_gpu_state_put, #endif - .create_address_space = adreno_iommu_create_address_space, + .create_address_space = a6xx_create_address_space, .create_private_address_space = a6xx_create_private_address_space, .get_rptr = a6xx_get_rptr, },drivers/gpu/drm/msm/adreno/adreno_gpu.c +0
-0
drivers/gpu/drm/msm/adreno/adreno_gpu.c int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid) · line 189
Quoted 4 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.c +1
-0@@ -186,11 +186,18 @@ int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid)return zap_shader_load_mdt (gpu, adreno_gpu->info->zapfw, pasid); } +void adreno_set_llc_attributes (struct iommu_domain *iommu)
Since this function is relative to the iommu part of this driver, I think that it would be appropriate to give it the same prefix as all the other functions that are "working in this context".
Hint: adreno_iommu_set_llc_attributes
Alternatively, this two lines function may just be a static inline in the header....
But then, what are we talking about, here?
Since you should stop code duplication and bring everything back in here (in a generic way!!!), then this helper would be of no use, at all, because then you would be just "throwing" these two lines back in the function adreno_iommu_create_address_space....
Quoted 54 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.c +7
-15+{ +struct io_pgtable_domain_attr pgtbl_cfg; + + pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ; +iommu_domain_set_attr (iommu,DOMAIN_ATTR_IO_PGTABLE_CFG , &pgtbl_cfg); +} +struct msm_gem_address_space *adreno_iommu_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) { -struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);struct iommu_domain *iommu;struct msm_mmu *mmu;struct msm_gem_address_space *aspace;@@ -200,20 +207,6 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu,if (!iommu)return NULL ; -if (adreno_is_a6xx (adreno_gpu)) { -struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); -struct io_pgtable_domain_attr pgtbl_cfg; - -/* -* This allows GPU to set the bus attributes required to use system -* cache on behalf of the iommu page table walker. -*/ -if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) { - pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ; -iommu_domain_set_attr (iommu,DOMAIN_ATTR_IO_PGTABLE_CFG , &pgtbl_cfg); - } - } - mmu =msm_iommu_new (&pdev->dev, iommu);if (IS_ERR (mmu)) {iommu_domain_free (iommu);drivers/gpu/drm/msm/adreno/adreno_gpu.h +0
-5@@ -212,11 +212,6 @@ static inline int adreno_is_a540(struct adreno_gpu *gpu)return gpu->revn ==540 ; } -static inline bool adreno_is_a6xx (struct adreno_gpu *gpu) -{ -return ((gpu->revn <700 && gpu->revn >599 )); -} -static inline int adreno_is_a618 (struct adreno_gpu *gpu) {return gpu->revn ==618 ;
drivers/gpu/drm/msm/adreno/adreno_gpu.h struct msm_gem_address_space * · line 280
Quoted 8 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.h +2
-0@@ -278,6 +273,8 @@ struct msm_gem_address_space * adreno_iommu_create_address_space(struct msm_gpu *gpu, struct platform_device *pdev); +void adreno_set_llc_attributes (struct iommu_domain *iommu); +/* * For a5xx and a6xx targets load the zap shader that is used to pull the GPU * out of secure mode
Regards,
- Angelo
Re: [PATCH 2/2] drm/msm/a6xx: Create an A6XX GPU specific address space
Quoted 73 lines · AngeloGioacchino Del Regno
Il 11/01/21 13:04, Sai Prakash Ranjan ha scritto:
Quoted 4 lines
A6XX GPUs have support for last level cache(LLC) also known as system cache and need to set the bus attributes to use it. Currently we use a generic adreno iommu address space implementation which are also used by older GPU generations
Quoted 9 lines
which do not have LLC and might introduce issues accidentally and is not clean in a way that anymore additions of GPUs supporting LLC would have to be guarded under ifdefs. So keep the generic code separate and make the address space creation A6XX specific. We also have a helper to set the llc attributes so that if the newer GPU generations do support them, we can use it instead of open coding domain attribute setting for each GPU.
Hello!
Quoted 55 lines
Signed-off-by Sai Prakash Ranjan <[email protected]>--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 46 ++++++++++++++++++++++++- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 23 +++++-------- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 7 ++-- 3 files changed, 55 insertions(+), 21 deletions(-)drivers/gpu/drm/msm/adreno/a6xx_gpu.c +40
-0@@ -1239,6 +1239,50 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu)return (unsigned long )busy_time; } +static struct msm_gem_address_space * +a6xx_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) +{ +struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu); +struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); +struct iommu_domain *iommu; +struct msm_mmu *mmu; +struct msm_gem_address_space *aspace; +u64 start, size; + + iommu =iommu_domain_alloc (&platform_bus_type); +if (!iommu) +return NULL ; + +/* +* This allows GPU to set the bus attributes required to use system +* cache on behalf of the iommu page table walker. +*/ +if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) +adreno_set_llc_attributes (iommu); + + mmu =msm_iommu_new (&pdev->dev, iommu); +if (IS_ERR (mmu)) { +iommu_domain_free (iommu); +return ERR_CAST (mmu); + } + +/* +* Use the aperture start or SZ_16M, whichever is greater. This will +* ensure that we align with the allocated pagetable range while still +* allowing room in the lower 32 bits for GMEM and whatnot +*/ + start =max_t (u64,SZ_16M , iommu->geometry.aperture_start); + size = iommu->geometry.aperture_end - start +1 ; + + aspace =msm_gem_address_space_create (mmu,"gpu" , + start &GENMASK_ULL (48 ,0 ), size); + +if (IS_ERR (aspace) && !IS_ERR (mmu)) + mmu->funcs->destroy (mmu);
Quoted 8 lines
drivers/gpu/drm/msm/adreno/a6xx_gpu.c static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu) · line 1285
Quoted 4 lines
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +4
-0+ +return aspace; +} +I get what you're trying to do - yes the intentions are good, however... you are effectively duplicating code 1:1, as this *is* the same as function
adreno_iommu_create_address_space.
I had suggested moving this to a6xx, to avoid breaking earlier gens so much.. (Note a2xx by necessity already has it's own version of create_address_space().) I would in general tend to favor a small bit of code duplication to lower the risk of breaking older gens which not everybody has hw to test.
But I suppose we could add a has_llcc() and move the htw_llc_slice up to the 'struct adreno_gpu' level. Casting to a6xx_gpu in common code isn't a great idea. Older gens which don't have LLCC (or don't have LLCC support yet) could leave the slice ptr NULL.
BR,
-R
Quoted 120 lines
I don't see adding two lines to a function as a valid justification to duplicate all the rest: perhaps, you may want to find another way to do this;
Here's one of the many ideas, perhaps you could:
1. Introduce a "generic feature" to signal LLCC support (perhaps in structadreno_info?)
2. If LLCC is supported, and LLCC slices are initialized, set the LLCC attributes on the IOMMU. Of course this would mean passing the init state of the slices (maybe just a bool would be fine) back to the genericadreno_gpu.cThis, unless you tell me that the entire function is going to be a6xx specific, but that doesn't seem to be the case at all.
Concerns are that when an hypotetical Adreno A7XX comes and perhaps also uses the LLCC slices, this function will be duplicated yet another time.
Quoted 17 lines
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1
-1static struct msm_gem_address_space *a6xx_create_private_address_space (struct msm_gpu *gpu) {@@ -1285,7 +1329,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_state_get = a6xx_gpu_state_get, .gpu_state_put = a6xx_gpu_state_put, #endif - .create_address_space = adreno_iommu_create_address_space, + .create_address_space = a6xx_create_address_space, .create_private_address_space = a6xx_create_private_address_space, .get_rptr = a6xx_get_rptr, },drivers/gpu/drm/msm/adreno/adreno_gpu.c +0
-0drivers/gpu/drm/msm/adreno/adreno_gpu.c int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid) · line 189
Quoted 4 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.c +1
-0@@ -186,11 +186,18 @@ int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid)return zap_shader_load_mdt (gpu, adreno_gpu->info->zapfw, pasid); } +void adreno_set_llc_attributes (struct iommu_domain *iommu)Since this function is relative to the iommu part of this driver, I think that it would be appropriate to give it the same prefix as all the other functions that are "working in this context".
Hint:adreno_iommu_set_llc_attributes
Alternatively, this two lines function may just be a static inline in the header....But then, what are we talking about, here?
Since you should stop code duplication and bring everything back in here (in a generic way!!!), then this helper would be of no use, at all, because then you would be just "throwing" these two lines back in the functionadreno_iommu_create_address_space....Quoted 54 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.c +7
-15+{ +struct io_pgtable_domain_attr pgtbl_cfg; + + pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ; +iommu_domain_set_attr (iommu,DOMAIN_ATTR_IO_PGTABLE_CFG , &pgtbl_cfg); +} +struct msm_gem_address_space *adreno_iommu_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) { -struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);struct iommu_domain *iommu;struct msm_mmu *mmu;struct msm_gem_address_space *aspace;@@ -200,20 +207,6 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu,if (!iommu)return NULL ; -if (adreno_is_a6xx (adreno_gpu)) { -struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); -struct io_pgtable_domain_attr pgtbl_cfg; - -/* -* This allows GPU to set the bus attributes required to use system -* cache on behalf of the iommu page table walker. -*/ -if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) { - pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ; -iommu_domain_set_attr (iommu,DOMAIN_ATTR_IO_PGTABLE_CFG , &pgtbl_cfg); - } - } - mmu =msm_iommu_new (&pdev->dev, iommu);if (IS_ERR (mmu)) {iommu_domain_free (iommu);drivers/gpu/drm/msm/adreno/adreno_gpu.h +0
-5@@ -212,11 +212,6 @@ static inline int adreno_is_a540(struct adreno_gpu *gpu)return gpu->revn ==540 ; } -static inline bool adreno_is_a6xx (struct adreno_gpu *gpu) -{ -return ((gpu->revn <700 && gpu->revn >599 )); -} -static inline int adreno_is_a618 (struct adreno_gpu *gpu) {return gpu->revn ==618 ;drivers/gpu/drm/msm/adreno/adreno_gpu.h struct msm_gem_address_space * · line 280
Quoted 8 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.h +2
-0@@ -278,6 +273,8 @@ struct msm_gem_address_space * adreno_iommu_create_address_space(struct msm_gpu *gpu, struct platform_device *pdev); +void adreno_set_llc_attributes (struct iommu_domain *iommu); +/* * For a5xx and a6xx targets load the zap shader that is used to pull the GPU * out of secure mode Regards,
- Angelo
Re: [PATCH 2/2] drm/msm/a6xx: Create an A6XX GPU specific address space
In reply to AngeloGioacchino Del RegnoHi Angelo,
Quoted 96 lines · AngeloGioacchino Del Regno
Il 11/01/21 13:04, Sai Prakash Ranjan ha scritto:
Quoted 4 lines
A6XX GPUs have support for last level cache(LLC) also known as system cache and need to set the bus attributes to use it. Currently we use a generic adreno iommu address space implementation which are also used by older GPU generations
Quoted 9 lines
which do not have LLC and might introduce issues accidentally and is not clean in a way that anymore additions of GPUs supporting LLC would have to be guarded under ifdefs. So keep the generic code separate and make the address space creation A6XX specific. We also have a helper to set the llc attributes so that if the newer GPU generations do support them, we can use it instead of open coding domain attribute setting for each GPU.
Hello!
Quoted 52 lines
Signed-off-by Sai Prakash Ranjan <[email protected]>--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 46 ++++++++++++++++++++++++- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 23 +++++-------- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 7 ++-- 3 files changed, 55 insertions(+), 21 deletions(-)drivers/gpu/drm/msm/adreno/a6xx_gpu.c +29
-0@@ -1239,6 +1239,50 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu)return (unsigned long )busy_time; } +static struct msm_gem_address_space * +a6xx_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) +{ +struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu); +struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); +struct iommu_domain *iommu; +struct msm_mmu *mmu; +struct msm_gem_address_space *aspace; +u64 start, size; + + iommu =iommu_domain_alloc (&platform_bus_type); +if (!iommu) +return NULL ; + +/* +* This allows GPU to set the bus attributes required to use system +* cache on behalf of the iommu page table walker. +*/ +if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) +adreno_set_llc_attributes (iommu); + + mmu =msm_iommu_new (&pdev->dev, iommu); +if (IS_ERR (mmu)) { +iommu_domain_free (iommu); +return ERR_CAST (mmu); + } + +/* +* Use the aperture start or SZ_16M, whichever is greater. This will +* ensure that we align with the allocated pagetable range while still
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +4
-0+* allowing room in the lower 32 bits for GMEM and whatnot +*/ + start =max_t (u64,SZ_16M , iommu->geometry.aperture_start); + size = iommu->geometry.aperture_end - start +1 ;drivers/gpu/drm/msm/adreno/a6xx_gpu.c static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu) · line 1285
Quoted 10 lines
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +10
-0+ + aspace =msm_gem_address_space_create (mmu,"gpu" , + start &GENMASK_ULL (48 ,0 ), size); + +if (IS_ERR (aspace) && !IS_ERR (mmu)) + mmu->funcs->destroy (mmu); + +return aspace; +} +I get what you're trying to do - yes the intentions are good, however... you are effectively duplicating code 1:1, as this *is* the same as function
adreno_iommu_create_address_space.I don't see adding two lines to a function as a valid justification to duplicate all the rest: perhaps, you may want to find another way to do this;
Here's one of the many ideas, perhaps you could:
1. Introduce a "generic feature" to signal LLCC support (perhaps in structadreno_info?)
2. If LLCC is supported, and LLCC slices are initialized, set the LLCC attributes on the IOMMU. Of course this would mean passing the init state of the slices (maybe just a bool would be fine) back to the genericadreno_gpu.c
Quoted 6 lines
This, unless you tell me that the entire function is going to be a6xx specific, but that doesn't seem to be the case at all.
Concerns are that when an hypotetical Adreno A7XX comes and perhaps also uses the LLCC slices, this function will be duplicated yet another time.
As Rob mentioned in other reply, this was more of a point to not break older gen gpus when we add some feature which is specific to a6xx. So there are a{3,4,5}xx using adreno_iommu_create_address_space and any addition to a6xx or let's say in future a7xx(assuming we do not have its own address space impl and use this generic one), then all these older gens need to be taken care of either via some conditions and test on those so that they won't break. IMO, rather than keep adding ifs in the generic code, it is better to have a separate address space impl.
Quoted 18 lines
Quoted 18 lines
static struct msm_gem_address_space * a6xx_create_private_address_space(struct msm_gpu *gpu) {drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1
-1@@ -1285,7 +1329,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_state_get = a6xx_gpu_state_get, .gpu_state_put = a6xx_gpu_state_put, #endif - .create_address_space = adreno_iommu_create_address_space, + .create_address_space = a6xx_create_address_space, .create_private_address_space = a6xx_create_private_address_space, .get_rptr = a6xx_get_rptr, },drivers/gpu/drm/msm/adreno/adreno_gpu.c +0
-0@@ -186,11 +186,18 @@ int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid)
Quoted 8 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.c int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid)
Quoted 3 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.c +0
-0return zap_shader_load_mdt (gpu, adreno_gpu->info->zapfw, pasid); } +void adreno_set_llc_attributes(struct iommu_domain *iommu)Since this function is relative to the iommu part of this driver, I think that it would be appropriate to give it the same prefix as all the other functions that are "working in this context".
Hint:adreno_iommu_set_llc_attributes
Yes, I will change the name to adreno_iommu_set_llc_attributes.
Quoted 11 lines
Alternatively, this two lines function may just be a static inline in the header....
But then, what are we talking about, here?
Since you should stop code duplication and bring everything back in here (in a generic way!!!), then this helper would be of no use, at all, because then you would be just "throwing" these two lines back in the functionadreno_iommu_create_address_space....
See above reply. Thanks for taking a look.
Thanks,
Sai
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [Freedreno] [PATCH 2/2] drm/msm/a6xx: Create an A6XX GPU specific address space
In reply to Sai Prakash RanjanQuoted 90 lines · Sai Prakash Ranjan
A6XX GPUs have support for last level cache(LLC) also known as system cache and need to set the bus attributes to use it. Currently we use a generic adreno iommu address space implementation which are also used by older GPU generations which do not have LLC and might introduce issues accidentally and is not clean in a way that anymore additions of GPUs supporting LLC would have to be guarded under ifdefs. So keep the generic code separate and make the address space creation A6XX specific. We also have a helper to set the llc attributes so that if the newer GPU generations do support them, we can use it instead of open coding domain attribute setting for each GPU.
Signed-off-by Sai Prakash Ranjan <[email protected]>--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 46 ++++++++++++++++++++++++- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 23 +++++-------- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 7 ++-- 3 files changed, 55 insertions(+), 21 deletions(-)drivers/gpu/drm/msm/adreno/a6xx_gpu.c +45
-1@@ -1239,6 +1239,50 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu)return (unsigned long )busy_time; } +static struct msm_gem_address_space * +a6xx_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) +{ +struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu); +struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); +struct iommu_domain *iommu; +struct msm_mmu *mmu; +struct msm_gem_address_space *aspace; +u64 start, size; + + iommu =iommu_domain_alloc (&platform_bus_type); +if (!iommu) +return NULL ; + +/* +* This allows GPU to set the bus attributes required to use system +* cache on behalf of the iommu page table walker. +*/ +if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) +adreno_set_llc_attributes (iommu); + + mmu =msm_iommu_new (&pdev->dev, iommu); +if (IS_ERR (mmu)) { +iommu_domain_free (iommu); +return ERR_CAST (mmu); + } + +/* +* Use the aperture start or SZ_16M, whichever is greater. This will +* ensure that we align with the allocated pagetable range while still +* allowing room in the lower 32 bits for GMEM and whatnot +*/ + start =max_t (u64,SZ_16M , iommu->geometry.aperture_start); + size = iommu->geometry.aperture_end - start +1 ; + + aspace =msm_gem_address_space_create (mmu,"gpu" , + start &GENMASK_ULL (48 ,0 ), size); + +if (IS_ERR (aspace) && !IS_ERR (mmu)) + mmu->funcs->destroy (mmu); + +return aspace; +} +static struct msm_gem_address_space *a6xx_create_private_address_space (struct msm_gpu *gpu) {@@ -1285,7 +1329,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_state_get = a6xx_gpu_state_get, .gpu_state_put = a6xx_gpu_state_put, #endif - .create_address_space = adreno_iommu_create_address_space, + .create_address_space = a6xx_create_address_space, .create_private_address_space = a6xx_create_private_address_space, .get_rptr = a6xx_get_rptr, },drivers/gpu/drm/msm/adreno/adreno_gpu.c +0
-0
drivers/gpu/drm/msm/adreno/adreno_gpu.c int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid) · line 193
Quoted 8 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.c +5
-0@@ -186,11 +186,18 @@ int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid)return zap_shader_load_mdt (gpu, adreno_gpu->info->zapfw, pasid); } +void adreno_set_llc_attributes (struct iommu_domain *iommu) +{ +struct io_pgtable_domain_attr pgtbl_cfg; + + pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ;
btw, since quirks is the only field in the struct currently, this is ok. But better practice to do something like:
struct io_pgtable_domain_attr pgtbl_cfg = {
.quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA,
};which will zero-initialize any additional fields which might be added later, rather than inherit random garbage from the stack.
BR,
-R
Quoted 65 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.c +3
-15+iommu_domain_set_attr (iommu,DOMAIN_ATTR_IO_PGTABLE_CFG , &pgtbl_cfg); +} +struct msm_gem_address_space *adreno_iommu_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) { -struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);struct iommu_domain *iommu;struct msm_mmu *mmu;struct msm_gem_address_space *aspace;@@ -200,20 +207,6 @@ adreno_iommu_create_address_space(struct msm_gpu *gpu,if (!iommu)return NULL ; -if (adreno_is_a6xx (adreno_gpu)) { -struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); -struct io_pgtable_domain_attr pgtbl_cfg; - -/* -* This allows GPU to set the bus attributes required to use system -* cache on behalf of the iommu page table walker. -*/ -if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) { - pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ; -iommu_domain_set_attr (iommu,DOMAIN_ATTR_IO_PGTABLE_CFG , &pgtbl_cfg); - } - } - mmu =msm_iommu_new (&pdev->dev, iommu);if (IS_ERR (mmu)) {iommu_domain_free (iommu);drivers/gpu/drm/msm/adreno/adreno_gpu.h +2
-6@@ -212,11 +212,6 @@ static inline int adreno_is_a540(struct adreno_gpu *gpu)return gpu->revn ==540 ; } -static inline bool adreno_is_a6xx (struct adreno_gpu *gpu) -{ -return ((gpu->revn <700 && gpu->revn >599 )); -} -static inline int adreno_is_a618 (struct adreno_gpu *gpu) {return gpu->revn ==618 ;@@ -278,6 +273,8 @@ struct msm_gem_address_space * adreno_iommu_create_address_space(struct msm_gpu *gpu, struct platform_device *pdev); +void adreno_set_llc_attributes (struct iommu_domain *iommu); +/* * For a5xx and a6xx targets load the zap shader that is used to pull the GPU * out of secure mode -- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
_______________________________________________
Freedreno mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/freedreno
Re: [Freedreno] [PATCH 2/2] drm/msm/a6xx: Create an A6XX GPU specific address space
Quoted 114 lines · Rob Clark
Quoted 98 lines · Sai Prakash Ranjan
A6XX GPUs have support for last level cache(LLC) also known as system cache and need to set the bus attributes to use it. Currently we use a generic adreno iommu address space implementation which are also used by older GPU generations which do not have LLC and might introduce issues accidentally and is not clean in a way that anymore additions of GPUs supporting LLC would have to be guarded under ifdefs. So keep the generic code separate and make the address space creation A6XX specific. We also have a helper to set the llc attributes so that if the newer GPU generations do support them, we can use it instead of open coding domain attribute setting for each GPU.
Signed-off-by Sai Prakash Ranjan <[email protected]>--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 46 ++++++++++++++++++++++++- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 23 +++++-------- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 7 ++-- 3 files changed, 55 insertions(+), 21 deletions(-)drivers/gpu/drm/msm/adreno/a6xx_gpu.c +16
-0@@ -1239,6 +1239,50 @@ static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu)return (unsigned long )busy_time; } +static struct msm_gem_address_space * +a6xx_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) +{ +struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu); +struct a6xx_gpu *a6xx_gpu =to_a6xx_gpu (adreno_gpu); +struct iommu_domain *iommu; +struct msm_mmu *mmu; +struct msm_gem_address_space *aspace; +u64 start, size; + + iommu =iommu_domain_alloc (&platform_bus_type); +if (!iommu) +return NULL ; + +/* +* This allows GPU to set the bus attributes required to use system
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +13
-0+* cache on behalf of the iommu page table walker. +*/ +if (!IS_ERR_OR_NULL (a6xx_gpu->htw_llc_slice)) +adreno_set_llc_attributes (iommu); + + mmu =msm_iommu_new (&pdev->dev, iommu); +if (IS_ERR (mmu)) { +iommu_domain_free (iommu); +return ERR_CAST (mmu); + } + +/* +* Use the aperture start or SZ_16M, whichever is greater. This will
+ * ensure that we align with the allocated pagetable range
while stilldrivers/gpu/drm/msm/adreno/a6xx_gpu.c +15
-1+* allowing room in the lower 32 bits for GMEM and whatnot +*/ + start =max_t (u64,SZ_16M , iommu->geometry.aperture_start); + size = iommu->geometry.aperture_end - start +1 ; + + aspace =msm_gem_address_space_create (mmu,"gpu" , + start &GENMASK_ULL (48 ,0 ), size); + +if (IS_ERR (aspace) && !IS_ERR (mmu)) + mmu->funcs->destroy (mmu); + +return aspace; +} +static struct msm_gem_address_space *a6xx_create_private_address_space (struct msm_gpu *gpu) {@@ -1285,7 +1329,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_state_get = a6xx_gpu_state_get, .gpu_state_put = a6xx_gpu_state_put,#endif - .create_address_space = adreno_iommu_create_address_space, + .create_address_space = a6xx_create_address_space, .create_private_address_space = a6xx_create_private_address_space,.get_rptr = a6xx_get_rptr, },drivers/gpu/drm/msm/adreno/adreno_gpu.c +0
-0drivers/gpu/drm/msm/adreno/adreno_gpu.c int adreno_zap_shader_load(struct msm_gpu *gpu, · line 195
Quoted 10 lines
drivers/gpu/drm/msm/adreno/adreno_gpu.c +5
-0@@ -186,11 +186,18 @@ int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid)return zap_shader_load_mdt (gpu, adreno_gpu->info->zapfw, pasid); } +void adreno_set_llc_attributes (struct iommu_domain *iommu) +{ +struct io_pgtable_domain_attr pgtbl_cfg; + + pgtbl_cfg.quirks =IO_PGTABLE_QUIRK_ARM_OUTER_WBWA ;btw, since quirks is the only field in the struct currently, this is ok. But better practice to do something like:
Quoted 7 lines
struct io_pgtable_domain_attr pgtbl_cfg = { .quirks = IO_PGTABLE_QUIRK_ARM_OUTER_WBWA, };which will zero-initialize any additional fields which might be added later, rather than inherit random garbage from the stack.
Right, I will correct this.
Thanks,
Sai
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
This series was applied to qcom/linux.git (refs/heads/for-next):
Hello:
This series was applied to qcom/linux.git (refs/heads/for-next):
Quoted 9 lines · On Mon, 11 Jan 2021 17:34:07 +0530 you wrote:
Patch 1 is a fix to not set the attributes when
CONFIG_QCOM_LLCCis disabled and Patch 2 is a cleanup to create an a6xx specific address space.Sai Prakash Ranjan (2):
drm/msm: Add proper checks for GPU LLCC support drm/msm/a6xx: Create an A6XX GPU specific address space[...]
Here is the summary with links:
- [1/2] drm/msm: Add proper checks for GPU LLCC support
https://git.kernel.org/qcom/c/276619c0923f - [2/2] drm/msm/a6xx: Create an A6XX GPU specific address space https://git.kernel.org/qcom/c/45596f254061
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html