iommu/arm-smmu-qcom: Only enable stall on smmu-v2
Subsystems (4)
PATCH v2 1/1 iommu/arm-smmu-qcom: Only enable stall on smmu-v2
From: Rob Clark <[email protected]>
On mmu-500, stall-on-fault seems to stall all context banks, causing the GMU to misbehave. So limit this feature to smmu-v2 for now.
This fixes an issue with an older mesa bug taking outo the system because of GMU going off into the weeds.
What we _think_ is happening is that, if the GPU generates 1000's of faults at ~once (which is something that GPUs can be good at), it can result in a sufficient number of stalled translations preventing other transactions from entering the same TBU.
---
v2: Adds a modparam to override the default behavior, for debugging
GPU faults in cases which do not (or might not) cause lockup. Also, rebased to not depend on Bibek's PRR support.
drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)
drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c +17 -2
@@ -16,6 +16,10 @@#define QCOM_DUMMY_VAL -1 +static int enable_stall =-1 ; +MODULE_PARM_DESC (enable_stall,"Enable stall on iova fault (1=on , 0=disable, -1=auto (default))" ); +module_param (enable_stall, int,0600 ); +static struct qcom_smmu *to_qcom_smmu (struct arm_smmu_device *smmu) { return container_of(smmu, struct qcom_smmu, smmu);
@@ -210,7 +214,9 @@ static bool qcom_adreno_can_do_ttbr1(struct arm_smmu_device *smmu)static int qcom_adreno_smmu_init_context (struct arm_smmu_domain *smmu_domain,struct io_pgtable_cfg *pgtbl_cfg,struct device *dev) { +const struct device_node *np = smmu_domain->smmu->dev->of_node;struct adreno_smmu_priv *priv; +bool stall_enabled; smmu_domain->cfg.flush_walk_prefer_tlbiasid = true;
@@ -237,8 +243,17 @@ static int qcom_adreno_smmu_init_context(struct arm_smmu_domain *smmu_domain, priv->get_ttbr1_cfg = qcom_adreno_smmu_get_ttbr1_cfg; priv->set_ttbr0_cfg = qcom_adreno_smmu_set_ttbr0_cfg; priv->get_fault_info = qcom_adreno_smmu_get_fault_info; - priv->set_stall = qcom_adreno_smmu_set_stall; - priv->resume_translation = qcom_adreno_smmu_resume_translation; + +if (enable_stall <0 ) { + stall_enabled =of_device_is_compatible (np,"qcom,smmu-v2" ); + }else { + stall_enabled = !!enable_stall; + } + +if (stall_enabled) { + priv->set_stall = qcom_adreno_smmu_set_stall; + priv->resume_translation = qcom_adreno_smmu_resume_translation; + }return 0 ; }
Signature · 1 line
2.47.1
Quoted 8 lines · Rob Clark
From: Rob Clark <[email protected]>
On mmu-500, stall-on-fault seems to stall all context banks, causing the GMU to misbehave. So limit this feature to smmu-v2 for now.
This fixes an issue with an older mesa bug taking outo the system because of GMU going off into the weeds.
Quoted 6 lines
What we _think_ is happening is that, if the GPU generates 1000's of faults at ~once (which is something that GPUs can be good at), it can result in a sufficient number of stalled translations preventing other transactions from entering the same TBU.
Signed-off-by Rob Clark <[email protected]>
-Akhil
Quoted 53 lines
---
v2: Adds a modparam to override the default behavior, for debugging
GPU faults in cases which do not (or might not) cause lockup. Also, rebased to not depend on Bibek's PRR support.drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)drivers/iommu/arm/arm-smmu/arm-smmu-qcom.c +17
-2@@ -16,6 +16,10 @@#define QCOM_DUMMY_VAL -1 +static int enable_stall =-1 ; +MODULE_PARM_DESC (enable_stall,"Enable stall on iova fault (1=on , 0=disable, -1=auto (default))" ); +module_param (enable_stall, int,0600 ); +static struct qcom_smmu *to_qcom_smmu (struct arm_smmu_device *smmu) { return container_of(smmu, struct qcom_smmu, smmu);@@ -210,7 +214,9 @@ static bool qcom_adreno_can_do_ttbr1(struct arm_smmu_device *smmu)static int qcom_adreno_smmu_init_context (struct arm_smmu_domain *smmu_domain,struct io_pgtable_cfg *pgtbl_cfg,struct device *dev) { +const struct device_node *np = smmu_domain->smmu->dev->of_node;struct adreno_smmu_priv *priv; +bool stall_enabled; smmu_domain->cfg.flush_walk_prefer_tlbiasid = true;@@ -237,8 +243,17 @@ static int qcom_adreno_smmu_init_context(struct arm_smmu_domain *smmu_domain, priv->get_ttbr1_cfg = qcom_adreno_smmu_get_ttbr1_cfg; priv->set_ttbr0_cfg = qcom_adreno_smmu_set_ttbr0_cfg; priv->get_fault_info = qcom_adreno_smmu_get_fault_info; - priv->set_stall = qcom_adreno_smmu_set_stall; - priv->resume_translation = qcom_adreno_smmu_resume_translation; + +if (enable_stall <0 ) { + stall_enabled =of_device_is_compatible (np,"qcom,smmu-v2" ); + }else { + stall_enabled = !!enable_stall; + } + +if (stall_enabled) { + priv->set_stall = qcom_adreno_smmu_set_stall; + priv->resume_translation = qcom_adreno_smmu_resume_translation; + }return 0 ; }
Quoted 9 lines · Akhil P Oommen
Quoted 8 lines · Rob Clark
From: Rob Clark <[email protected]>
On mmu-500, stall-on-fault seems to stall all context banks, causing the GMU to misbehave. So limit this feature to smmu-v2 for now.
This fixes an issue with an older mesa bug taking outo the system because of GMU going off into the weeds.
Quoted 9 lines
Quoted 6 lines
What we _think_ is happening is that, if the GPU generates 1000's of faults at ~once (which is something that GPUs can be good at), it can result in a sufficient number of stalled translations preventing other transactions from entering the same TBU.
Signed-off-by Rob Clark <[email protected]>Reviewed-by Akhil P Oommen <[email protected]>
Btw, if stall is not enabled, I think there is no point in capturing coredump from adreno pagefault handler. By the time we start coredump, gpu might have switched context.
-Akhil.
Quoted 2 lines
-Akhil
Quoted 29 lines · Akhil P Oommen
Quoted 9 lines · Akhil P Oommen
Quoted 8 lines · Rob Clark
From: Rob Clark <[email protected]>
On mmu-500, stall-on-fault seems to stall all context banks, causing the GMU to misbehave. So limit this feature to smmu-v2 for now.
This fixes an issue with an older mesa bug taking outo the system because of GMU going off into the weeds.
Quoted 9 lines
Quoted 6 lines
What we _think_ is happening is that, if the GPU generates 1000's of faults at ~once (which is something that GPUs can be good at), it can result in a sufficient number of stalled translations preventing other transactions from entering the same TBU.
Signed-off-by Rob Clark <[email protected]>Reviewed-by Akhil P Oommen <[email protected]>Btw, if stall is not enabled, I think there is no point in capturing coredump from adreno pagefault handler. By the time we start coredump, gpu might have switched context.
-Akhil.
Quoted 2 lines
-Akhil
Quoted 16 lines · Akhil P Oommen
Quoted 14 lines · Akhil P Oommen
Quoted 13 lines · Rob Clark
From: Rob Clark <[email protected]>
On mmu-500, stall-on-fault seems to stall all context banks, causing the GMU to misbehave. So limit this feature to smmu-v2 for now.
This fixes an issue with an older mesa bug taking outo the system because of GMU going off into the weeds.
What we _think_ is happening is that, if the GPU generates 1000's of faults at ~once (which is something that GPUs can be good at), it can result in a sufficient number of stalled translations preventing other transactions from entering the same TBU.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Signed-off-by Rob Clark <[email protected]>Reviewed-by Akhil P Oommen <[email protected]>Btw, if stall is not enabled, I think there is no point in capturing coredump from adreno pagefault handler. By the time we start coredump, gpu might have switched context.
Hmm, we do at least capture ttbr0 both in fault info and from the current submit, so it would at least be possible to tell if you are looking at the wrong context.
BR,
-R
Will Deacon
maintainerQuoted 5 lines · Rob Clark
From: Rob Clark <[email protected]>
On mmu-500, stall-on-fault seems to stall all context banks, causing the GMU to misbehave. So limit this feature to smmu-v2 for now.
Quoted 7 lines
This fixes an issue with an older mesa bug taking outo the system because of GMU going off into the weeds.
What we _think_ is happening is that, if the GPU generates 1000's of faults at ~once (which is something that GPUs can be good at), it can result in a sufficient number of stalled translations preventing other transactions from entering the same TBU.
MMU-500 is an implementation of the SMMUv2 architecture, so this feels upside-down to me. That is, it should always be valid to probe with the less specific "SMMUv2" compatible string (modulo hardware errata) and be limited to the architectural behaviour.
So what is about MMU-500 that means stalling doesn't work when compared to any other SMMUv2 implementation?
Will
Quoted 15 lines · Will Deacon
Quoted 5 lines · Rob Clark
From: Rob Clark <[email protected]>
On mmu-500, stall-on-fault seems to stall all context banks, causing the GMU to misbehave. So limit this feature to smmu-v2 for now.
Quoted 7 lines
This fixes an issue with an older mesa bug taking outo the system because of GMU going off into the weeds.
What we _think_ is happening is that, if the GPU generates 1000's of faults at ~once (which is something that GPUs can be good at), it can result in a sufficient number of stalled translations preventing other transactions from entering the same TBU.
Quoted 4 lines
MMU-500 is an implementation of the SMMUv2 architecture, so this feels upside-down to me. That is, it should always be valid to probe with the less specific "SMMUv2" compatible string (modulo hardware errata) and be limited to the architectural behaviour.
I should have been more specific and referred to qcom,smmu-v2
Quoted 2 lines
So what is about MMU-500 that means stalling doesn't work when compared to any other SMMUv2 implementation?
Well, I have a limited # of data points, in the sense that there aren't too many a6xx devices prior to the switch to qcom,smmu-500.. but I have access to crash metrics for a lot of sc7180 devices (qcom,smmu-v2), and I've been unable to find any signs of this sort of stall related issue.
So maybe I can't 100% say this is qcom,smmu-500 vs qcom,smmu-v2, vs some other change in later gens that used qcom,smmu-500 or some other factor, I'm not sure what other conclusion to draw.
BR,
-R
Quoted 32 lines · Rob Clark
Quoted 15 lines · Will Deacon
Quoted 5 lines · Rob Clark
From: Rob Clark <[email protected]>
On mmu-500, stall-on-fault seems to stall all context banks, causing the GMU to misbehave. So limit this feature to smmu-v2 for now.
Quoted 7 lines
This fixes an issue with an older mesa bug taking outo the system because of GMU going off into the weeds.
What we _think_ is happening is that, if the GPU generates 1000's of faults at ~once (which is something that GPUs can be good at), it can result in a sufficient number of stalled translations preventing other transactions from entering the same TBU.
Quoted 4 lines
MMU-500 is an implementation of the SMMUv2 architecture, so this feels upside-down to me. That is, it should always be valid to probe with the less specific "SMMUv2" compatible string (modulo hardware errata) and be limited to the architectural behaviour.
I should have been more specific and referred to qcom,smmu-v2
Quoted 2 lines
So what is about MMU-500 that means stalling doesn't work when compared to any other SMMUv2 implementation?
Well, I have a limited # of data points, in the sense that there aren't too many a6xx devices prior to the switch to qcom,smmu-500.. but I have access to crash metrics for a lot of sc7180 devices (qcom,smmu-v2), and I've been unable to find any signs of this sort of stall related issue.
Quoted 3 lines
So maybe I can't 100% say this is qcom,smmu-500 vs qcom,smmu-v2, vs some other change in later gens that used qcom,smmu-500 or some other factor, I'm not sure what other conclusion to draw.
Might it be that v2 was an actual hw, but mmu-500 is somehow virtualized? And as such by these stalls we might be observing some kind of FW bug in hyp?
Quoted 3 lines
BR,
-R
Signature · 2 lines
With best wishes Dmitry