Skip to contentSkip to conversation

iommu/tegra-smmu: Add pagetable mappings to debugfs

PATCH In review Nicolin Chen 3 participants 23 messages Sep 14, 2021 – Oct 7, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v6 cover iommu/tegra-smmu: Add pagetable mappings to debugfs

This series of patches adds a new mappings node to debugfs for tegra-smmu driver. The first five patches are all preparational changes for PATCH-6, based on Thierry's review feedback against
v5: https://lkml.org/lkml/2021/3/16/447

Changelog
v6:
* Added PATCH1-3 for better naming conventions
* Added PATCH4-5 to embed previous struct tegra_smmu_group_debug into struct tegra_smmu_group
* Dropped parentheses at SMMU_PTE_ATTR_SHIFT
* Dropped swgrp->reg print
* Replaced ptb_reg contents with as->attr and as->pd_dma
* Added "index" and "count" in the PD entries for readability
* Removed Dmitry's Tested-by and Reviewed-by for the big change from v5 to v6.
v5: https://lkml.org/lkml/2021/3/15/2473
* Fixed a typo in commit message
* Split a long line into two lines
* Rearranged variable defines by length
* Added Tested-by and Reviewed-by from Dmitry
v4: https://lkml.org/lkml/2021/3/14/429
* Changed %d to %u for unsigned variables
* Fixed print format mismatch warnings on ARM32
v3: https://lkml.org/lkml/2021/3/14/30
* Fixed PHYS and IOVA print formats
* Changed variables to unsigned int type
* Changed the table outputs to be compact
v2: https://lkml.org/lkml/2021/3/9/1382
* Expanded mutex range to the entire function
* Added as->lock to protect pagetable walkthrough
* Replaced devm_kzalloc with devm_kcalloc for group_debug
* Added "PTE RANGE" and "SIZE" columns to group contiguous mappings
* Dropped as->count check
* Added WARN_ON when as->count mismatches pd[pd_index]
v1: https://lkml.org/lkml/2020/9/26/70

Nicolin Chen (6):
iommu/tegra-smmu: Rename struct iommu_group *group to *grp iommu/tegra-smmu: Rename struct tegra_smmu_group_soc *soc to
*group_soc
iommu/tegra-smmu: Rename struct tegra_smmu_swgroup *group to *swgrp iommu/tegra-smmu: Use swgrp pointer instead of swgroup id iommu/tegra-smmu: Attach as pointer to tegra_smmu_group iommu/tegra-smmu: Add pagetable mappings to debugfs

 drivers/iommu/tegra-smmu.c | 312 +++++++++++++++++++++++++++++++------
 1 file changed, 262 insertions(+), 50 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH v6 1/6 iommu/tegra-smmu: Rename struct iommu_group *group to *grp

There are a few structs using "group" for their pointer instances. This gets confusing sometimes. The instance of struct iommu_group is used in local function with an alias "grp", which can separate it from others.

So this patch simply renames "group" to "grp" as a cleanup.

---
 drivers/iommu/tegra-smmu.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
drivers/iommu/tegra-smmu.c +8 -8
index 0a281833f611..6ebae635d3aa 100644
@@ -23,7 +23,7 @@ struct tegra_smmu_group {
 	struct list_head list;
 	struct tegra_smmu *smmu;
 	const struct tegra_smmu_group_soc *soc;
-	struct iommu_group *group;
+	struct iommu_group *grp;
 	unsigned int swgroup;
 };
 
@@ -909,7 +909,7 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	/* Find existing iommu_group associating with swgroup or group_soc */
 	list_for_each_entry(group, &smmu->groups, list)
 		if ((group->swgroup == swgroup) || (soc && group->soc == soc)) {
-			grp = iommu_group_ref_get(group->group);
+			grp = iommu_group_ref_get(group->grp);
 			mutex_unlock(&smmu->lock);
 			return grp;
 		}
@@ -926,23 +926,23 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	group->soc = soc;
 
 	if (dev_is_pci(dev))
-		group->group = pci_device_group(dev);
+		group->grp = pci_device_group(dev);
 	else
-		group->group = generic_device_group(dev);
+		group->grp = generic_device_group(dev);
 
-	if (IS_ERR(group->group)) {
+	if (IS_ERR(group->grp)) {
 		devm_kfree(smmu->dev, group);
 		mutex_unlock(&smmu->lock);
 		return NULL;
 	}
 
-	iommu_group_set_iommudata(group->group, group, tegra_smmu_group_release);
+	iommu_group_set_iommudata(group->grp, group, tegra_smmu_group_release);
 	if (soc)
-		iommu_group_set_name(group->group, soc->name);
+		iommu_group_set_name(group->grp, soc->name);
 	list_add_tail(&group->list, &smmu->groups);
 	mutex_unlock(&smmu->lock);
 
-	return group->group;
+	return group->grp;
 }
 
 static int tegra_smmu_of_xlate(struct device *dev,
Signature · 1 line
2.17.1
Acked-by on v6 1/6
#2 lore raw

Re: [PATCH v6 1/6] iommu/tegra-smmu: Rename struct iommu_group *group to *grp

Quoted 11 lines · Nicolin Chen

There are a few structs using "group" for their pointer instances. This gets confusing sometimes. The instance of struct iommu_group is used in local function with an alias "grp", which can separate it from others.

So this patch simply renames "group" to "grp" as a cleanup.

---
 drivers/iommu/tegra-smmu.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
#3 lore raw

PATCH v6 2/6 iommu/tegra-smmu: Rename struct tegra_smmu_group_soc *soc to *group_soc

In reply to Nicolin Chen

There are both tegra_smmu_soc and tegra_smmu_group_soc using "soc" for their pointer instances. This patch renames the one of struct tegra_smmu_group_soc from "soc" to "group_soc" to distinguish it.

---
 drivers/iommu/tegra-smmu.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
drivers/iommu/tegra-smmu.c +9 -8
index 6ebae635d3aa..a32ed347e25d 100644
@@ -22,7 +22,7 @@
 struct tegra_smmu_group {
 	struct list_head list;
 	struct tegra_smmu *smmu;
-	const struct tegra_smmu_group_soc *soc;
+	const struct tegra_smmu_group_soc *group_soc;
 	struct iommu_group *grp;
 	unsigned int swgroup;
 };
@@ -870,7 +870,7 @@ static struct iommu_device *tegra_smmu_probe_device(struct device *dev)
 static void tegra_smmu_release_device(struct device *dev) {}
 
 static const struct tegra_smmu_group_soc *
-tegra_smmu_find_group(struct tegra_smmu *smmu, unsigned int swgroup)
+tegra_smmu_find_group_soc(struct tegra_smmu *smmu, unsigned int swgroup)
 {
 	unsigned int i, j;
 
@@ -896,19 +896,20 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 {
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
 	struct tegra_smmu *smmu = dev_iommu_priv_get(dev);
-	const struct tegra_smmu_group_soc *soc;
+	const struct tegra_smmu_group_soc *group_soc;
 	unsigned int swgroup = fwspec->ids[0];
 	struct tegra_smmu_group *group;
 	struct iommu_group *grp;
 
 	/* Find group_soc associating with swgroup */
-	soc = tegra_smmu_find_group(smmu, swgroup);
+	group_soc = tegra_smmu_find_group_soc(smmu, swgroup);
 
 	mutex_lock(&smmu->lock);
 
 	/* Find existing iommu_group associating with swgroup or group_soc */
 	list_for_each_entry(group, &smmu->groups, list)
-		if ((group->swgroup == swgroup) || (soc && group->soc == soc)) {
+		if ((group->swgroup == swgroup) ||
+		    (group_soc && group->group_soc == group_soc)) {
 			grp = iommu_group_ref_get(group->grp);
 			mutex_unlock(&smmu->lock);
 			return grp;
@@ -921,9 +922,9 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	}
 
 	INIT_LIST_HEAD(&group->list);
+	group->group_soc = group_soc;
 	group->swgroup = swgroup;
 	group->smmu = smmu;
-	group->soc = soc;
 
 	if (dev_is_pci(dev))
 		group->grp = pci_device_group(dev);
@@ -937,8 +938,8 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	}
 
 	iommu_group_set_iommudata(group->grp, group, tegra_smmu_group_release);
-	if (soc)
-		iommu_group_set_name(group->grp, soc->name);
+	if (group_soc)
+		iommu_group_set_name(group->grp, group_soc->name);
 	list_add_tail(&group->list, &smmu->groups);
 	mutex_unlock(&smmu->lock);
 
Signature · 1 line
2.17.1
#4 lore raw

Re: [PATCH v6 2/6] iommu/tegra-smmu: Rename struct tegra_smmu_group_soc *soc to *group_soc

Quoted 8 lines · Nicolin Chen

There are both tegra_smmu_soc and tegra_smmu_group_soc using "soc" for their pointer instances. This patch renames the one of struct tegra_smmu_group_soc from "soc" to "group_soc" to distinguish it.

---
 drivers/iommu/tegra-smmu.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

I think the context makes it clear which one this is. The "soc" field in struct tegra_smmu_group clearly refers to the group SoC data, whereas the "soc" field in struct tegra_smmu refers to the SMMU SoC data.

Quoted 15 lines
drivers/iommu/tegra-smmu.c +1 -1
index 6ebae635d3aa..a32ed347e25d 100644
@@ -22,7 +22,7 @@
 struct tegra_smmu_group {
 	struct list_head list;
 	struct tegra_smmu *smmu;
-	const struct tegra_smmu_group_soc *soc;
+	const struct tegra_smmu_group_soc *group_soc;
 	struct iommu_group *grp;
 	unsigned int swgroup;
 };

drivers/iommu/tegra-smmu.c static struct iommu_device *tegra_smmu_probe_device(struct device *dev) · line 873

Quoted 5 lines
drivers/iommu/tegra-smmu.c +1 -1
@@ -870,7 +870,7 @@ static struct iommu_device *tegra_smmu_probe_device(struct device *dev)
 static void tegra_smmu_release_device(struct device *dev) {}
 
 static const struct tegra_smmu_group_soc *
-tegra_smmu_find_group(struct tegra_smmu *smmu, unsigned int swgroup)
+tegra_smmu_find_group_soc(struct tegra_smmu *smmu, unsigned int swgroup)

This one might be okay to disambiguate, but even here I think this isn't really necessary. It's already clear from the return value what's being returned.

Quoted 28 lines
drivers/iommu/tegra-smmu.c +4 -3
 {
 	unsigned int i, j;
 
@@ -896,19 +896,20 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 {
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
 	struct tegra_smmu *smmu = dev_iommu_priv_get(dev);
-	const struct tegra_smmu_group_soc *soc;
+	const struct tegra_smmu_group_soc *group_soc;
 	unsigned int swgroup = fwspec->ids[0];
 	struct tegra_smmu_group *group;
 	struct iommu_group *grp;
 
 	/* Find group_soc associating with swgroup */
-	soc = tegra_smmu_find_group(smmu, swgroup);
+	group_soc = tegra_smmu_find_group_soc(smmu, swgroup);
 
 	mutex_lock(&smmu->lock);
 
 	/* Find existing iommu_group associating with swgroup or group_soc */
 	list_for_each_entry(group, &smmu->groups, list)
-		if ((group->swgroup == swgroup) || (soc && group->soc == soc)) {
+		if ((group->swgroup == swgroup) ||
+		    (group_soc && group->group_soc == group_soc)) {
 			grp = iommu_group_ref_get(group->grp);
 			mutex_unlock(&smmu->lock);
 			return grp;

drivers/iommu/tegra-smmu.c static struct iommu_group *tegra_smmu_device_group(struct device *dev) · line 926

Quoted 7 lines
drivers/iommu/tegra-smmu.c +1 -1
@@ -921,9 +922,9 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	}
 
 	INIT_LIST_HEAD(&group->list);
+	group->group_soc = group_soc;
 	group->swgroup = swgroup;
 	group->smmu = smmu;
-	group->soc = soc;

As another example, it's pretty evident that group->soc refers to the group SoC data rather than the SMMU SoC data. The latter can be obtained from group->smmu->soc, which again is enough context to make it clear what this is.

So I don't think this makes things any clearer. It only makes the names more redundant and awkward to write.

Thierry

#5 lore raw

Re: [PATCH v6 2/6] iommu/tegra-smmu: Rename struct tegra_smmu_group_soc *soc to *group_soc

Quoted 7 lines · Thierry Reding

drivers/iommu/tegra-smmu.c static struct iommu_device *tegra_smmu_probe_device(struct device *dev) · line 873

Quoted 3 lines

static const struct tegra_smmu_group_soc *

drivers/iommu/tegra-smmu.c +1 -1
-tegra_smmu_find_group(struct tegra_smmu *smmu, unsigned int swgroup)
+tegra_smmu_find_group_soc(struct tegra_smmu *smmu, unsigned int swgroup)

This one might be okay to disambiguate, but even here I think this isn't really necessary. It's already clear from the return value what's being returned.

The point here is to disambiguate "group", as there are quite a few places using the same naming for different structures. You may argue that it's clear by looking at the return value/type. But it is still hard to tell when reading the code of its caller, right?

Quoted 14 lines

drivers/iommu/tegra-smmu.c static struct iommu_group *tegra_smmu_device_group(struct device *dev) · line 926

Quoted 8 lines
drivers/iommu/tegra-smmu.c +1 -1
@@ -921,9 +922,9 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	}
 
 	INIT_LIST_HEAD(&group->list);
+	group->group_soc = group_soc;
 	group->swgroup = swgroup;
 	group->smmu = smmu;
-	group->soc = soc;

As another example, it's pretty evident that group->soc refers to the group SoC data rather than the SMMU SoC data. The latter can be obtained from group->smmu->soc, which again is enough context to make it clear what this is.

Quoted 2 lines

So I don't think this makes things any clearer. It only makes the names more redundant and awkward to write.

Okay. I will drop the part of s/soc/group_soc.

#6 lore raw

PATCH v6 3/6 iommu/tegra-smmu: Rename struct tegra_smmu_swgroup *group to *swgrp

In reply to Nicolin Chen

There are both tegra_smmu_swgroup and tegra_smmu_group structs using "group" for their pointer instances. This gets confusing to read the driver sometimes.

So this patch renames "group" of struct tegra_smmu_swgroup to "swgrp" as a cleanup. Also renames its "find" function.

Note that we already have "swgroup" being used for an unsigned int type variable that is inside struct tegra_smmu_swgroup, so it's not able to use "swgroup" but only something like "swgrp".

---
 drivers/iommu/tegra-smmu.c | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)
drivers/iommu/tegra-smmu.c +17 -17
index a32ed347e25d..0f3883045ffa 100644
@@ -334,35 +334,35 @@ static void tegra_smmu_domain_free(struct iommu_domain *domain)
 }
 
 static const struct tegra_smmu_swgroup *
-tegra_smmu_find_swgroup(struct tegra_smmu *smmu, unsigned int swgroup)
+tegra_smmu_find_swgrp(struct tegra_smmu *smmu, unsigned int swgroup)
 {
-	const struct tegra_smmu_swgroup *group = NULL;
+	const struct tegra_smmu_swgroup *swgrp = NULL;
 	unsigned int i;
 
 	for (i = 0; i < smmu->soc->num_swgroups; i++) {
 		if (smmu->soc->swgroups[i].swgroup == swgroup) {
-			group = &smmu->soc->swgroups[i];
+			swgrp = &smmu->soc->swgroups[i];
 			break;
 		}
 	}
 
-	return group;
+	return swgrp;
 }
 
 static void tegra_smmu_enable(struct tegra_smmu *smmu, unsigned int swgroup,
 			      unsigned int asid)
 {
-	const struct tegra_smmu_swgroup *group;
+	const struct tegra_smmu_swgroup *swgrp;
 	unsigned int i;
 	u32 value;
 
-	group = tegra_smmu_find_swgroup(smmu, swgroup);
-	if (group) {
-		value = smmu_readl(smmu, group->reg);
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+	if (swgrp) {
+		value = smmu_readl(smmu, swgrp->reg);
 		value &= ~SMMU_ASID_MASK;
 		value |= SMMU_ASID_VALUE(asid);
 		value |= SMMU_ASID_ENABLE;
-		smmu_writel(smmu, value, group->reg);
+		smmu_writel(smmu, value, swgrp->reg);
 	} else {
 		pr_warn("%s group from swgroup %u not found\n", __func__,
 				swgroup);
@@ -385,17 +385,17 @@ static void tegra_smmu_enable(struct tegra_smmu *smmu, unsigned int swgroup,
 static void tegra_smmu_disable(struct tegra_smmu *smmu, unsigned int swgroup,
 			       unsigned int asid)
 {
-	const struct tegra_smmu_swgroup *group;
+	const struct tegra_smmu_swgroup *swgrp;
 	unsigned int i;
 	u32 value;
 
-	group = tegra_smmu_find_swgroup(smmu, swgroup);
-	if (group) {
-		value = smmu_readl(smmu, group->reg);
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+	if (swgrp) {
+		value = smmu_readl(smmu, swgrp->reg);
 		value &= ~SMMU_ASID_MASK;
 		value |= SMMU_ASID_VALUE(asid);
 		value &= ~SMMU_ASID_ENABLE;
-		smmu_writel(smmu, value, group->reg);
+		smmu_writel(smmu, value, swgrp->reg);
 	}
 
 	for (i = 0; i < smmu->soc->num_clients; i++) {
@@ -1008,11 +1008,11 @@ static int tegra_smmu_swgroups_show(struct seq_file *s, void *data)
 	seq_printf(s, "------------------------\n");
 
 	for (i = 0; i < smmu->soc->num_swgroups; i++) {
-		const struct tegra_smmu_swgroup *group = &smmu->soc->swgroups[i];
+		const struct tegra_smmu_swgroup *swgrp = &smmu->soc->swgroups[i];
 		const char *status;
 		unsigned int asid;
 
-		value = smmu_readl(smmu, group->reg);
+		value = smmu_readl(smmu, swgrp->reg);
 
 		if (value & SMMU_ASID_ENABLE)
 			status = "yes";
@@ -1021,7 +1021,7 @@ static int tegra_smmu_swgroups_show(struct seq_file *s, void *data)
 
 		asid = value & SMMU_ASID_MASK;
 
-		seq_printf(s, "%-9s  %-7s  %#04x\n", group->name, status,
+		seq_printf(s, "%-9s  %-7s  %#04x\n", swgrp->name, status,
 			   asid);
 	}
 
Signature · 1 line
2.17.1
#7 lore raw

Re: [PATCH v6 3/6] iommu/tegra-smmu: Rename struct tegra_smmu_swgroup *group to *swgrp

Quoted 21 lines · Nicolin Chen

There are both tegra_smmu_swgroup and tegra_smmu_group structs using "group" for their pointer instances. This gets confusing to read the driver sometimes.

So this patch renames "group" of struct tegra_smmu_swgroup to "swgrp" as a cleanup. Also renames its "find" function.

Note that we already have "swgroup" being used for an unsigned int type variable that is inside struct tegra_smmu_swgroup, so it's not able to use "swgroup" but only something like "swgrp".

---
 drivers/iommu/tegra-smmu.c | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)
drivers/iommu/tegra-smmu.c +0 -0
index a32ed347e25d..0f3883045ffa 100644

drivers/iommu/tegra-smmu.c static void tegra_smmu_domain_free(struct iommu_domain *domain) · line 337

Quoted 5 lines
drivers/iommu/tegra-smmu.c +1 -1
@@ -334,35 +334,35 @@ static void tegra_smmu_domain_free(struct iommu_domain *domain)
 }
 
 static const struct tegra_smmu_swgroup *
-tegra_smmu_find_swgroup(struct tegra_smmu *smmu, unsigned int swgroup)
+tegra_smmu_find_swgrp(struct tegra_smmu *smmu, unsigned int swgroup)

This makes things inconsistent now. The tegra_smmu_find_swgroup() name indicates that we're looking for some "swgroup" entity within an "smmu" object. The entity that we're looking for is a struct tegra_smmu_swgroup so I think it makes sense to use that full name in the function name.

drivers/iommu/tegra-smmu.c static void tegra_smmu_domain_free(struct iommu_domain *domain) · line 339

Quoted 3 lines
drivers/iommu/tegra-smmu.c +1 -1
 {
-	const struct tegra_smmu_swgroup *group = NULL;
+	const struct tegra_smmu_swgroup *swgrp = NULL;

I don't think the existing naming is confusing. The variable name "group" is consistently used for tegra_smmu_swgroup structures and there are no cases where we would confuse them with struct tegra_smmu_group instances.

However, I don't feel strongly about it, so I'm fine with changing the variable names to "swgrp" if you think that makes things less confusing.

Thierry

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#8 lore raw

Re: [PATCH v6 3/6] iommu/tegra-smmu: Rename struct tegra_smmu_swgroup *group to *swgrp

Quoted 24 lines · Thierry Reding
Quoted 23 lines · Nicolin Chen

There are both tegra_smmu_swgroup and tegra_smmu_group structs using "group" for their pointer instances. This gets confusing to read the driver sometimes.

So this patch renames "group" of struct tegra_smmu_swgroup to "swgrp" as a cleanup. Also renames its "find" function.

Note that we already have "swgroup" being used for an unsigned int type variable that is inside struct tegra_smmu_swgroup, so it's not able to use "swgroup" but only something like "swgrp".

---
 drivers/iommu/tegra-smmu.c | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)
drivers/iommu/tegra-smmu.c +0 -0
index a32ed347e25d..0f3883045ffa 100644
@@ -334,35 +334,35 @@ static void tegra_smmu_domain_free(struct iommu_domain *domain)
 }
 
Quoted 8 lines

drivers/iommu/tegra-smmu.c static void tegra_smmu_domain_free(struct iommu_domain *domain) · line 337

Quoted 3 lines
drivers/iommu/tegra-smmu.c +1 -1
 static const struct tegra_smmu_swgroup *
-tegra_smmu_find_swgroup(struct tegra_smmu *smmu, unsigned int swgroup)
+tegra_smmu_find_swgrp(struct tegra_smmu *smmu, unsigned int swgroup)

This makes things inconsistent now. The tegra_smmu_find_swgroup() name indicates that we're looking for some "swgroup" entity within an "smmu" object. The entity that we're looking for is a struct tegra_smmu_swgroup so I think it makes sense to use that full name in the function name.

This is more like an indirect change to keep consistency between function name and pointer name.

Quoted 8 lines

drivers/iommu/tegra-smmu.c static void tegra_smmu_domain_free(struct iommu_domain *domain) · line 339

Quoted 3 lines
drivers/iommu/tegra-smmu.c +1 -1
 {
-	const struct tegra_smmu_swgroup *group = NULL;
+	const struct tegra_smmu_swgroup *swgrp = NULL;

I don't think the existing naming is confusing. The variable name "group" is consistently used for tegra_smmu_swgroup structures and there are no cases where we would confuse them with struct tegra_smmu_group instances.

If we don't rename it, then PATCH-4 adds to struct tegra_smmu_group a "struct tegra_smmu_swgroup *group", which results in a confusing group->group...

Quoted 2 lines

However, I don't feel strongly about it, so I'm fine with changing the variable names to "swgrp" if you think that makes things less confusing.

Yea, I'd like to keep this change. I will respin it in next version after fixing other comments.
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#9 lore raw

PATCH v6 4/6 iommu/tegra-smmu: Use swgrp pointer instead of swgroup id

In reply to Nicolin Chen

This patch changes in struct tegra_smmu_group to use swgrp pointer instead of swgroup, as a preparational change for the "mappings" debugfs feature.

---
 drivers/iommu/tegra-smmu.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
drivers/iommu/tegra-smmu.c +8 -4
index 0f3883045ffa..8fd4985ac91e 100644
@@ -23,8 +23,8 @@ struct tegra_smmu_group {
 	struct list_head list;
 	struct tegra_smmu *smmu;
 	const struct tegra_smmu_group_soc *group_soc;
+	const struct tegra_smmu_swgroup *swgrp;
 	struct iommu_group *grp;
-	unsigned int swgroup;
 };
 
 struct tegra_smmu {
@@ -897,18 +897,22 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
 	struct tegra_smmu *smmu = dev_iommu_priv_get(dev);
 	const struct tegra_smmu_group_soc *group_soc;
+	const struct tegra_smmu_swgroup *swgrp;
 	unsigned int swgroup = fwspec->ids[0];
 	struct tegra_smmu_group *group;
 	struct iommu_group *grp;
 
+	/* Find swgrp according to the swgroup id */
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+
 	/* Find group_soc associating with swgroup */
 	group_soc = tegra_smmu_find_group_soc(smmu, swgroup);
 
 	mutex_lock(&smmu->lock);
 
-	/* Find existing iommu_group associating with swgroup or group_soc */
+	/* Find existing iommu_group associating with swgrp or group_soc */
 	list_for_each_entry(group, &smmu->groups, list)
-		if ((group->swgroup == swgroup) ||
+		if ((swgrp && group->swgrp == swgrp) ||
 		    (group_soc && group->group_soc == group_soc)) {
 			grp = iommu_group_ref_get(group->grp);
 			mutex_unlock(&smmu->lock);
@@ -923,7 +927,7 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 
 	INIT_LIST_HEAD(&group->list);
 	group->group_soc = group_soc;
-	group->swgroup = swgroup;
+	group->swgrp = swgrp;
 	group->smmu = smmu;
 
 	if (dev_is_pci(dev))
Signature · 1 line
2.17.1
Acked-by on v6 4/6
#10 lore raw

Re: [PATCH v6 4/6] iommu/tegra-smmu: Use swgrp pointer instead of swgroup id

Quoted 8 lines · Nicolin Chen

This patch changes in struct tegra_smmu_group to use swgrp pointer instead of swgroup, as a preparational change for the "mappings" debugfs feature.

---
 drivers/iommu/tegra-smmu.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

Seems fine:

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#11 lore raw

PATCH v6 5/6 iommu/tegra-smmu: Attach as pointer to tegra_smmu_group

In reply to Nicolin Chen

This could ease driver to access corresponding as pointer when having tegra_smmu_group pointer only, which can help new mappings debugfs nodes.

Also moving tegra_smmu_find_group_soc() upward, for using it in new tegra_smmu_attach_as(); and it's better to have all tegra_smmu_find_* functions together.

---
 drivers/iommu/tegra-smmu.c | 94 +++++++++++++++++++++++++++++++-------
 1 file changed, 78 insertions(+), 16 deletions(-)
drivers/iommu/tegra-smmu.c +78 -16
index 8fd4985ac91e..68c34a4a0ecc 100644
@@ -24,6 +24,7 @@ struct tegra_smmu_group {
 	struct tegra_smmu *smmu;
 	const struct tegra_smmu_group_soc *group_soc;
 	const struct tegra_smmu_swgroup *swgrp;
+	struct tegra_smmu_as *as;
 	struct iommu_group *grp;
 };
 
@@ -349,6 +350,19 @@ tegra_smmu_find_swgrp(struct tegra_smmu *smmu, unsigned int swgroup)
 	return swgrp;
 }
 
+static const struct tegra_smmu_group_soc *
+tegra_smmu_find_group_soc(struct tegra_smmu *smmu, unsigned int swgroup)
+{
+	unsigned int i, j;
+
+	for (i = 0; i < smmu->soc->num_groups; i++)
+		for (j = 0; j < smmu->soc->groups[i].num_swgroups; j++)
+			if (smmu->soc->groups[i].swgroups[j] == swgroup)
+				return &smmu->soc->groups[i];
+
+	return NULL;
+}
+
 static void tegra_smmu_enable(struct tegra_smmu *smmu, unsigned int swgroup,
 			      unsigned int asid)
 {
@@ -482,6 +496,57 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,
 	mutex_unlock(&smmu->lock);
 }
 
+static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
+				 struct tegra_smmu_as *as,
+				 unsigned int swgroup)
+{
+	const struct tegra_smmu_swgroup *swgrp;
+	struct tegra_smmu_group *group;
+
+	/* Find swgrp according to the swgroup id */
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+	if (!swgrp)
+		return;
+
+	mutex_lock(&smmu->lock);
+
+	list_for_each_entry(group, &smmu->groups, list) {
+		if (group->swgrp != swgrp)
+			continue;
+		if (group->as)
+			dev_warn(smmu->dev,
+				 "overwriting group->as for swgroup: %s\n", swgrp->name);
+		group->as = as;
+		break;
+	}
+
+	mutex_unlock(&smmu->lock);
+}
+
+static void tegra_smmu_detach_as(struct tegra_smmu *smmu,
+				 unsigned int swgroup)
+{
+	const struct tegra_smmu_swgroup *swgrp;
+	struct tegra_smmu_group *group;
+	struct dentry *d;
+
+	/* Find swgrp according to the swgroup id */
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+	if (!swgrp)
+		return;
+
+	mutex_lock(&smmu->lock);
+
+	list_for_each_entry(group, &smmu->groups, list) {
+		if (group->swgrp != swgrp)
+			continue;
+		group->as = NULL;
+		break;
+	}
+
+	mutex_unlock(&smmu->lock);
+}
+
 static int tegra_smmu_attach_dev(struct iommu_domain *domain,
 				 struct device *dev)
 {
@@ -495,11 +560,15 @@ static int tegra_smmu_attach_dev(struct iommu_domain *domain,
 		return -ENOENT;
 
 	for (index = 0; index < fwspec->num_ids; index++) {
+		unsigned int swgroup = fwspec->ids[index];
+
 		err = tegra_smmu_as_prepare(smmu, as);
 		if (err)
 			goto disable;
 
-		tegra_smmu_enable(smmu, fwspec->ids[index], as->id);
+		tegra_smmu_attach_as(smmu, as, swgroup);
+
+		tegra_smmu_enable(smmu, swgroup, as->id);
 	}
 
 	if (index == 0)
@@ -509,7 +578,10 @@ static int tegra_smmu_attach_dev(struct iommu_domain *domain,
 
 disable:
 	while (index--) {
-		tegra_smmu_disable(smmu, fwspec->ids[index], as->id);
+		unsigned int swgroup = fwspec->ids[index];
+
+		tegra_smmu_disable(smmu, swgroup, as->id);
+		tegra_smmu_detach_as(smmu, swgroup);
 		tegra_smmu_as_unprepare(smmu, as);
 	}
 
@@ -527,7 +599,10 @@ static void tegra_smmu_detach_dev(struct iommu_domain *domain, struct device *de
 		return;
 
 	for (index = 0; index < fwspec->num_ids; index++) {
-		tegra_smmu_disable(smmu, fwspec->ids[index], as->id);
+		unsigned int swgroup = fwspec->ids[index];
+
+		tegra_smmu_disable(smmu, swgroup, as->id);
+		tegra_smmu_detach_as(smmu, swgroup);
 		tegra_smmu_as_unprepare(smmu, as);
 	}
 }
@@ -869,19 +944,6 @@ static struct iommu_device *tegra_smmu_probe_device(struct device *dev)
 
 static void tegra_smmu_release_device(struct device *dev) {}
 
-static const struct tegra_smmu_group_soc *
-tegra_smmu_find_group_soc(struct tegra_smmu *smmu, unsigned int swgroup)
-{
-	unsigned int i, j;
-
-	for (i = 0; i < smmu->soc->num_groups; i++)
-		for (j = 0; j < smmu->soc->groups[i].num_swgroups; j++)
-			if (smmu->soc->groups[i].swgroups[j] == swgroup)
-				return &smmu->soc->groups[i];
-
-	return NULL;
-}
-
 static void tegra_smmu_group_release(void *iommu_data)
 {
 	struct tegra_smmu_group *group = iommu_data;
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Acked-by on v6 5/6
#12 lore raw

Re: [PATCH v6 5/6] iommu/tegra-smmu: Attach as pointer to tegra_smmu_group

Quoted 4 lines · Nicolin Chen

This could ease driver to access corresponding as pointer when having tegra_smmu_group pointer only, which can help new mappings debugfs nodes.

Quoted 8 lines

Also moving tegra_smmu_find_group_soc() upward, for using it in new tegra_smmu_attach_as(); and it's better to have all tegra_smmu_find_* functions together.

---
 drivers/iommu/tegra-smmu.c | 94 +++++++++++++++++++++++++++++++-------
 1 file changed, 78 insertions(+), 16 deletions(-)
#13 lore raw

PATCH v6 6/6 iommu/tegra-smmu: Add pagetable mappings to debugfs

In reply to Nicolin Chen

This patch dumps all active mapping entries from pagetable to a debugfs directory named "mappings".

Attaching an example:

SWGROUP: hc
as->id: 0
as->attr: R|W|N
as->pd_dma: 0x0000000080c03000

{
        [index: 1023] 0xf0080c3e (count: 2)
        {
                PTE RANGE      | ATTR | PHYS               | IOVA               | SIZE
                [#1022, #1023] | 0x5  | 0x000000010bbf1000 | 0x00000000ffffe000 | 0x2000
        }
}

Total PDE count: 1
Total PTE count: 2

---
 drivers/iommu/tegra-smmu.c | 145 +++++++++++++++++++++++++++++++++++++
 1 file changed, 145 insertions(+)
drivers/iommu/tegra-smmu.c +145 -0
index 68c34a4a0ecc..aac977e181f6 100644
@@ -46,6 +46,7 @@ struct tegra_smmu {
 	struct list_head list;
 
 	struct dentry *debugfs;
+	struct dentry *debugfs_mappings;
 
 	struct iommu_device iommu;	/* IOMMU Core code handle */
 };
@@ -153,6 +154,9 @@ static inline u32 smmu_readl(struct tegra_smmu *smmu, unsigned long offset)
 
 #define SMMU_PDE_ATTR		(SMMU_PDE_READABLE | SMMU_PDE_WRITABLE | \
 				 SMMU_PDE_NONSECURE)
+#define SMMU_PTE_ATTR		(SMMU_PTE_READABLE | SMMU_PTE_WRITABLE | \
+				 SMMU_PTE_NONSECURE)
+#define SMMU_PTE_ATTR_SHIFT	29
 
 static unsigned int iova_pd_index(unsigned long iova)
 {
@@ -164,6 +168,12 @@ static unsigned int iova_pt_index(unsigned long iova)
 	return (iova >> SMMU_PTE_SHIFT) & (SMMU_NUM_PTE - 1);
 }
 
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return ((dma_addr_t)pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       ((dma_addr_t)pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}
+
 static bool smmu_dma_addr_valid(struct tegra_smmu *smmu, dma_addr_t addr)
 {
 	addr >>= 12;
@@ -496,6 +506,8 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,
 	mutex_unlock(&smmu->lock);
 }
 
+static const struct file_operations tegra_smmu_debugfs_mappings_fops;
+
 static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
 				 struct tegra_smmu_as *as,
 				 unsigned int swgroup)
@@ -517,6 +529,12 @@ static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
 			dev_warn(smmu->dev,
 				 "overwriting group->as for swgroup: %s\n", swgrp->name);
 		group->as = as;
+
+		if (smmu->debugfs_mappings)
+			debugfs_create_file(group->swgrp->name, 0444,
+					    smmu->debugfs_mappings, group,
+					    &tegra_smmu_debugfs_mappings_fops);
+
 		break;
 	}
 
@@ -541,6 +559,12 @@ static void tegra_smmu_detach_as(struct tegra_smmu *smmu,
 		if (group->swgrp != swgrp)
 			continue;
 		group->as = NULL;
+
+		if (smmu->debugfs_mappings) {
+			d = debugfs_lookup(group->swgrp->name, smmu->debugfs_mappings);
+			debugfs_remove(d);
+		}
+
 		break;
 	}
 
@@ -1124,6 +1148,125 @@ static int tegra_smmu_clients_show(struct seq_file *s, void *data)
 
 DEFINE_SHOW_ATTRIBUTE(tegra_smmu_clients);
 
+static int tegra_smmu_debugfs_mappings_show(struct seq_file *s, void *data)
+{
+	struct tegra_smmu_group *group = s->private;
+	const struct tegra_smmu_swgroup *swgrp;
+	struct tegra_smmu_as *as;
+	struct tegra_smmu *smmu;
+	unsigned int pd_index;
+	unsigned int pt_index;
+	unsigned long flags;
+	u64 pte_count = 0;
+	u32 pde_count = 0;
+	u32 *pd, val;
+
+	if (!group || !group->as || !group->swgrp)
+		return 0;
+
+	swgrp = group->swgrp;
+	smmu = group->smmu;
+	as = group->as;
+
+	mutex_lock(&smmu->lock);
+
+	val = smmu_readl(smmu, swgrp->reg) & SMMU_ASID_ENABLE;
+	if (!val)
+		goto unlock;
+
+	pd = page_address(as->pd);
+	if (!pd)
+		goto unlock;
+
+	seq_printf(s, "\nSWGROUP: %s\n", swgrp->name);
+	seq_printf(s, "as->id: %d\nas->attr: %c|%c|%s\nas->pd_dma: %pad\n", as->id,
+		   as->attr & SMMU_PD_READABLE ? 'R' : '-',
+		   as->attr & SMMU_PD_WRITABLE ? 'W' : '-',
+		   as->attr & SMMU_PD_NONSECURE ? "NS" : "S",
+		   &as->pd_dma);
+	seq_puts(s, "{\n");
+
+	spin_lock_irqsave(&as->lock, flags);
+
+	for (pd_index = 0; pd_index < SMMU_NUM_PDE; pd_index++) {
+		struct page *pt_page;
+		unsigned int i;
+		u32 *addr;
+
+		/* An empty PDE should not have a pte use count */
+		WARN_ON_ONCE(!pd[pd_index] ^ !as->count[pd_index]);
+
+		/* Skip this empty PDE */
+		if (!pd[pd_index])
+			continue;
+
+		pde_count++;
+		pte_count += as->count[pd_index];
+		seq_printf(s, "\t[index: %u] 0x%x (count: %d)\n",
+			   pd_index, pd[pd_index], as->count[pd_index]);
+		pt_page = as->pts[pd_index];
+		addr = page_address(pt_page);
+
+		seq_puts(s, "\t{\n");
+		seq_printf(s, "\t\t%-14s | %-4s | %-10s%s | %-10s%s | %-11s\n",
+			   "PTE RANGE", "ATTR",
+			   "PHYS", sizeof(phys_addr_t) > 4 ? "        " : "",
+			   "IOVA", sizeof(dma_addr_t)  > 4 ? "        " : "",
+			   "SIZE");
+		for (pt_index = 0; pt_index < SMMU_NUM_PTE; pt_index += i) {
+			size_t size = SMMU_SIZE_PT;
+			dma_addr_t iova;
+			phys_addr_t pa;
+
+			i = 1;
+
+			if (!addr[pt_index])
+				continue;
+
+			iova = pd_pt_index_iova(pd_index, pt_index);
+			pa = SMMU_PFN_PHYS(addr[pt_index] & ~SMMU_PTE_ATTR);
+
+			/* Check contiguous mappings and increase size */
+			while (pt_index + i < SMMU_NUM_PTE) {
+				dma_addr_t next_iova;
+				phys_addr_t next_pa;
+
+				if (!addr[pt_index + i])
+					break;
+
+				next_iova = pd_pt_index_iova(pd_index, pt_index + i);
+				next_pa = SMMU_PFN_PHYS(addr[pt_index + i] & ~SMMU_PTE_ATTR);
+
+				/* Break at the end of a linear mapping */
+				if ((next_iova - iova != SMMU_SIZE_PT * i) ||
+				    (next_pa - pa != SMMU_SIZE_PT * i))
+					break;
+
+				i++;
+			}
+
+			seq_printf(s, "\t\t[#%-4u, #%-4u] | 0x%-2x | %pa | %pad | 0x%-9zx\n",
+				   pt_index, pt_index + i - 1,
+				   addr[pt_index] >> SMMU_PTE_ATTR_SHIFT,
+				   &pa, &iova, size * i);
+		}
+		seq_puts(s, "\t}\n");
+	}
+
+	spin_unlock_irqrestore(&as->lock, flags);
+
+	seq_puts(s, "}\n");
+	seq_printf(s, "Total PDE count: %u\n", pde_count);
+	seq_printf(s, "Total PTE count: %llu\n", pte_count);
+
+unlock:
+	mutex_unlock(&smmu->lock);
+
+	return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(tegra_smmu_debugfs_mappings);
+
 static void tegra_smmu_debugfs_init(struct tegra_smmu *smmu)
 {
 	smmu->debugfs = debugfs_create_dir("smmu", NULL);
@@ -1134,6 +1277,8 @@ static void tegra_smmu_debugfs_init(struct tegra_smmu *smmu)
 			    &tegra_smmu_swgroups_fops);
 	debugfs_create_file("clients", S_IRUGO, smmu->debugfs, smmu,
 			    &tegra_smmu_clients_fops);
+
+	smmu->debugfs_mappings = debugfs_create_dir("mappings", smmu->debugfs);
 }
 
 static void tegra_smmu_debugfs_exit(struct tegra_smmu *smmu)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#14 lore raw

14.09.2021 04:38, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static unsigned int iova_pt_index(unsigned long iova) · line 175

Quoted 5 lines
drivers/iommu/tegra-smmu.c +5 -0
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return ((dma_addr_t)pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       ((dma_addr_t)pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}

We know that IOVA is fixed to u32 for this controller. Can we avoid all these dma_addr_t castings? It should make code cleaner a tad, IMO.

#15 lore raw
Quoted 9 lines · Dmitry Osipenko

14.09.2021 04:38, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static unsigned int iova_pt_index(unsigned long iova) · line 175

Quoted 5 lines
drivers/iommu/tegra-smmu.c +5 -0
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return ((dma_addr_t)pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       ((dma_addr_t)pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}

We know that IOVA is fixed to u32 for this controller. Can we avoid all these dma_addr_t castings? It should make code cleaner a tad, IMO.

Tegra210 actually supports 34-bit IOVA...

#16 lore raw

14.09.2021 21:49, Nicolin Chen пишет:

Quoted 11 lines · Dmitry Osipenko
Quoted 9 lines

14.09.2021 04:38, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static unsigned int iova_pt_index(unsigned long iova) · line 175

Quoted 5 lines
drivers/iommu/tegra-smmu.c +5 -0
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return ((dma_addr_t)pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       ((dma_addr_t)pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}

We know that IOVA is fixed to u32 for this controller. Can we avoid all these dma_addr_t castings? It should make code cleaner a tad, IMO.

Quoted 2 lines

Tegra210 actually supports 34-bit IOVA...

It doesn't. 34-bit is PA, 32-bit is VA.

Quote from T210 TRM:

"The SMMU is a centralized virtual-to-physical translation for MSS. It maps a 32-bit virtual address to a 34-bit physical address. If the client address is 40 bits then bits 39:32 are ignored."

Even if it supported more than 32bit, then the returned ulong is 32bit, which doesn't make sense.
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#17 lore raw
Quoted 15 lines · Dmitry Osipenko

14.09.2021 21:49, Nicolin Chen пишет:

Quoted 11 lines · Dmitry Osipenko
Quoted 9 lines

14.09.2021 04:38, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static unsigned int iova_pt_index(unsigned long iova) · line 175

Quoted 5 lines
drivers/iommu/tegra-smmu.c +5 -0
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return ((dma_addr_t)pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       ((dma_addr_t)pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}

We know that IOVA is fixed to u32 for this controller. Can we avoid all these dma_addr_t castings? It should make code cleaner a tad, IMO.

Quoted 2 lines

Tegra210 actually supports 34-bit IOVA...

Quoted 7 lines

It doesn't. 34-bit is PA, 32-bit is VA.

Quote from T210 TRM:

"The SMMU is a centralized virtual-to-physical translation for MSS. It maps a 32-bit virtual address to a 34-bit physical address. If the client address is 40 bits then bits 39:32 are ignored."

If you scroll down by a couple of sections, you can see 34-bit virtual addresses in section 18.6.1.2; and if checking one ASID register, you can see it mention the extra two bits va[33:32].

However, the driver currently sets its geometry.aperture_end to 32-bit, and we can only get 32-bit IOVAs using PDE and PTE only, so I think it should be safe to remove the castings here. I'll wait for a couple of days and see if there'd be other comments for me to address in next version.

Quoted 2 lines

Even if it supported more than 32bit, then the returned ulong is 32bit, which doesn't make sense.

On ARM64 (Tegra210), isn't ulong 64-bit?

#18 lore raw

15.09.2021 07:38, Nicolin Chen пишет:

Quoted 20 lines · Dmitry Osipenko
Quoted 19 lines

14.09.2021 21:49, Nicolin Chen пишет:

Quoted 11 lines · Dmitry Osipenko
Quoted 9 lines

14.09.2021 04:38, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static unsigned int iova_pt_index(unsigned long iova) · line 175

Quoted 5 lines
drivers/iommu/tegra-smmu.c +5 -0
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return ((dma_addr_t)pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       ((dma_addr_t)pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}

We know that IOVA is fixed to u32 for this controller. Can we avoid all these dma_addr_t castings? It should make code cleaner a tad, IMO.

Quoted 2 lines

Tegra210 actually supports 34-bit IOVA...

It doesn't. 34-bit is PA, 32-bit is VA.

Quote from T210 TRM:

Quoted 7 lines
Quoted 3 lines

"The SMMU is a centralized virtual-to-physical translation for MSS. It maps a 32-bit virtual address to a 34-bit physical address. If the client address is 40 bits then bits 39:32 are ignored."

If you scroll down by a couple of sections, you can see 34-bit virtual addresses in section 18.6.1.2; and if checking one ASID register, you can see it mention the extra two bits va[33:32].

Thanks for the pointer. It says that only certain memory clients allow to combine 4 ASIDs to form 34bit VA space. In this case the PA space is split into 4GB areas and there are additional bitfields which configure the ASID mapping of each 4GB area. Still each ASID is 32bit.

This is what TRM says:

"For the GPU and other clients with 34-bit address interfaces, the ASID registers are extended to point to four ASIDs. The SMMU supports 4GB of virtual address space per ASID, so mapping addr[33:32] into ASID[1:0] extends the virtual address space of a client to 16GB."

Quoted 5 lines

However, the driver currently sets its geometry.aperture_end to 32-bit, and we can only get 32-bit IOVAs using PDE and PTE only, so I think it should be safe to remove the castings here. I'll wait for a couple of days and see if there'd be other comments for me to address in next version.

You will need to read the special "ASID Assignment Register" which supports 4 sub-ASIDs to translate the PA address into the actual VA. By default all clients are limited to a single ASID and upstream kernel doesn't support programming of 34bit VAs. So doesn't worth the effort to fully translate the VA, IMO.

Quoted 4 lines
Quoted 2 lines

Even if it supported more than 32bit, then the returned ulong is 32bit, which doesn't make sense.

On ARM64 (Tegra210), isn't ulong 64-bit?

Yes, indeed.

#19 lore raw

15.09.2021 15:09, Dmitry Osipenko пишет:

Quoted 41 lines

15.09.2021 07:38, Nicolin Chen пишет:

Quoted 20 lines · Dmitry Osipenko
Quoted 19 lines

14.09.2021 21:49, Nicolin Chen пишет:

Quoted 11 lines · Dmitry Osipenko
Quoted 9 lines

14.09.2021 04:38, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static unsigned int iova_pt_index(unsigned long iova) · line 175

Quoted 5 lines
drivers/iommu/tegra-smmu.c +5 -0
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return ((dma_addr_t)pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       ((dma_addr_t)pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}

We know that IOVA is fixed to u32 for this controller. Can we avoid all these dma_addr_t castings? It should make code cleaner a tad, IMO.

Quoted 2 lines

Tegra210 actually supports 34-bit IOVA...

It doesn't. 34-bit is PA, 32-bit is VA.

Quote from T210 TRM:

Quoted 7 lines
Quoted 3 lines

"The SMMU is a centralized virtual-to-physical translation for MSS. It maps a 32-bit virtual address to a 34-bit physical address. If the client address is 40 bits then bits 39:32 are ignored."

If you scroll down by a couple of sections, you can see 34-bit virtual addresses in section 18.6.1.2; and if checking one ASID register, you can see it mention the extra two bits va[33:32].

Thanks for the pointer. It says that only certain memory clients allow to combine 4 ASIDs to form 34bit VA space. In this case the PA space is split into 4GB areas and there are additional bitfields which configure the ASID mapping of each 4GB area. Still each ASID is 32bit.

This is what TRM says:

"For the GPU and other clients with 34-bit address interfaces, the ASID registers are extended to point to four ASIDs. The SMMU supports 4GB of virtual address space per ASID, so mapping addr[33:32] into ASID[1:0] extends the virtual address space of a client to 16GB."

Quoted 8 lines
Quoted 5 lines

However, the driver currently sets its geometry.aperture_end to 32-bit, and we can only get 32-bit IOVAs using PDE and PTE only, so I think it should be safe to remove the castings here. I'll wait for a couple of days and see if there'd be other comments for me to address in next version.

You will need to read the special "ASID Assignment Register" which supports 4 sub-ASIDs to translate the PA address into the actual VA. By

* VA to PA

Quoted 11 lines

default all clients are limited to a single ASID and upstream kernel doesn't support programming of 34bit VAs. So doesn't worth the effort to fully translate the VA, IMO.

Quoted 4 lines
Quoted 2 lines

Even if it supported more than 32bit, then the returned ulong is 32bit, which doesn't make sense.

On ARM64 (Tegra210), isn't ulong 64-bit?

Yes, indeed.

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#20 lore raw
In reply to Dmitry Osipenko
Quoted 25 lines · Dmitry Osipenko

15.09.2021 07:38, Nicolin Chen пишет:

Quoted 24 lines · Dmitry Osipenko
Quoted 22 lines

14.09.2021 21:49, Nicolin Chen пишет:

Quoted 11 lines · Dmitry Osipenko
Quoted 9 lines

14.09.2021 04:38, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static unsigned int iova_pt_index(unsigned long iova) · line 175

Quoted 5 lines
drivers/iommu/tegra-smmu.c +5 -0
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return ((dma_addr_t)pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       ((dma_addr_t)pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}

We know that IOVA is fixed to u32 for this controller. Can we avoid all these dma_addr_t castings? It should make code cleaner a tad, IMO.

Quoted 2 lines

Tegra210 actually supports 34-bit IOVA...

It doesn't. 34-bit is PA, 32-bit is VA.

Quote from T210 TRM:

"The SMMU is a centralized virtual-to-physical translation for MSS. It maps a 32-bit virtual address to a 34-bit physical address. If the client address is 40 bits then bits 39:32 are ignored."

Quoted 8 lines
Quoted 3 lines

If you scroll down by a couple of sections, you can see 34-bit virtual addresses in section 18.6.1.2; and if checking one ASID register, you can see it mention the extra two bits va[33:32].

Thanks for the pointer. It says that only certain memory clients allow to combine 4 ASIDs to form 34bit VA space. In this case the PA space is split into 4GB areas and there are additional bitfields which configure the ASID mapping of each 4GB area. Still each ASID is 32bit.

True.

Quoted 13 lines

This is what TRM says:

"For the GPU and other clients with 34-bit address interfaces, the ASID registers are extended to point to four ASIDs. The SMMU supports 4GB of virtual address space per ASID, so mapping addr[33:32] into ASID[1:0] extends the virtual address space of a client to 16GB."

Quoted 5 lines

However, the driver currently sets its geometry.aperture_end to 32-bit, and we can only get 32-bit IOVAs using PDE and PTE only, so I think it should be safe to remove the castings here. I'll wait for a couple of days and see if there'd be other comments for me to address in next version.

Quoted 5 lines

You will need to read the special "ASID Assignment Register" which supports 4 sub-ASIDs to translate the PA address into the actual VA. By default all clients are limited to a single ASID and upstream kernel doesn't support programming of 34bit VAs. So doesn't worth the effort to fully translate the VA, IMO.

Yea. It'd be easier to just stay in 32-bit. I will remove those castings in the next version, waiting for Thierry taking a look at this v6 first.

#21 lore raw
In reply to Nicolin Chen
Quoted 60 lines · Nicolin Chen

This patch dumps all active mapping entries from pagetable to a debugfs directory named "mappings".

Attaching an example:

SWGROUP: hc
as->id: 0
as->attr: R|W|N
as->pd_dma: 0x0000000080c03000

{
        [index: 1023] 0xf0080c3e (count: 2)
        {
                PTE RANGE      | ATTR | PHYS               | IOVA               | SIZE
                [#1022, #1023] | 0x5  | 0x000000010bbf1000 | 0x00000000ffffe000 | 0x2000
        }
}

Total PDE count: 1
Total PTE count: 2

---
 drivers/iommu/tegra-smmu.c | 145 +++++++++++++++++++++++++++++++++++++
 1 file changed, 145 insertions(+)
drivers/iommu/tegra-smmu.c +10 -0
index 68c34a4a0ecc..aac977e181f6 100644
@@ -46,6 +46,7 @@ struct tegra_smmu {
 	struct list_head list;
 
 	struct dentry *debugfs;
+	struct dentry *debugfs_mappings;
 
 	struct iommu_device iommu;	/* IOMMU Core code handle */
 };
@@ -153,6 +154,9 @@ static inline u32 smmu_readl(struct tegra_smmu *smmu, unsigned long offset)
 
 #define SMMU_PDE_ATTR		(SMMU_PDE_READABLE | SMMU_PDE_WRITABLE | \
 				 SMMU_PDE_NONSECURE)
+#define SMMU_PTE_ATTR		(SMMU_PTE_READABLE | SMMU_PTE_WRITABLE | \
+				 SMMU_PTE_NONSECURE)
+#define SMMU_PTE_ATTR_SHIFT	29
 
 static unsigned int iova_pd_index(unsigned long iova)
 {
@@ -164,6 +168,12 @@ static unsigned int iova_pt_index(unsigned long iova)
 	return (iova >> SMMU_PTE_SHIFT) & (SMMU_NUM_PTE - 1);
 }
 
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return ((dma_addr_t)pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       ((dma_addr_t)pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}
+
 static bool smmu_dma_addr_valid(struct tegra_smmu *smmu, dma_addr_t addr)
 {
 	addr >>= 12;

drivers/iommu/tegra-smmu.c static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu, · line 509

Quoted 4 lines
drivers/iommu/tegra-smmu.c +1 -0
@@ -496,6 +506,8 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,
 	mutex_unlock(&smmu->lock);
 }
 
+static const struct file_operations tegra_smmu_debugfs_mappings_fops;

Could the implementation be moved up here to avoid the forward declaration?

Quoted 61 lines
drivers/iommu/tegra-smmu.c +40 -0
+
 static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
 				 struct tegra_smmu_as *as,
 				 unsigned int swgroup)
@@ -517,6 +529,12 @@ static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
 			dev_warn(smmu->dev,
 				 "overwriting group->as for swgroup: %s\n", swgrp->name);
 		group->as = as;
+
+		if (smmu->debugfs_mappings)
+			debugfs_create_file(group->swgrp->name, 0444,
+					    smmu->debugfs_mappings, group,
+					    &tegra_smmu_debugfs_mappings_fops);
+
 		break;
 	}
 
@@ -541,6 +559,12 @@ static void tegra_smmu_detach_as(struct tegra_smmu *smmu,
 		if (group->swgrp != swgrp)
 			continue;
 		group->as = NULL;
+
+		if (smmu->debugfs_mappings) {
+			d = debugfs_lookup(group->swgrp->name, smmu->debugfs_mappings);
+			debugfs_remove(d);
+		}
+
 		break;
 	}
 
@@ -1124,6 +1148,125 @@ static int tegra_smmu_clients_show(struct seq_file *s, void *data)
 
 DEFINE_SHOW_ATTRIBUTE(tegra_smmu_clients);
 
+static int tegra_smmu_debugfs_mappings_show(struct seq_file *s, void *data)
+{
+	struct tegra_smmu_group *group = s->private;
+	const struct tegra_smmu_swgroup *swgrp;
+	struct tegra_smmu_as *as;
+	struct tegra_smmu *smmu;
+	unsigned int pd_index;
+	unsigned int pt_index;
+	unsigned long flags;
+	u64 pte_count = 0;
+	u32 pde_count = 0;
+	u32 *pd, val;
+
+	if (!group || !group->as || !group->swgrp)
+		return 0;
+
+	swgrp = group->swgrp;
+	smmu = group->smmu;
+	as = group->as;
+
+	mutex_lock(&smmu->lock);
+
+	val = smmu_readl(smmu, swgrp->reg) & SMMU_ASID_ENABLE;
+	if (!val)
+		goto unlock;
+
+	pd = page_address(as->pd);

drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1187

Quoted 10 lines
drivers/iommu/tegra-smmu.c +10 -0
+	if (!pd)
+		goto unlock;
+
+	seq_printf(s, "\nSWGROUP: %s\n", swgrp->name);
+	seq_printf(s, "as->id: %d\nas->attr: %c|%c|%s\nas->pd_dma: %pad\n", as->id,
+		   as->attr & SMMU_PD_READABLE ? 'R' : '-',
+		   as->attr & SMMU_PD_WRITABLE ? 'W' : '-',
+		   as->attr & SMMU_PD_NONSECURE ? "NS" : "S",
+		   &as->pd_dma);
+	seq_puts(s, "{\n");

Maybe this can be more compact by putting the name, ID, attributes and base address onto a single line? Maybe also use "'-' : 'S'" for the non-secure attribute to keep in line with what you've done for readable and writable attributes.

Then again, this is going to be very verbose output anyway, so maybe it isn't worth it.

Thierry

#22 lore raw
Quoted 8 lines · Thierry Reding

drivers/iommu/tegra-smmu.c static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu, · line 509

Quoted 5 lines
drivers/iommu/tegra-smmu.c +1 -0
@@ -496,6 +506,8 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,
 	mutex_unlock(&smmu->lock);
 }
 
+static const struct file_operations tegra_smmu_debugfs_mappings_fops;

Could the implementation be moved up here to avoid the forward declaration?

I thought that keeping all debugfs fops together would be preferable. But yes, I will move it if you prefer no-additional forward declare.

Quoted 4 lines
Quoted 4 lines
drivers/iommu/tegra-smmu.c +4 -0
+	seq_printf(s, "\nSWGROUP: %s\n", swgrp->name);
+	seq_printf(s, "as->id: %d\nas->attr: %c|%c|%s\nas->pd_dma: %pad\n", as->id,
+		   as->attr & SMMU_PD_READABLE ? 'R' : '-',
+		   as->attr & SMMU_PD_WRITABLE ? 'W' : '-',
Quoted 8 lines

drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1187

Quoted 3 lines
drivers/iommu/tegra-smmu.c +3 -0
+		   as->attr & SMMU_PD_NONSECURE ? "NS" : "S",
+		   &as->pd_dma);
+	seq_puts(s, "{\n");

Maybe this can be more compact by putting the name, ID, attributes and base address onto a single line? Maybe also use "'-' : 'S'" for the non-secure attribute to keep in line with what you've done for readable and writable attributes.

Okay. Will change that.

Quoted 2 lines

Then again, this is going to be very verbose output anyway, so maybe it isn't worth it.

Are you saying the whole debugfs thing or just attributes? Yet, for either case, I don't think so, as mappings info would help for sure from our past experience while the attributes are just one line...
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu