Skip to contentSkip to conversation

iommu/tegra-smmu: Add pagetable mappings to debugfs

PATCH Superseded Nicolin Chen 2 participants 11 messages Dec 8, 2021

Subsystems (3)

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v7 cover iommu/tegra-smmu: Add pagetable mappings to debugfs

This series of patches adds a new mappings node to debugfs for tegra-smmu driver. The first five patches are all preparational changes for PATCH-6, based on Thierry's review feedback against v5.

Changelog
v7:
* Added "Acked-by" from Thierry to PATCH1,4,5
* No other changes for PATCH1,3,4,5
* PATCH-2: dropped "s/soc/group_soc" change
* PATCH-6:
* * avoided forward declaration
* * dropped castings in pd_pt_index_iova()
* * used "'-' : 'S'" for non-secure attribute
* * changed multi-line outputs to single-line format
v6: https://lore.kernel.org/linux-iommu/20210915043806.GA19185@Asurada-Nvidia/t/
* Added PATCH1-3 for better naming conventions
* Added PATCH4-5 to embed previous struct tegra_smmu_group_debug
into struct tegra_smmu_group
* Dropped parentheses at SMMU_PTE_ATTR_SHIFT
* Dropped swgrp->reg print
* Replaced ptb_reg contents with as->attr and as->pd_dma
* Added "index" and "count" in the PD entries for readability
* Removed Dmitry's Tested-by and Reviewed-by for the big change
from v5 to v6.
v5: https://lore.kernel.org/linux-iommu/[email protected]/
* Fixed a typo in commit message
* Split a long line into two lines
* Rearranged variable defines by length
* Added Tested-by and Reviewed-by from Dmitry
v4: https://lore.kernel.org/lkml/[email protected]/
* Changed %d to %u for unsigned variables
* Fixed print format mismatch warnings on ARM32
v3: https://lore.kernel.org/linux-iommu/20210315031530.GA15245@Asurada-Nvidia/T/
* Fixed PHYS and IOVA print formats
* Changed variables to unsigned int type
* Changed the table outputs to be compact
v2: https://lore.kernel.org/linux-iommu/20210312010932.GB29926@Asurada-Nvidia/T/
* Expanded mutex range to the entire function
* Added as->lock to protect pagetable walkthrough
* Replaced devm_kzalloc with devm_kcalloc for group_debug
* Added "PTE RANGE" and "SIZE" columns to group contiguous mappings
* Dropped as->count check
* Added WARN_ON when as->count mismatches pd[pd_index]
v1: https://lkml.org/lkml/2020/9/26/70

Nicolin Chen (6):
iommu/tegra-smmu: Rename struct iommu_group *group to *grp iommu/tegra-smmu: Rename tegra_smmu_find_group to
tegra_smmu_find_group_soc
iommu/tegra-smmu: Rename struct tegra_smmu_swgroup *group to *swgrp iommu/tegra-smmu: Use swgrp pointer instead of swgroup id iommu/tegra-smmu: Attach as pointer to tegra_smmu_group iommu/tegra-smmu: Add pagetable mappings to debugfs

 drivers/iommu/tegra-smmu.c | 301 +++++++++++++++++++++++++++++++------
 1 file changed, 255 insertions(+), 46 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH v7 1/6 iommu/tegra-smmu: Rename struct iommu_group *group to *grp

There are a few structs using "group" for their pointer instances. This gets confusing sometimes. The instance of struct iommu_group is used in local function with an alias "grp", which can separate it from others.

So this patch simply renames "group" to "grp" as a cleanup.

---
 drivers/iommu/tegra-smmu.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
drivers/iommu/tegra-smmu.c +8 -8
index c5fa8b8673b6..f874ee2600e5 100644
@@ -23,7 +23,7 @@ struct tegra_smmu_group {
 	struct list_head list;
 	struct tegra_smmu *smmu;
 	const struct tegra_smmu_group_soc *soc;
-	struct iommu_group *group;
+	struct iommu_group *grp;
 	unsigned int swgroup;
 };
 
@@ -909,7 +909,7 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	/* Find existing iommu_group associating with swgroup or group_soc */
 	list_for_each_entry(group, &smmu->groups, list)
 		if ((group->swgroup == swgroup) || (soc && group->soc == soc)) {
-			grp = iommu_group_ref_get(group->group);
+			grp = iommu_group_ref_get(group->grp);
 			mutex_unlock(&smmu->lock);
 			return grp;
 		}
@@ -926,23 +926,23 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	group->soc = soc;
 
 	if (dev_is_pci(dev))
-		group->group = pci_device_group(dev);
+		group->grp = pci_device_group(dev);
 	else
-		group->group = generic_device_group(dev);
+		group->grp = generic_device_group(dev);
 
-	if (IS_ERR(group->group)) {
+	if (IS_ERR(group->grp)) {
 		devm_kfree(smmu->dev, group);
 		mutex_unlock(&smmu->lock);
 		return NULL;
 	}
 
-	iommu_group_set_iommudata(group->group, group, tegra_smmu_group_release);
+	iommu_group_set_iommudata(group->grp, group, tegra_smmu_group_release);
 	if (soc)
-		iommu_group_set_name(group->group, soc->name);
+		iommu_group_set_name(group->grp, soc->name);
 	list_add_tail(&group->list, &smmu->groups);
 	mutex_unlock(&smmu->lock);
 
-	return group->group;
+	return group->grp;
 }
 
 static int tegra_smmu_of_xlate(struct device *dev,
Signature · 1 line
2.17.1
#2 lore raw

PATCH v7 2/6 iommu/tegra-smmu: Rename tegra_smmu_find_group to tegra_smmu_find_group_soc

In reply to Nicolin Chen

The existing function tegra_smmu_find_group really finds group->soc pointer, so naming it "find_group" might not be clear by looking at it alone. This patch renames it to tegra_smmu_group_soc in order to disambiguate the use of "group" in this driver.

---
 drivers/iommu/tegra-smmu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/iommu/tegra-smmu.c +2 -2
index f874ee2600e5..9e9f37e4894c 100644
@@ -870,7 +870,7 @@ static struct iommu_device *tegra_smmu_probe_device(struct device *dev)
 static void tegra_smmu_release_device(struct device *dev) {}
 
 static const struct tegra_smmu_group_soc *
-tegra_smmu_find_group(struct tegra_smmu *smmu, unsigned int swgroup)
+tegra_smmu_find_group_soc(struct tegra_smmu *smmu, unsigned int swgroup)
 {
 	unsigned int i, j;
 
@@ -902,7 +902,7 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	struct iommu_group *grp;
 
 	/* Find group_soc associating with swgroup */
-	soc = tegra_smmu_find_group(smmu, swgroup);
+	soc = tegra_smmu_find_group_soc(smmu, swgroup);
 
 	mutex_lock(&smmu->lock);
 
Signature · 1 line
2.17.1
#3 lore raw

PATCH v7 3/6 iommu/tegra-smmu: Rename struct tegra_smmu_swgroup *group to *swgrp

In reply to Nicolin Chen

There are both tegra_smmu_swgroup and tegra_smmu_group structs using "group" for their pointer instances. This gets confusing to read the driver sometimes.

So this patch renames "group" of struct tegra_smmu_swgroup to "swgrp" as a cleanup. Also renames its "find" function.

Note that we already have "swgroup" being used for an unsigned int type variable that is inside struct tegra_smmu_swgroup, so it's not able to use "swgroup" but only something like "swgrp".

---
 drivers/iommu/tegra-smmu.c | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)
drivers/iommu/tegra-smmu.c +17 -17
index 9e9f37e4894c..42af216ef65c 100644
@@ -334,35 +334,35 @@ static void tegra_smmu_domain_free(struct iommu_domain *domain)
 }
 
 static const struct tegra_smmu_swgroup *
-tegra_smmu_find_swgroup(struct tegra_smmu *smmu, unsigned int swgroup)
+tegra_smmu_find_swgrp(struct tegra_smmu *smmu, unsigned int swgroup)
 {
-	const struct tegra_smmu_swgroup *group = NULL;
+	const struct tegra_smmu_swgroup *swgrp = NULL;
 	unsigned int i;
 
 	for (i = 0; i < smmu->soc->num_swgroups; i++) {
 		if (smmu->soc->swgroups[i].swgroup == swgroup) {
-			group = &smmu->soc->swgroups[i];
+			swgrp = &smmu->soc->swgroups[i];
 			break;
 		}
 	}
 
-	return group;
+	return swgrp;
 }
 
 static void tegra_smmu_enable(struct tegra_smmu *smmu, unsigned int swgroup,
 			      unsigned int asid)
 {
-	const struct tegra_smmu_swgroup *group;
+	const struct tegra_smmu_swgroup *swgrp;
 	unsigned int i;
 	u32 value;
 
-	group = tegra_smmu_find_swgroup(smmu, swgroup);
-	if (group) {
-		value = smmu_readl(smmu, group->reg);
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+	if (swgrp) {
+		value = smmu_readl(smmu, swgrp->reg);
 		value &= ~SMMU_ASID_MASK;
 		value |= SMMU_ASID_VALUE(asid);
 		value |= SMMU_ASID_ENABLE;
-		smmu_writel(smmu, value, group->reg);
+		smmu_writel(smmu, value, swgrp->reg);
 	} else {
 		pr_warn("%s group from swgroup %u not found\n", __func__,
 				swgroup);
@@ -385,17 +385,17 @@ static void tegra_smmu_enable(struct tegra_smmu *smmu, unsigned int swgroup,
 static void tegra_smmu_disable(struct tegra_smmu *smmu, unsigned int swgroup,
 			       unsigned int asid)
 {
-	const struct tegra_smmu_swgroup *group;
+	const struct tegra_smmu_swgroup *swgrp;
 	unsigned int i;
 	u32 value;
 
-	group = tegra_smmu_find_swgroup(smmu, swgroup);
-	if (group) {
-		value = smmu_readl(smmu, group->reg);
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+	if (swgrp) {
+		value = smmu_readl(smmu, swgrp->reg);
 		value &= ~SMMU_ASID_MASK;
 		value |= SMMU_ASID_VALUE(asid);
 		value &= ~SMMU_ASID_ENABLE;
-		smmu_writel(smmu, value, group->reg);
+		smmu_writel(smmu, value, swgrp->reg);
 	}
 
 	for (i = 0; i < smmu->soc->num_clients; i++) {
@@ -1007,11 +1007,11 @@ static int tegra_smmu_swgroups_show(struct seq_file *s, void *data)
 	seq_printf(s, "------------------------\n");
 
 	for (i = 0; i < smmu->soc->num_swgroups; i++) {
-		const struct tegra_smmu_swgroup *group = &smmu->soc->swgroups[i];
+		const struct tegra_smmu_swgroup *swgrp = &smmu->soc->swgroups[i];
 		const char *status;
 		unsigned int asid;
 
-		value = smmu_readl(smmu, group->reg);
+		value = smmu_readl(smmu, swgrp->reg);
 
 		if (value & SMMU_ASID_ENABLE)
 			status = "yes";
@@ -1020,7 +1020,7 @@ static int tegra_smmu_swgroups_show(struct seq_file *s, void *data)
 
 		asid = value & SMMU_ASID_MASK;
 
-		seq_printf(s, "%-9s  %-7s  %#04x\n", group->name, status,
+		seq_printf(s, "%-9s  %-7s  %#04x\n", swgrp->name, status,
 			   asid);
 	}
 
Signature · 1 line
2.17.1
#4 lore raw

PATCH v7 4/6 iommu/tegra-smmu: Use swgrp pointer instead of swgroup id

In reply to Nicolin Chen

This patch changes in struct tegra_smmu_group to use swgrp pointer instead of swgroup, as a preparational change for the "mappings" debugfs feature.

---
 drivers/iommu/tegra-smmu.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
drivers/iommu/tegra-smmu.c +8 -4
index 42af216ef65c..b0a04cc8f560 100644
@@ -23,8 +23,8 @@ struct tegra_smmu_group {
 	struct list_head list;
 	struct tegra_smmu *smmu;
 	const struct tegra_smmu_group_soc *soc;
+	const struct tegra_smmu_swgroup *swgrp;
 	struct iommu_group *grp;
-	unsigned int swgroup;
 };
 
 struct tegra_smmu {
@@ -897,18 +897,22 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
 	struct tegra_smmu *smmu = dev_iommu_priv_get(dev);
 	const struct tegra_smmu_group_soc *soc;
+	const struct tegra_smmu_swgroup *swgrp;
 	unsigned int swgroup = fwspec->ids[0];
 	struct tegra_smmu_group *group;
 	struct iommu_group *grp;
 
+	/* Find swgrp according to the swgroup id */
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+
 	/* Find group_soc associating with swgroup */
 	soc = tegra_smmu_find_group_soc(smmu, swgroup);
 
 	mutex_lock(&smmu->lock);
 
-	/* Find existing iommu_group associating with swgroup or group_soc */
+	/* Find existing iommu_group associating with swgrp or group_soc */
 	list_for_each_entry(group, &smmu->groups, list)
-		if ((group->swgroup == swgroup) || (soc && group->soc == soc)) {
+		if ((swgrp && group->swgrp == swgrp) || (soc && group->soc == soc)) {
 			grp = iommu_group_ref_get(group->grp);
 			mutex_unlock(&smmu->lock);
 			return grp;
@@ -921,7 +925,7 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)
 	}
 
 	INIT_LIST_HEAD(&group->list);
-	group->swgroup = swgroup;
+	group->swgrp = swgrp;
 	group->smmu = smmu;
 	group->soc = soc;
 
Signature · 1 line
2.17.1
#5 lore raw

PATCH v7 5/6 iommu/tegra-smmu: Attach as pointer to tegra_smmu_group

In reply to Nicolin Chen

This could ease driver to access corresponding as pointer when having tegra_smmu_group pointer only, which can help new mappings debugfs nodes.

Also moving tegra_smmu_find_group_soc() upward, for using it in new tegra_smmu_attach_as(); and it's better to have all tegra_smmu_find_* functions together.

---
 drivers/iommu/tegra-smmu.c | 94 +++++++++++++++++++++++++++++++-------
 1 file changed, 78 insertions(+), 16 deletions(-)
drivers/iommu/tegra-smmu.c +78 -16
index b0a04cc8f560..1fbb93215463 100644
@@ -24,6 +24,7 @@ struct tegra_smmu_group {
 	struct tegra_smmu *smmu;
 	const struct tegra_smmu_group_soc *soc;
 	const struct tegra_smmu_swgroup *swgrp;
+	struct tegra_smmu_as *as;
 	struct iommu_group *grp;
 };
 
@@ -349,6 +350,19 @@ tegra_smmu_find_swgrp(struct tegra_smmu *smmu, unsigned int swgroup)
 	return swgrp;
 }
 
+static const struct tegra_smmu_group_soc *
+tegra_smmu_find_group_soc(struct tegra_smmu *smmu, unsigned int swgroup)
+{
+	unsigned int i, j;
+
+	for (i = 0; i < smmu->soc->num_groups; i++)
+		for (j = 0; j < smmu->soc->groups[i].num_swgroups; j++)
+			if (smmu->soc->groups[i].swgroups[j] == swgroup)
+				return &smmu->soc->groups[i];
+
+	return NULL;
+}
+
 static void tegra_smmu_enable(struct tegra_smmu *smmu, unsigned int swgroup,
 			      unsigned int asid)
 {
@@ -482,6 +496,57 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,
 	mutex_unlock(&smmu->lock);
 }
 
+static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
+				 struct tegra_smmu_as *as,
+				 unsigned int swgroup)
+{
+	const struct tegra_smmu_swgroup *swgrp;
+	struct tegra_smmu_group *group;
+
+	/* Find swgrp according to the swgroup id */
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+	if (!swgrp)
+		return;
+
+	mutex_lock(&smmu->lock);
+
+	list_for_each_entry(group, &smmu->groups, list) {
+		if (group->swgrp != swgrp)
+			continue;
+		if (group->as)
+			dev_warn(smmu->dev,
+				 "overwriting group->as for swgroup: %s\n", swgrp->name);
+		group->as = as;
+		break;
+	}
+
+	mutex_unlock(&smmu->lock);
+}
+
+static void tegra_smmu_detach_as(struct tegra_smmu *smmu,
+				 unsigned int swgroup)
+{
+	const struct tegra_smmu_swgroup *swgrp;
+	struct tegra_smmu_group *group;
+	struct dentry *d;
+
+	/* Find swgrp according to the swgroup id */
+	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
+	if (!swgrp)
+		return;
+
+	mutex_lock(&smmu->lock);
+
+	list_for_each_entry(group, &smmu->groups, list) {
+		if (group->swgrp != swgrp)
+			continue;
+		group->as = NULL;
+		break;
+	}
+
+	mutex_unlock(&smmu->lock);
+}
+
 static int tegra_smmu_attach_dev(struct iommu_domain *domain,
 				 struct device *dev)
 {
@@ -495,11 +560,15 @@ static int tegra_smmu_attach_dev(struct iommu_domain *domain,
 		return -ENOENT;
 
 	for (index = 0; index < fwspec->num_ids; index++) {
+		unsigned int swgroup = fwspec->ids[index];
+
 		err = tegra_smmu_as_prepare(smmu, as);
 		if (err)
 			goto disable;
 
-		tegra_smmu_enable(smmu, fwspec->ids[index], as->id);
+		tegra_smmu_attach_as(smmu, as, swgroup);
+
+		tegra_smmu_enable(smmu, swgroup, as->id);
 	}
 
 	if (index == 0)
@@ -509,7 +578,10 @@ static int tegra_smmu_attach_dev(struct iommu_domain *domain,
 
 disable:
 	while (index--) {
-		tegra_smmu_disable(smmu, fwspec->ids[index], as->id);
+		unsigned int swgroup = fwspec->ids[index];
+
+		tegra_smmu_disable(smmu, swgroup, as->id);
+		tegra_smmu_detach_as(smmu, swgroup);
 		tegra_smmu_as_unprepare(smmu, as);
 	}
 
@@ -527,7 +599,10 @@ static void tegra_smmu_detach_dev(struct iommu_domain *domain, struct device *de
 		return;
 
 	for (index = 0; index < fwspec->num_ids; index++) {
-		tegra_smmu_disable(smmu, fwspec->ids[index], as->id);
+		unsigned int swgroup = fwspec->ids[index];
+
+		tegra_smmu_disable(smmu, swgroup, as->id);
+		tegra_smmu_detach_as(smmu, swgroup);
 		tegra_smmu_as_unprepare(smmu, as);
 	}
 }
@@ -869,19 +944,6 @@ static struct iommu_device *tegra_smmu_probe_device(struct device *dev)
 
 static void tegra_smmu_release_device(struct device *dev) {}
 
-static const struct tegra_smmu_group_soc *
-tegra_smmu_find_group_soc(struct tegra_smmu *smmu, unsigned int swgroup)
-{
-	unsigned int i, j;
-
-	for (i = 0; i < smmu->soc->num_groups; i++)
-		for (j = 0; j < smmu->soc->groups[i].num_swgroups; j++)
-			if (smmu->soc->groups[i].swgroups[j] == swgroup)
-				return &smmu->soc->groups[i];
-
-	return NULL;
-}
-
 static void tegra_smmu_group_release(void *iommu_data)
 {
 	struct tegra_smmu_group *group = iommu_data;
Signature · 1 line
2.17.1
#6 lore raw

PATCH v7 6/6 iommu/tegra-smmu: Add pagetable mappings to debugfs

In reply to Nicolin Chen

This patch dumps all active mapping entries from pagetable to a debugfs directory named "mappings".

Attaching an example:

[SWGROUP: xusb_host] [as: (id: 5), (attr: R|W|-), (pd_dma: 0x0000000080005000)]

{
        [index: 1023] 0xf0080040 (count: 52)
        {
                PTE RANGE      | ATTR | PHYS               | IOVA               | SIZE
                [#913 , #913 ] | 0x7  | 0x0000000102674000 | 0x00000000fff91000 | 0x1000
                [#914 , #914 ] | 0x7  | 0x0000000102672000 | 0x00000000fff92000 | 0x1000
                [#915 , #915 ] | 0x7  | 0x0000000102671000 | 0x00000000fff93000 | 0x1000
                [#916 , #916 ] | 0x7  | 0x0000000102670000 | 0x00000000fff94000 | 0x1000
                [#921 , #921 ] | 0x7  | 0x00000000fcc00000 | 0x00000000fff99000 | 0x1000
                [#922 , #922 ] | 0x7  | 0x000000010266d000 | 0x00000000fff9a000 | 0x1000
                [#923 , #923 ] | 0x7  | 0x000000010266c000 | 0x00000000fff9b000 | 0x1000
                [#948 , #948 ] | 0x7  | 0x0000000102668000 | 0x00000000fffb4000 | 0x1000
                [#949 , #949 ] | 0x7  | 0x0000000102667000 | 0x00000000fffb5000 | 0x1000
                [#950 , #950 ] | 0x7  | 0x0000000102666000 | 0x00000000fffb6000 | 0x1000
                [#951 , #951 ] | 0x7  | 0x0000000102665000 | 0x00000000fffb7000 | 0x1000
                [#952 , #952 ] | 0x7  | 0x000000010264b000 | 0x00000000fffb8000 | 0x1000
                [#953 , #953 ] | 0x7  | 0x000000010264a000 | 0x00000000fffb9000 | 0x1000
                [#954 , #954 ] | 0x7  | 0x0000000102649000 | 0x00000000fffba000 | 0x1000
                [#955 , #955 ] | 0x7  | 0x0000000102648000 | 0x00000000fffbb000 | 0x1000
                [#956 , #956 ] | 0x7  | 0x000000010260f000 | 0x00000000fffbc000 | 0x1000
                [#957 , #957 ] | 0x7  | 0x000000010260e000 | 0x00000000fffbd000 | 0x1000
                [#958 , #958 ] | 0x7  | 0x000000010260d000 | 0x00000000fffbe000 | 0x1000
                [#959 , #959 ] | 0x7  | 0x000000010260b000 | 0x00000000fffbf000 | 0x1000
                [#960 , #992 ] | 0x7  | 0x00000001025ea000 | 0x00000000fffc0000 | 0x21000
        }
}

Total PDEs: 1, total PTEs: 52

Note that the example above was output after I locally enabled IOMMU_DOMAIN_DMA, which is not merged to mainline yet due to a known framebuffer issue.

---
 drivers/iommu/tegra-smmu.c | 143 +++++++++++++++++++++++++++++++++++++
 1 file changed, 143 insertions(+)
drivers/iommu/tegra-smmu.c +143 -0
index 1fbb93215463..575e82076270 100644
@@ -46,6 +46,7 @@ struct tegra_smmu {
 	struct list_head list;
 
 	struct dentry *debugfs;
+	struct dentry *debugfs_mappings;
 
 	struct iommu_device iommu;	/* IOMMU Core code handle */
 };
@@ -153,6 +154,9 @@ static inline u32 smmu_readl(struct tegra_smmu *smmu, unsigned long offset)
 
 #define SMMU_PDE_ATTR		(SMMU_PDE_READABLE | SMMU_PDE_WRITABLE | \
 				 SMMU_PDE_NONSECURE)
+#define SMMU_PTE_ATTR		(SMMU_PTE_READABLE | SMMU_PTE_WRITABLE | \
+				 SMMU_PTE_NONSECURE)
+#define SMMU_PTE_ATTR_SHIFT	29
 
 static unsigned int iova_pd_index(unsigned long iova)
 {
@@ -164,6 +168,12 @@ static unsigned int iova_pt_index(unsigned long iova)
 	return (iova >> SMMU_PTE_SHIFT) & (SMMU_NUM_PTE - 1);
 }
 
+static unsigned long pd_pt_index_iova(unsigned int pd_index, unsigned int pt_index)
+{
+	return (pd_index & (SMMU_NUM_PDE - 1)) << SMMU_PDE_SHIFT |
+	       (pt_index & (SMMU_NUM_PTE - 1)) << SMMU_PTE_SHIFT;
+}
+
 static bool smmu_dma_addr_valid(struct tegra_smmu *smmu, dma_addr_t addr)
 {
 	addr >>= 12;
@@ -496,6 +506,125 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,
 	mutex_unlock(&smmu->lock);
 }
 
+static int tegra_smmu_debugfs_mappings_show(struct seq_file *s, void *data)
+{
+	struct tegra_smmu_group *group = s->private;
+	const struct tegra_smmu_swgroup *swgrp;
+	struct tegra_smmu_as *as;
+	struct tegra_smmu *smmu;
+	unsigned int pd_index;
+	unsigned int pt_index;
+	unsigned long flags;
+	u64 pte_count = 0;
+	u32 pde_count = 0;
+	u32 *pd, val;
+
+	if (!group || !group->as || !group->swgrp)
+		return 0;
+
+	swgrp = group->swgrp;
+	smmu = group->smmu;
+	as = group->as;
+
+	mutex_lock(&smmu->lock);
+
+	val = smmu_readl(smmu, swgrp->reg) & SMMU_ASID_ENABLE;
+	if (!val)
+		goto unlock;
+
+	pd = page_address(as->pd);
+	if (!pd)
+		goto unlock;
+
+	seq_printf(s, "[SWGROUP: %s] ", swgrp->name);
+	seq_printf(s, "[as: (id: %d), ", as->id);
+	seq_printf(s, "(attr: %c|%c|%c), ",
+		   as->attr & SMMU_PD_READABLE ? 'R' : '-',
+		   as->attr & SMMU_PD_WRITABLE ? 'W' : '-',
+		   as->attr & SMMU_PD_NONSECURE ? '-' : 'S');
+	seq_printf(s, "(pd_dma: %pad)]\n", &as->pd_dma);
+	seq_puts(s, "{\n");
+
+	spin_lock_irqsave(&as->lock, flags);
+
+	for (pd_index = 0; pd_index < SMMU_NUM_PDE; pd_index++) {
+		struct page *pt_page;
+		unsigned int i;
+		u32 *addr;
+
+		/* An empty PDE should not have a pte use count */
+		WARN_ON_ONCE(!pd[pd_index] ^ !as->count[pd_index]);
+
+		/* Skip this empty PDE */
+		if (!pd[pd_index])
+			continue;
+
+		pde_count++;
+		pte_count += as->count[pd_index];
+		seq_printf(s, "\t[index: %u] 0x%x (count: %d)\n",
+			   pd_index, pd[pd_index], as->count[pd_index]);
+		pt_page = as->pts[pd_index];
+		addr = page_address(pt_page);
+
+		seq_puts(s, "\t{\n");
+		seq_printf(s, "\t\t%-14s | %-4s | %-10s%s | %-10s%s | %-11s\n",
+			   "PTE RANGE", "ATTR",
+			   "PHYS", sizeof(phys_addr_t) > 4 ? "        " : "",
+			   "IOVA", sizeof(dma_addr_t)  > 4 ? "        " : "",
+			   "SIZE");
+		for (pt_index = 0; pt_index < SMMU_NUM_PTE; pt_index += i) {
+			size_t size = SMMU_SIZE_PT;
+			dma_addr_t iova;
+			phys_addr_t pa;
+
+			i = 1;
+
+			if (!addr[pt_index])
+				continue;
+
+			iova = pd_pt_index_iova(pd_index, pt_index);
+			pa = SMMU_PFN_PHYS(addr[pt_index] & ~SMMU_PTE_ATTR);
+
+			/* Check contiguous mappings and increase size */
+			while (pt_index + i < SMMU_NUM_PTE) {
+				dma_addr_t next_iova;
+				phys_addr_t next_pa;
+
+				if (!addr[pt_index + i])
+					break;
+
+				next_iova = pd_pt_index_iova(pd_index, pt_index + i);
+				next_pa = SMMU_PFN_PHYS(addr[pt_index + i] & ~SMMU_PTE_ATTR);
+
+				/* Break at the end of a linear mapping */
+				if ((next_iova - iova != SMMU_SIZE_PT * i) ||
+				    (next_pa - pa != SMMU_SIZE_PT * i))
+					break;
+
+				i++;
+			}
+
+			seq_printf(s, "\t\t[#%-4u, #%-4u] | 0x%-2x | %pa | %pad | 0x%-9zx\n",
+				   pt_index, pt_index + i - 1,
+				   addr[pt_index] >> SMMU_PTE_ATTR_SHIFT,
+				   &pa, &iova, size * i);
+		}
+		seq_puts(s, "\t}\n");
+	}
+
+	spin_unlock_irqrestore(&as->lock, flags);
+
+	seq_puts(s, "}\n");
+	seq_printf(s, "Total PDEs: %u, total PTEs: %llu\n ", pde_count, pte_count);
+
+unlock:
+	mutex_unlock(&smmu->lock);
+
+	return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(tegra_smmu_debugfs_mappings);
+
 static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
 				 struct tegra_smmu_as *as,
 				 unsigned int swgroup)
@@ -517,6 +646,12 @@ static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
 			dev_warn(smmu->dev,
 				 "overwriting group->as for swgroup: %s\n", swgrp->name);
 		group->as = as;
+
+		if (smmu->debugfs_mappings)
+			debugfs_create_file(group->swgrp->name, 0444,
+					    smmu->debugfs_mappings, group,
+					    &tegra_smmu_debugfs_mappings_fops);
+
 		break;
 	}
 
@@ -541,6 +676,12 @@ static void tegra_smmu_detach_as(struct tegra_smmu *smmu,
 		if (group->swgrp != swgrp)
 			continue;
 		group->as = NULL;
+
+		if (smmu->debugfs_mappings) {
+			d = debugfs_lookup(group->swgrp->name, smmu->debugfs_mappings);
+			debugfs_remove(d);
+		}
+
 		break;
 	}
 
@@ -1133,6 +1274,8 @@ static void tegra_smmu_debugfs_init(struct tegra_smmu *smmu)
 			    &tegra_smmu_swgroups_fops);
 	debugfs_create_file("clients", S_IRUGO, smmu->debugfs, smmu,
 			    &tegra_smmu_clients_fops);
+
+	smmu->debugfs_mappings = debugfs_create_dir("mappings", smmu->debugfs);
 }
 
 static void tegra_smmu_debugfs_exit(struct tegra_smmu *smmu)
Signature · 6 lines
2.17.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#7 lore raw

Hi,

08.12.2021 11:47, Nicolin Chen пишет:

Quoted 32 lines

This patch dumps all active mapping entries from pagetable to a debugfs directory named "mappings".

Attaching an example:

[SWGROUP: xusb_host] [as: (id: 5), (attr: R|W|-), (pd_dma: 0x0000000080005000)]

{
        [index: 1023] 0xf0080040 (count: 52)
        {
                PTE RANGE      | ATTR | PHYS               | IOVA               | SIZE
                [#913 , #913 ] | 0x7  | 0x0000000102674000 | 0x00000000fff91000 | 0x1000
                [#914 , #914 ] | 0x7  | 0x0000000102672000 | 0x00000000fff92000 | 0x1000
                [#915 , #915 ] | 0x7  | 0x0000000102671000 | 0x00000000fff93000 | 0x1000
                [#916 , #916 ] | 0x7  | 0x0000000102670000 | 0x00000000fff94000 | 0x1000
                [#921 , #921 ] | 0x7  | 0x00000000fcc00000 | 0x00000000fff99000 | 0x1000
                [#922 , #922 ] | 0x7  | 0x000000010266d000 | 0x00000000fff9a000 | 0x1000
                [#923 , #923 ] | 0x7  | 0x000000010266c000 | 0x00000000fff9b000 | 0x1000
                [#948 , #948 ] | 0x7  | 0x0000000102668000 | 0x00000000fffb4000 | 0x1000
                [#949 , #949 ] | 0x7  | 0x0000000102667000 | 0x00000000fffb5000 | 0x1000
                [#950 , #950 ] | 0x7  | 0x0000000102666000 | 0x00000000fffb6000 | 0x1000
                [#951 , #951 ] | 0x7  | 0x0000000102665000 | 0x00000000fffb7000 | 0x1000
                [#952 , #952 ] | 0x7  | 0x000000010264b000 | 0x00000000fffb8000 | 0x1000
                [#953 , #953 ] | 0x7  | 0x000000010264a000 | 0x00000000fffb9000 | 0x1000
                [#954 , #954 ] | 0x7  | 0x0000000102649000 | 0x00000000fffba000 | 0x1000
                [#955 , #955 ] | 0x7  | 0x0000000102648000 | 0x00000000fffbb000 | 0x1000
                [#956 , #956 ] | 0x7  | 0x000000010260f000 | 0x00000000fffbc000 | 0x1000
                [#957 , #957 ] | 0x7  | 0x000000010260e000 | 0x00000000fffbd000 | 0x1000
                [#958 , #958 ] | 0x7  | 0x000000010260d000 | 0x00000000fffbe000 | 0x1000
                [#959 , #959 ] | 0x7  | 0x000000010260b000 | 0x00000000fffbf000 | 0x1000
                [#960 , #992 ] | 0x7  | 0x00000001025ea000 | 0x00000000fffc0000 | 0x21000
        }
}
Quoted 1 line

Total PDEs: 1, total PTEs: 52

The patch is almost good to me, there is one nit.

On older SoCs we put multiple devices into the same shared group and the debugfs shows it as the first member of the group.

This is what we get on T30 using this v7:

# ls/sys/kernel/debug/smmu/mappings
g2  hc  vde
# cat /sys/kernel/debug/smmu/mappings/g2 
[SWGROUP: g2] [as: (id: 2), (attr: R|W|-), (pd_dma: 0x834a6000)]
{
        [index: 0] 0xf0083494 (count: 1000)
        {
                PTE RANGE      | ATTR | PHYS       | IOVA       | SIZE       
                [#0   , #15  ] | 0x7  | 0xbfde0000 | 0x00000000 | 0x10000    
                [#16  , #47  ] | 0x7  | 0xbfdc0000 | 0x00010000 | 0x20000    
                [#48  , #111 ] | 0x7  | 0xbfd80000 | 0x00030000 | 0x40000    
                [#112 , #239 ] | 0x7  | 0xbfd00000 | 0x00070000 | 0x80000    
                [#240 , #495 ] | 0x7  | 0xbfc00000 | 0x000f0000 | 0x100000   
                [#496 , #999 ] | 0x7  | 0xbf400000 | 0x001f0000 | 0x1f8000   
        }
}
Total PDEs: 1, total PTEs: 1000

See that name is "g2", meanwhile these mappings are made by display client driver.

I changed your patch to use the proper group name and to show all members of the group, see that change in the end of this email.

With my change applied, we get:

# ls/sys/kernel/debug/smmu/mappings
drm  hc  vde
# cat /sys/kernel/debug/smmu/mappings/drm 
[SWGROUP: dc, dcb, g2, nv, nv2] [as: (id: 2), (attr: R|W|-), (pd_dma: 0x82480000)]
{
        [index: 0] 0xf0083583 (count: 1000)
        {
                PTE RANGE      | ATTR | PHYS       | IOVA       | SIZE       
                [#0   , #15  ] | 0x7  | 0xbfde0000 | 0x00000000 | 0x10000    
                [#16  , #47  ] | 0x7  | 0xbfdc0000 | 0x00010000 | 0x20000    
                [#48  , #111 ] | 0x7  | 0xbfd80000 | 0x00030000 | 0x40000    
                [#112 , #239 ] | 0x7  | 0xbfd00000 | 0x00070000 | 0x80000    
                [#240 , #495 ] | 0x7  | 0xbfc00000 | 0x000f0000 | 0x100000   
                [#496 , #999 ] | 0x7  | 0xbf400000 | 0x001f0000 | 0x1f8000   
        }
}
Total PDEs: 1, total PTEs: 1000

--- >8 ---

drivers/iommu/tegra-smmu.c +43 -3
index 575e82076270..fb1326a72038 100644
@@ -509,6 +509,7 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,
 static int tegra_smmu_debugfs_mappings_show(struct seq_file *s, void *data)
 {
 	struct tegra_smmu_group *group = s->private;
+	const struct tegra_smmu_group_soc *soc;
 	const struct tegra_smmu_swgroup *swgrp;
 	struct tegra_smmu_as *as;
 	struct tegra_smmu *smmu;
@@ -524,6 +525,7 @@ static int tegra_smmu_debugfs_mappings_show(struct seq_file *s, void *data)
 
 	swgrp = group->swgrp;
 	smmu = group->smmu;
+	soc = group->soc;
 	as = group->as;
 
 	mutex_lock(&smmu->lock);
@@ -536,7 +538,38 @@ static int tegra_smmu_debugfs_mappings_show(struct seq_file *s, void *data)
 	if (!pd)
 		goto unlock;
 
-	seq_printf(s, "[SWGROUP: %s] ", swgrp->name);
+	seq_puts(s, "[SWGROUP: ");
+	if (soc) {
+		bool first_swgroup = true;
+		unsigned int i;
+
+		for (i = 0; i < soc->num_swgroups; i++) {
+			swgrp = tegra_smmu_find_swgrp(smmu, soc->swgroups[i]);
+
+			if (WARN_ON(!swgrp))
+				goto unlock;
+
+			val = smmu_readl(smmu, swgrp->reg);
+
+			if (!(val & SMMU_ASID_ENABLE))
+				continue;
+
+			if (WARN_ON((val & SMMU_ASID_MASK) != as->id))
+				continue;
+
+			if (first_swgroup)
+				first_swgroup = false;
+			else
+				seq_puts(s, ", ");
+
+			seq_printf(s, "%s", swgrp->name);
+		}
+	} else {
+		WARN_ON((val & SMMU_ASID_MASK) != as->id);
+		seq_printf(s, "%s", swgrp->name);
+	}
+	seq_puts(s, "] ");
+
 	seq_printf(s, "[as: (id: %d), ", as->id);
 	seq_printf(s, "(attr: %c|%c|%c), ",
 		   as->attr & SMMU_PD_READABLE ? 'R' : '-',
@@ -631,6 +664,7 @@ static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
 {
 	const struct tegra_smmu_swgroup *swgrp;
 	struct tegra_smmu_group *group;
+	const char *name;
 
 	/* Find swgrp according to the swgroup id */
 	swgrp = tegra_smmu_find_swgrp(smmu, swgroup);
@@ -647,10 +681,16 @@ static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
 				 "overwriting group->as for swgroup: %s\n", swgrp->name);
 		group->as = as;
 
-		if (smmu->debugfs_mappings)
-			debugfs_create_file(group->swgrp->name, 0444,
+		if (smmu->debugfs_mappings) {
+			if (group->soc)
+				name = group->soc->name;
+			else
+				name = group->swgrp->name;
+
+			debugfs_create_file(name, 0444,
 					    smmu->debugfs_mappings, group,
 					    &tegra_smmu_debugfs_mappings_fops);
+		}
 
 		break;
 	}
#8 lore raw
In reply to Nicolin Chen

08.12.2021 11:47, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu, · line 573

Quoted 4 lines
drivers/iommu/tegra-smmu.c +4 -0
+		seq_printf(s, "\t\t%-14s | %-4s | %-10s%s | %-10s%s | %-11s\n",
+			   "PTE RANGE", "ATTR",
+			   "PHYS", sizeof(phys_addr_t) > 4 ? "        " : "",
+			   "IOVA", sizeof(dma_addr_t)  > 4 ? "        " : "",

Can we change IOVA to u32?

#9 lore raw
In reply to Nicolin Chen

08.12.2021 11:47, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static void tegra_smmu_attach_as(struct tegra_smmu *smmu, · line 653

Quoted 12 lines

static void tegra_smmu_attach_as(struct tegra_smmu *smmu,

drivers/iommu/tegra-smmu.c +5 -0
 				 struct tegra_smmu_as *as,
 				 unsigned int swgroup)
@@ -517,6 +646,12 @@ static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
 			dev_warn(smmu->dev,
 				 "overwriting group->as for swgroup: %s\n", swgrp->name);
 		group->as = as;
+
+		if (smmu->debugfs_mappings)
+			debugfs_create_file(group->swgrp->name, 0444,
+					    smmu->debugfs_mappings, group,
+					    &tegra_smmu_debugfs_mappings_fops);

I noticed this in KMSG:

tegra-mc 7000f000.memory-controller: overwriting group->as for swgroup: g2
debugfs: File 'g2' in directory 'mappings' already present! tegra-mc 7000f000.memory-controller: overwriting group->as for swgroup: g2
debugfs: File 'g2' in directory 'mappings' already present

Doesn't look okay, please fix.
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#10 lore raw
Quoted 19 lines · Dmitry Osipenko

External email: Use caution opening links or attachments

08.12.2021 11:47, Nicolin Chen пишет:

drivers/iommu/tegra-smmu.c static void tegra_smmu_attach_as(struct tegra_smmu *smmu, · line 653

Quoted 12 lines

static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
struct tegra_smmu_as *as, unsigned int swgroup)

drivers/iommu/tegra-smmu.c +5 -0
@@ -517,6 +646,12 @@ static void tegra_smmu_attach_as(struct tegra_smmu *smmu,
                      dev_warn(smmu->dev,
                               "overwriting group->as for swgroup: %s\n", swgrp->name);
              group->as = as;
+
+             if (smmu->debugfs_mappings)
+                     debugfs_create_file(group->swgrp->name, 0444,
+                                         smmu->debugfs_mappings, group,
+                                         &tegra_smmu_debugfs_mappings_fops);

I noticed this in KMSG:

Quoted 6 lines

tegra-mc 7000f000.memory-controller: overwriting group->as for swgroup: g2
debugfs: File 'g2' in directory 'mappings' already present! tegra-mc 7000f000.memory-controller: overwriting group->as for swgroup: g2
debugfs: File 'g2' in directory 'mappings' already present

Doesn't look okay, please fix.

Thanks for testing and reporting!

I will see and fix it along with your other comments.