iommu/tegra-smmu: Adding PCI support and mappings debugfs node
cover iommu/tegra-smmu: Adding PCI support and mappings debugfs node
This series of patches are some followup patches for tegra-smmu.
There are four parts:
1, PATCH-1 is a clean-up patch for the recently applied SWGROUP change. 2, PATCH-2 fixes a potential race condition
3, PATCH-3/4 adds PCI device support
4, PATCH-5 adds a debugfs node for phys<=>iova mappings
Each of the four parts is sort of functional independent, so we may apply them separately or incrementally depending on the reviews.
Nicolin Chen (5):
iommu/tegra-smmu: Unwrap tegra_smmu_group_get iommu/tegra-smmu: Expend mutex protection range iommu/tegra-smmu: Use iommu_fwspec in .probe_/.attach_device() iommu/tegra-smmu: Add PCI support
iommu/tegra-smmu: Add pagetable mappings to debugfs
drivers/iommu/tegra-smmu.c | 287 ++++++++++++++++++++++++++++--------- drivers/memory/tegra/mc.c | 2 +- include/soc/tegra/mc.h | 2 + 3 files changed, 221 insertions(+), 70 deletions(-)
Signature · 1 line
2.17.1
PATCH 1/5 iommu/tegra-smmu: Unwrap tegra_smmu_group_get
The tegra_smmu_group_get was added to group devices in different SWGROUPs and it'd return a NULL group pointer upon a mismatch at tegra_smmu_find_group(), so for most of clients/devices, it very likely would mismatch and need a fallback generic_device_group().
But now tegra_smmu_group_get handles devices in same SWGROUP too, which means that it would allocate a group for every new SWGROUP or would directly return an existing one upon matching a SWGROUP, i.e. any device will go through this function.
So possibility of having a NULL group pointer in device_group() is upon failure of either devm_kzalloc() or iommu_group_alloc(). In either case, calling generic_device_group() no longer makes a sense. Especially for devm_kzalloc() failing case, it'd cause a problem if it fails at devm_kzalloc() yet succeeds at a fallback generic_device_group(), because it does not create a group->list for other devices to match.
This patch simply unwraps the function to clean it up.
--- drivers/iommu/tegra-smmu.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-)
drivers/iommu/tegra-smmu.c +4 -15
@@ -903,11 +903,13 @@ static void tegra_smmu_group_release(void *iommu_data)mutex_unlock (&smmu->lock); } -static struct iommu_group *tegra_smmu_group_get (struct tegra_smmu *smmu, -unsigned int swgroup) +static struct iommu_group *tegra_smmu_device_group (struct device *dev) { +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev); +struct tegra_smmu *smmu =dev_iommu_priv_get (dev);const struct tegra_smmu_group_soc *soc;struct tegra_smmu_group *group; +int swgroup = fwspec->ids[0 ];struct iommu_group *grp;/* Find group_soc associating with swgroup */
@@ -950,19 +952,6 @@ static struct iommu_group *tegra_smmu_group_get(struct tegra_smmu *smmu,return group->group; } -static struct iommu_group *tegra_smmu_device_group (struct device *dev) -{ -struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev); -struct tegra_smmu *smmu =dev_iommu_priv_get (dev); -struct iommu_group *group; - - group =tegra_smmu_group_get (smmu, fwspec->ids[0 ]); -if (!group) - group =generic_device_group (dev); - -return group; -} -static int tegra_smmu_of_xlate (struct device *dev,struct of_phandle_args *args) {
Signature · 1 line
2.17.1
Re: [PATCH 1/5] iommu/tegra-smmu: Unwrap tegra_smmu_group_get
Quoted 32 lines · Nicolin Chen
The
tegra_smmu_group_getwas added to group devices in different SWGROUPs and it'd return a NULL group pointer upon a mismatch attegra_smmu_find_group(), so for most of clients/devices, it very likely would mismatch and need a fallbackgeneric_device_group().But now
tegra_smmu_group_gethandles devices in same SWGROUP too, which means that it would allocate a group for every new SWGROUP or would directly return an existing one upon matching a SWGROUP, i.e. any device will go through this function.So possibility of having a NULL group pointer in
device_group()is upon failure of eitherdevm_kzalloc()oriommu_group_alloc(). In either case, callinggeneric_device_group()no longer makes a sense. Especially fordevm_kzalloc()failing case, it'd cause a problem if it fails atdevm_kzalloc()yet succeeds at a fallbackgeneric_device_group(), because it does not create agroup->listfor other devices to match.This patch simply unwraps the function to clean it up.
Signed-off-by Nicolin Chen <[email protected]>--- drivers/iommu/tegra-smmu.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-)drivers/iommu/tegra-smmu.c +0
-0@@ -903,11 +903,13 @@ static void tegra_smmu_group_release(void *iommu_data)mutex_unlock (&smmu->lock); }
drivers/iommu/tegra-smmu.c static void tegra_smmu_group_release(void *iommu_data) · line 912
Quoted 10 lines
drivers/iommu/tegra-smmu.c +4
-2-static struct iommu_group *tegra_smmu_group_get (struct tegra_smmu *smmu, -unsigned int swgroup) +static struct iommu_group *tegra_smmu_device_group (struct device *dev) { +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev); +struct tegra_smmu *smmu =dev_iommu_priv_get (dev);const struct tegra_smmu_group_soc *soc;struct tegra_smmu_group *group; +int swgroup = fwspec->ids[0 ];
This should be unsigned int to match the type of swgroup elsewhere. Also, it might not be worth having an extra local variable for this since it's only used once.
Thierry
Re: [PATCH 1/5] iommu/tegra-smmu: Unwrap tegra_smmu_group_get
Hi Thierry,
Thanks for the review.
Quoted 5 lines · Thierry Reding
Quoted 5 lines
drivers/iommu/tegra-smmu.c +2
-2-static struct iommu_group *tegra_smmu_group_get (struct tegra_smmu *smmu, -unsigned int swgroup) +static struct iommu_group *tegra_smmu_device_group (struct device *dev) { +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev);
Quoted 8 lines
drivers/iommu/tegra-smmu.c static void tegra_smmu_group_release(void *iommu_data) · line 912
Quoted 4 lines
drivers/iommu/tegra-smmu.c +2
-0+struct tegra_smmu *smmu =dev_iommu_priv_get (dev);const struct tegra_smmu_group_soc *soc;struct tegra_smmu_group *group; +int swgroup = fwspec->ids[0 ];This should be unsigned int to match the type of swgroup elsewhere. Also, it might not be worth having an extra local variable for this since it's only used once.
Will change to unsigned int. There are actually a few places using it in this function, previously tegra_smmu_group_get().
PATCH 2/5 iommu/tegra-smmu: Expend mutex protection range
In reply to Nicolin ChenThis is used to protect potential race condition at use_count. since probes of client drivers, calling attach_dev(), may run concurrently.
--- drivers/iommu/tegra-smmu.c | 34 +++++++++++++++++++++------------- 1 file changed, 21 insertions(+), 13 deletions(-)
drivers/iommu/tegra-smmu.c +21 -13
@@ -256,26 +256,19 @@ static int tegra_smmu_alloc_asid(struct tegra_smmu *smmu, unsigned int *idp) {unsigned long id; -mutex_lock (&smmu->lock); - id =find_first_zero_bit (smmu->asids, smmu->soc->num_asids); -if (id >= smmu->soc->num_asids) { -mutex_unlock (&smmu->lock); +if (id >= smmu->soc->num_asids)return -ENOSPC ; - }set_bit (id, smmu->asids); *idp = id; -mutex_unlock (&smmu->lock);return 0 ; }static void tegra_smmu_free_asid (struct tegra_smmu *smmu,unsigned int id) { -mutex_lock (&smmu->lock);clear_bit (id, smmu->asids); -mutex_unlock (&smmu->lock); }static bool tegra_smmu_capable (enum iommu_cap cap)
@@ -420,17 +413,21 @@ static int tegra_smmu_as_prepare(struct tegra_smmu *smmu, struct tegra_smmu_as *as) {u32 value; -int err; +int err =0 ; + +mutex_lock (&smmu->lock);if (as->use_count >0 ) { as->use_count++; -return 0 ; + goto err_unlock; } as->pd_dma =dma_map_page (smmu->dev, as->pd,0 ,SMMU_SIZE_PD ,DMA_TO_DEVICE ); -if (dma_mapping_error (smmu->dev, as->pd_dma)) -return -ENOMEM ; +if (dma_mapping_error (smmu->dev, as->pd_dma)) { + err = -ENOMEM ; + goto err_unlock; + }/* We can't handle 64-bit DMA addresses */ if (!smmu_dma_addr_valid (smmu, as->pd_dma)) {
@@ -453,24 +450,35 @@ static int tegra_smmu_as_prepare(struct tegra_smmu *smmu, as->smmu = smmu; as->use_count++; +mutex_unlock (&smmu->lock); +return 0 ; err_unmap:dma_unmap_page (smmu->dev, as->pd_dma,SMMU_SIZE_PD ,DMA_TO_DEVICE ); +err_unlock: +mutex_unlock (&smmu->lock); +return err; }static void tegra_smmu_as_unprepare (struct tegra_smmu *smmu,struct tegra_smmu_as *as) { -if (--as->use_count >0 ) +mutex_lock (&smmu->lock); + +if (--as->use_count >0 ) { +mutex_unlock (&smmu->lock);return ; + }tegra_smmu_free_asid (smmu, as->id);dma_unmap_page (smmu->dev, as->pd_dma,SMMU_SIZE_PD ,DMA_TO_DEVICE ); as->smmu =NULL ; + +mutex_unlock (&smmu->lock); } static int tegra_smmu_attach_dev(struct iommu_domain *domain,
Signature · 1 line
2.17.1
PATCH 3/5 iommu/tegra-smmu: Use iommu_fwspec in .probe_/.attach_device()
In reply to Nicolin ChenThe tegra_smmu_probe_device() function searches in DT for the iommu phandler to get "smmu" pointer. This works for most of SMMU clients that exist in the DTB. But a PCI device will not be added to iommu, since it doesn't have a DT node.
Fortunately, for a client with a DT node, tegra_smmu_probe_device() calls tegra_smmu_of_xlate() via tegra_smmu_configure(), while for a PCI device, of_pci_iommu_init() in the IOMMU core calls .of_xlate() as well, even before running tegra_smmu_probe_device(). And in both cases, tegra_smmu_of_xlate() prepares a valid iommu_fwspec pointer that allows us to get the mc->smmu pointer via dev_get_drvdata() by calling driver_find_device_by_fwnode().
So this patch uses iommu_fwspec in .probe_device() and related code for a client that does not exist in the DTB, especially a PCI one.
--- drivers/iommu/tegra-smmu.c | 89 +++++++++++++++++++++++--------------- drivers/memory/tegra/mc.c | 2 +- include/soc/tegra/mc.h | 2 + 3 files changed, 56 insertions(+), 37 deletions(-)
drivers/iommu/tegra-smmu.c +53 -36
@@ -13,6 +13,7 @@#include <linux/platform_device.h> #include <linux/slab.h> #include <linux/spinlock.h> +#include <linux/dma-iommu.h> #include <linux/dma-mapping.h> #include <soc/tegra/ahb.h>
@@ -61,6 +62,8 @@ struct tegra_smmu_as {u32 attr; }; +static const struct iommu_ops tegra_smmu_ops; +static struct tegra_smmu_as *to_smmu_as (struct iommu_domain *dom) { return container_of(dom, struct tegra_smmu_as, domain);
@@ -484,60 +487,49 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,static int tegra_smmu_attach_dev (struct iommu_domain *domain,struct device *dev) { +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev);struct tegra_smmu *smmu =dev_iommu_priv_get (dev);struct tegra_smmu_as *as =to_smmu_as (domain); -struct device_node *np = dev->of_node; -struct of_phandle_args args; -unsigned int index =0 ; -int err =0 ; - -while (!of_parse_phandle_with_args (np,"iommus" ,"#iommu-cells" , index, - &args)) { -unsigned int swgroup = args.args[0 ]; - -if (args.np != smmu->dev->of_node) { -of_node_put (args.np); -continue ; - } +int index, err =0 ; -of_node_put (args.np); +if (!fwspec || fwspec->ops != &tegra_smmu_ops) +return -ENOENT ; +for (index =0 ; index < fwspec->num_ids; index++) { err =tegra_smmu_as_prepare (smmu, as); -if (err <0 ) -return err; +if (err) + goto err_disable; -tegra_smmu_enable (smmu, swgroup, as->id); - index++; +tegra_smmu_enable (smmu, fwspec->ids[index], as->id); }if (index ==0 )return -ENODEV ;return 0 ; + +err_disable: +for (index--; index >=0 ; index--) { +tegra_smmu_disable (smmu, fwspec->ids[index], as->id); +tegra_smmu_as_unprepare (smmu, as); + } + +return err; }static void tegra_smmu_detach_dev (struct iommu_domain *domain,struct device *dev) { +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev);struct tegra_smmu_as *as =to_smmu_as (domain); -struct device_node *np = dev->of_node;struct tegra_smmu *smmu = as->smmu; -struct of_phandle_args args;unsigned int index =0 ; -while (!of_parse_phandle_with_args (np,"iommus" ,"#iommu-cells" , index, - &args)) { -unsigned int swgroup = args.args[0 ]; - -if (args.np != smmu->dev->of_node) { -of_node_put (args.np); -continue ; - } - -of_node_put (args.np); +if (!fwspec || fwspec->ops != &tegra_smmu_ops) +return ; -tegra_smmu_disable (smmu, swgroup, as->id); +for (index =0 ; index < fwspec->num_ids; index++) { +tegra_smmu_disable (smmu, fwspec->ids[index], as->id);tegra_smmu_as_unprepare (smmu, as); - index++; } }
@@ -845,10 +837,25 @@ static int tegra_smmu_configure(struct tegra_smmu *smmu, struct device *dev,return 0 ; } +static struct tegra_smmu *tegra_smmu_get_by_fwnode (struct fwnode_handle *fwnode) +{ +struct device *dev =driver_find_device_by_fwnode (&tegra_mc_driver.driver, fwnode); +struct tegra_mc *mc; + +if (!dev) +return NULL ; + +put_device (dev); + mc =dev_get_drvdata (dev); + +return mc->smmu; +} +static struct iommu_device *tegra_smmu_probe_device (struct device *dev) {struct device_node *np = dev->of_node;struct tegra_smmu *smmu =NULL ; +struct iommu_fwspec *fwspec;struct of_phandle_args args;unsigned int index =0 ;int err;
@@ -868,8 +875,6 @@ static struct iommu_device *tegra_smmu_probe_device(struct device *dev)* supported by the Linux kernel, so abort after the * first match. */ -dev_iommu_priv_set (dev, smmu); -break ; }
@@ -877,8 +882,20 @@ static struct iommu_device *tegra_smmu_probe_device(struct device *dev) index++; } -if (!smmu) -return ERR_PTR (-ENODEV ); +/* Any device should be able to get smmu pointer using fwspec */ +if (!smmu) { + fwspec =dev_iommu_fwspec_get (dev); +if (!fwspec || fwspec->ops != &tegra_smmu_ops) +return ERR_PTR (-ENODEV ); + + smmu =tegra_smmu_get_by_fwnode (fwspec->iommu_fwnode); + } + +/* NULL smmu pointer means that SMMU driver is not probed yet */ +if (unlikely (!smmu)) +return ERR_PTR (-EPROBE_DEFER ); + +dev_iommu_priv_set (dev, smmu);return &smmu->iommu; }
drivers/memory/tegra/mc.c +1 -1
@@ -735,7 +735,7 @@ static const struct dev_pm_ops tegra_mc_pm_ops = { .resume = tegra_mc_resume, }; -static struct platform_driver tegra_mc_driver = { +struct platform_driver tegra_mc_driver = { .driver = { .name ="tegra-mc" , .of_match_table = tegra_mc_of_match,
include/soc/tegra/mc.h +2 -0
@@ -184,4 +184,6 @@ struct tegra_mc {int tegra_mc_write_emem_configuration (struct tegra_mc *mc,unsigned long rate);unsigned int tegra_mc_get_emem_device_count (struct tegra_mc *mc); +extern struct platform_driver tegra_mc_driver; +#endif /* __SOC_TEGRA_MC_H__ */
Signature · 1 line
2.17.1
Re: [PATCH 3/5] iommu/tegra-smmu: Use iommu_fwspec in .probe_/.attach_device()
26.09.2020 11:07, Nicolin Chen пишет:
...
Review
Quoted 3 lines
(quoted diff) +3
-0+ /* NULL smmu pointer means that SMMU driver is not probed yet */ + if (unlikely(!smmu)) + return ERR_PTR(-EPROBE_DEFER);
Hello, Nicolin!
Please don't pollute code with likely/unlikely. This is not a performance-critical code.
...
Quoted 10 lines
(quoted diff) +1
-1-static struct platform_driver tegra_mc_driver = { +struct platform_driver tegra_mc_driver = { .driver = { .name = "tegra-mc", .of_match_table = tegra_mc_of_match,include/soc/tegra/mc.h +0
-0
include/soc/tegra/mc.h struct tegra_mc { · line 187
Quoted 4 lines
include/soc/tegra/mc.h +1
-0@@ -184,4 +184,6 @@ struct tegra_mc {int tegra_mc_write_emem_configuration (struct tegra_mc *mc,unsigned long rate);unsigned int tegra_mc_get_emem_device_count (struct tegra_mc *mc); +extern struct platform_driver tegra_mc_driver;
No global variables, please. See for the example:
https://elixir.bootlin.com/linux/v5.9-rc6/source/drivers/devfreq/tegra20-devfreq.c#L100
The tegra_get_memory_controller() is now needed by multiple Tegra drivers, I think it should be good to have it added into the MC driver and then make it globally available for all drivers by making use of of_find_matching_node_and_match().
drivers/memory/tegra/mc.c +22 -0
@@ -43,6 +43,29 @@ static const struct of_device_id tegra_mc_of_match[] = { };MODULE_DEVICE_TABLE (of, tegra_mc_of_match); +struct tegra_mc *tegra_get_memory_controller (void ) +{ +struct platform_device *pdev; +struct device_node *np; +struct tegra_mc *mc; + + np =of_find_matching_node_and_match (NULL , tegra_mc_of_match,NULL ); +if (!np) +return ERR_PTR (-ENOENT ); + + pdev =of_find_device_by_node (np); +of_node_put (np); +if (!pdev) +return ERR_PTR (-ENODEV ); + + mc =platform_get_drvdata (pdev); +if (!mc) +return ERR_PTR (-EPROBE_DEFER ); + +return mc; +} +EXPORT_SYMBOL_GPL (tegra_get_memory_controller);
Re: [PATCH 3/5] iommu/tegra-smmu: Use iommu_fwspec in .probe_/.attach_device()
Hi Dmitry,
Thank you for the review and comments!
Quoted 10 lines · Dmitry Osipenko
26.09.2020 11:07, Nicolin Chen пишет:
...Review
Quoted 3 lines
(quoted diff) +3
-0+ /* NULL smmu pointer means that SMMU driver is not probed yet */ + if (unlikely(!smmu)) + return ERR_PTR(-EPROBE_DEFER);Hello, Nicolin!
Please don't pollute code with likely/unlikely. This is not a performance-critical code.
Will drop it. Thanks.
Quoted 14 lines
...
Quoted 13 lines
(quoted diff) +1
-1-static struct platform_driver tegra_mc_driver = { +struct platform_driver tegra_mc_driver = { .driver = { .name = "tegra-mc", .of_match_table = tegra_mc_of_match,include/soc/tegra/mc.h +0
-0@@ -184,4 +184,6 @@ struct tegra_mc {int tegra_mc_write_emem_configuration (struct tegra_mc *mc,unsigned long rate);unsigned int tegra_mc_get_emem_device_count (struct tegra_mc *mc);
Quoted 5 lines
include/soc/tegra/mc.h struct tegra_mc { · line 187
Quoted 1 line
include/soc/tegra/mc.h +1
-0+extern struct platform_driver tegra_mc_driver;No global variables, please. See for the example:
https://elixir.bootlin.com/linux/v5.9-rc6/source/drivers/devfreq/tegra20-devfreq.c#L100
Will fix it. Thanks for the example.
Quoted 26 lines
The
tegra_get_memory_controller()is now needed by multiple Tegra drivers, I think it should be good to have it added into the MC driver and then make it globally available for all drivers by making use ofof_find_matching_node_and_match().drivers/memory/tegra/mc.c +12
-0@@ -43,6 +43,29 @@ static const struct of_device_id tegra_mc_of_match[] = { };MODULE_DEVICE_TABLE (of, tegra_mc_of_match); +struct tegra_mc *tegra_get_memory_controller (void ) +{ +struct platform_device *pdev; +struct device_node *np; +struct tegra_mc *mc; + + np =of_find_matching_node_and_match (NULL , tegra_mc_of_match,NULL ); +if (!np) +return ERR_PTR (-ENOENT ); + + pdev =of_find_device_by_node (np); +of_node_put (np);
drivers/memory/tegra/mc.c static const struct of_device_id tegra_mc_of_match[] = { · line 67
Quoted 10 lines
drivers/memory/tegra/mc.c +10
-0+if (!pdev) +return ERR_PTR (-ENODEV ); + + mc =platform_get_drvdata (pdev); +if (!mc) +return ERR_PTR (-EPROBE_DEFER ); + +return mc; +} +EXPORT_SYMBOL_GPL (tegra_get_memory_controller);
Will try this one and integrate into my next version.
Thanks
Nic
Re: [PATCH 3/5] iommu/tegra-smmu: Use iommu_fwspec in .probe_/.attach_device()
In reply to Dmitry OsipenkoQuoted 56 lines · Dmitry Osipenko
26.09.2020 11:07, Nicolin Chen пишет:
...Review
Quoted 3 lines
(quoted diff) +3
-0+ /* NULL smmu pointer means that SMMU driver is not probed yet */ + if (unlikely(!smmu)) + return ERR_PTR(-EPROBE_DEFER);Hello, Nicolin!
Please don't pollute code with likely/unlikely. This is not a performance-critical code.
...
Quoted 10 lines
(quoted diff) +1
-1-static struct platform_driver tegra_mc_driver = { +struct platform_driver tegra_mc_driver = { .driver = { .name = "tegra-mc", .of_match_table = tegra_mc_of_match,include/soc/tegra/mc.h +0
-0include/soc/tegra/mc.h struct tegra_mc { · line 187
Quoted 4 lines
include/soc/tegra/mc.h +1
-0@@ -184,4 +184,6 @@ struct tegra_mc {int tegra_mc_write_emem_configuration (struct tegra_mc *mc,unsigned long rate);unsigned int tegra_mc_get_emem_device_count (struct tegra_mc *mc); +extern struct platform_driver tegra_mc_driver;No global variables, please. See for the example:
https://elixir.bootlin.com/linux/v5.9-rc6/source/drivers/devfreq/tegra20-devfreq.c#L100
The
tegra_get_memory_controller()is now needed by multiple Tegra drivers, I think it should be good to have it added into the MC driver and then make it globally available for all drivers by making use ofof_find_matching_node_and_match().drivers/memory/tegra/mc.c +12
-0@@ -43,6 +43,29 @@ static const struct of_device_id tegra_mc_of_match[] = { };MODULE_DEVICE_TABLE (of, tegra_mc_of_match); +struct tegra_mc *tegra_get_memory_controller (void ) +{ +struct platform_device *pdev; +struct device_node *np; +struct tegra_mc *mc; + + np =of_find_matching_node_and_match (NULL , tegra_mc_of_match,NULL ); +if (!np) +return ERR_PTR (-ENOENT ); + + pdev =of_find_device_by_node (np); +of_node_put (np);
drivers/memory/tegra/mc.c static const struct of_device_id tegra_mc_of_match[] = { · line 67
Quoted 10 lines
drivers/memory/tegra/mc.c +10
-0+if (!pdev) +return ERR_PTR (-ENODEV ); + + mc =platform_get_drvdata (pdev); +if (!mc) +return ERR_PTR (-EPROBE_DEFER ); + +return mc; +} +EXPORT_SYMBOL_GPL (tegra_get_memory_controller);
We already have tegra_smmu_find(), which should be enough for this particular use-case.
Thierry
Re: [PATCH 3/5] iommu/tegra-smmu: Use iommu_fwspec in .probe_/.attach_device()
In reply to Nicolin ChenQuoted 28 lines · Nicolin Chen
The
tegra_smmu_probe_device()function searches in DT for the iommu phandler to get "smmu" pointer. This works for most of SMMU clients that exist in the DTB. But a PCI device will not be added to iommu, since it doesn't have a DT node.Fortunately, for a client with a DT node,
tegra_smmu_probe_device()callstegra_smmu_of_xlate()viategra_smmu_configure(), while for a PCI device,of_pci_iommu_init()in the IOMMU core calls .of_xlate() as well, even before runningtegra_smmu_probe_device(). And in both cases,tegra_smmu_of_xlate()prepares a validiommu_fwspecpointer that allows us to get themc->smmupointer viadev_get_drvdata()by callingdriver_find_device_by_fwnode().So this patch uses
iommu_fwspecin .probe_device() and related code for a client that does not exist in the DTB, especially a PCI one.Signed-off-by Nicolin Chen <[email protected]>--- drivers/iommu/tegra-smmu.c | 89 +++++++++++++++++++++++--------------- drivers/memory/tegra/mc.c | 2 +- include/soc/tegra/mc.h | 2 + 3 files changed, 56 insertions(+), 37 deletions(-)drivers/iommu/tegra-smmu.c +0
-0
drivers/iommu/tegra-smmu.c line 16
Quoted 4 lines
drivers/iommu/tegra-smmu.c +1
-0@@ -13,6 +13,7 @@#include <linux/platform_device.h> #include <linux/slab.h> #include <linux/spinlock.h> +#include <linux/dma-iommu.h>
Why is this needed? I don't see any of the symbols declared in that file used here.
Quoted 34 lines
drivers/iommu/tegra-smmu.c +4
-13#include <linux/dma-mapping.h> #include <soc/tegra/ahb.h> @@ -61,6 +62,8 @@ struct tegra_smmu_as {u32 attr; }; +static const struct iommu_ops tegra_smmu_ops; +static struct tegra_smmu_as *to_smmu_as (struct iommu_domain *dom) { return container_of(dom, struct tegra_smmu_as, domain);@@ -484,60 +487,49 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,static int tegra_smmu_attach_dev (struct iommu_domain *domain,struct device *dev) { +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev);struct tegra_smmu *smmu =dev_iommu_priv_get (dev);struct tegra_smmu_as *as =to_smmu_as (domain); -struct device_node *np = dev->of_node; -struct of_phandle_args args; -unsigned int index =0 ; -int err =0 ; - -while (!of_parse_phandle_with_args (np,"iommus" ,"#iommu-cells" , index, - &args)) { -unsigned int swgroup = args.args[0 ]; - -if (args.np != smmu->dev->of_node) { -of_node_put (args.np); -continue ; - } +int index, err =0 ;
drivers/iommu/tegra-smmu.c static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu, · line 501
Quoted 10 lines
drivers/iommu/tegra-smmu.c +5
-3-of_node_put (args.np); +if (!fwspec || fwspec->ops != &tegra_smmu_ops) +return -ENOENT ; +for (index =0 ; index < fwspec->num_ids; index++) { err =tegra_smmu_as_prepare (smmu, as); -if (err <0 ) -return err; +if (err) + goto err_disable;
I'd personally drop the err_ prefix here because it's pretty obvious that we're going to do this as a result of an error happening.
Quoted 6 lines
drivers/iommu/tegra-smmu.c +1
-2-tegra_smmu_enable (smmu, swgroup, as->id); - index++; +tegra_smmu_enable (smmu, fwspec->ids[index], as->id); }
drivers/iommu/tegra-smmu.c static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu, · line 515
Quoted 10 lines
drivers/iommu/tegra-smmu.c +6
-0if (index ==0 )return -ENODEV ;return 0 ; + +err_disable: +for (index--; index >=0 ; index--) { +tegra_smmu_disable (smmu, fwspec->ids[index], as->id); +tegra_smmu_as_unprepare (smmu, as); + }
I think a more idiomatic version of doing this would be:
while (index--) {
...
}Quoted 42 lines
drivers/iommu/tegra-smmu.c +11
-14+ +return err; }static void tegra_smmu_detach_dev (struct iommu_domain *domain,struct device *dev) { +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev);struct tegra_smmu_as *as =to_smmu_as (domain); -struct device_node *np = dev->of_node;struct tegra_smmu *smmu = as->smmu; -struct of_phandle_args args;unsigned int index =0 ; -while (!of_parse_phandle_with_args (np,"iommus" ,"#iommu-cells" , index, - &args)) { -unsigned int swgroup = args.args[0 ]; - -if (args.np != smmu->dev->of_node) { -of_node_put (args.np); -continue ; - } - -of_node_put (args.np); +if (!fwspec || fwspec->ops != &tegra_smmu_ops) +return ; -tegra_smmu_disable (smmu, swgroup, as->id); +for (index =0 ; index < fwspec->num_ids; index++) { +tegra_smmu_disable (smmu, fwspec->ids[index], as->id);tegra_smmu_as_unprepare (smmu, as); - index++; } }@@ -845,10 +837,25 @@ static int tegra_smmu_configure(struct tegra_smmu *smmu, struct device *dev,return 0 ; } +static struct tegra_smmu *tegra_smmu_get_by_fwnode (struct fwnode_handle *fwnode) +{ +struct device *dev =driver_find_device_by_fwnode (&tegra_mc_driver.driver, fwnode); +struct tegra_mc *mc;
drivers/iommu/tegra-smmu.c static int tegra_smmu_configure(struct tegra_smmu *smmu, struct device *dev, · line 853
Quoted 10 lines
drivers/iommu/tegra-smmu.c +10
-0+ +if (!dev) +return NULL ; + +put_device (dev); + mc =dev_get_drvdata (dev); + +return mc->smmu; +} +
As I mentioned in another reply, I think tegra_smmu_find() should be all you need in this case.
Quoted 27 lines
drivers/iommu/tegra-smmu.c +5
-4static struct iommu_device *tegra_smmu_probe_device (struct device *dev) {struct device_node *np = dev->of_node;struct tegra_smmu *smmu =NULL ; +struct iommu_fwspec *fwspec;struct of_phandle_args args;unsigned int index =0 ;int err;@@ -868,8 +875,6 @@ static struct iommu_device *tegra_smmu_probe_device(struct device *dev)* supported by the Linux kernel, so abort after the * first match. */ -dev_iommu_priv_set (dev, smmu); -break ; }@@ -877,8 +882,20 @@ static struct iommu_device *tegra_smmu_probe_device(struct device *dev) index++; } -if (!smmu) -return ERR_PTR (-ENODEV ); +/* Any device should be able to get smmu pointer using fwspec */ +if (!smmu) { + fwspec =dev_iommu_fwspec_get (dev); +if (!fwspec || fwspec->ops != &tegra_smmu_ops)
drivers/iommu/tegra-smmu.c static struct iommu_device *tegra_smmu_probe_device(struct device *dev) · line 898
Quoted 10 lines
drivers/iommu/tegra-smmu.c +10
-0+return ERR_PTR (-ENODEV ); + + smmu =tegra_smmu_get_by_fwnode (fwspec->iommu_fwnode); + } + +/* NULL smmu pointer means that SMMU driver is not probed yet */ +if (unlikely (!smmu)) +return ERR_PTR (-EPROBE_DEFER ); + +dev_iommu_priv_set (dev, smmu);
Can this not be unified with the OF code above? Basically in all cases where we use Tegra SMMU, a fwnode_node should correspond 1:1 to a struct device_node, which makes the code you added here effectively the same as what's already there.
Thierry
Re: [PATCH 3/5] iommu/tegra-smmu: Use iommu_fwspec in .probe_/.attach_device()
Quoted 9 lines · Thierry Reding
drivers/iommu/tegra-smmu.c line 16
Quoted 5 lines · Nicolin Chen
drivers/iommu/tegra-smmu.c +1
-0@@ -13,6 +13,7 @@#include <linux/platform_device.h> #include <linux/slab.h> #include <linux/spinlock.h> +#include <linux/dma-iommu.h> Why is this needed? I don't see any of the symbols declared in that file used here.
Hmm..I think I mixed with some other change that needs this header file. Removing it....
Quoted 38 lines
Quoted 38 lines
drivers/iommu/tegra-smmu.c +6
-14#include <linux/dma-mapping.h> #include <soc/tegra/ahb.h> @@ -61,6 +62,8 @@ struct tegra_smmu_as {u32 attr; }; +static const struct iommu_ops tegra_smmu_ops; +static struct tegra_smmu_as *to_smmu_as (struct iommu_domain *dom) { return container_of(dom, struct tegra_smmu_as, domain);@@ -484,60 +487,49 @@ static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu,static int tegra_smmu_attach_dev (struct iommu_domain *domain,struct device *dev) { +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (dev);struct tegra_smmu *smmu =dev_iommu_priv_get (dev);struct tegra_smmu_as *as =to_smmu_as (domain); -struct device_node *np = dev->of_node; -struct of_phandle_args args; -unsigned int index =0 ; -int err =0 ; - -while (!of_parse_phandle_with_args (np,"iommus" ,"#iommu-cells" , index, - &args)) { -unsigned int swgroup = args.args[0 ]; - -if (args.np != smmu->dev->of_node) { -of_node_put (args.np); -continue ; - } +int index, err =0 ; -of_node_put (args.np); +if (!fwspec || fwspec->ops != &tegra_smmu_ops) +return -ENOENT ;
Quoted 9 lines
drivers/iommu/tegra-smmu.c static void tegra_smmu_as_unprepare(struct tegra_smmu *smmu, · line 501
Quoted 6 lines
drivers/iommu/tegra-smmu.c +3
-2+for (index =0 ; index < fwspec->num_ids; index++) { err =tegra_smmu_as_prepare (smmu, as); -if (err <0 ) -return err; +if (err) + goto err_disable;I'd personally drop the
err_prefix here because it's pretty obvious that we're going to do this as a result of an error happening.
Changing to "goto disable".
Quoted 13 lines
Quoted 13 lines
drivers/iommu/tegra-smmu.c +9
-0@@ -845,10 +837,25 @@ static int tegra_smmu_configure(struct tegra_smmu *smmu, struct device *dev,return 0 ; } +static struct tegra_smmu *tegra_smmu_get_by_fwnode (struct fwnode_handle *fwnode) +{ +struct device *dev =driver_find_device_by_fwnode (&tegra_mc_driver.driver, fwnode); +struct tegra_mc *mc; + +if (!dev) +return NULL ; + +put_device (dev);
Quoted 8 lines
drivers/iommu/tegra-smmu.c static int tegra_smmu_configure(struct tegra_smmu *smmu, struct device *dev, · line 853
Quoted 5 lines
drivers/iommu/tegra-smmu.c +5
-0+ mc =dev_get_drvdata (dev); + +return mc->smmu; +} +As I mentioned in another reply, I think
tegra_smmu_find()should be all you need in this case.
This function is used by .probe_device() where its dev pointer is an SMMU client. IIUC, tegra_smmu_find() needs np pointer of "mc". For a PCI device that doesn't have a DT node with iommus property, not sure how we can use tegra_smmu_find().
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 3/5] iommu/tegra-smmu: Use iommu_fwspec in .probe_/.attach_device()
...
Quoted 7 lines
Quoted 2 lines
As I mentioned in another reply, I think
tegra_smmu_find()should be all you need in this case.This function is used by .probe_device() where its dev pointer is an SMMU client. IIUC,
tegra_smmu_find()needs np pointer of "mc". For a PCI device that doesn't have a DT node with iommus property, not sure how we can usetegra_smmu_find().
Perhaps you could get np from struct pci_dev.bus?
Re: [PATCH 3/5] iommu/tegra-smmu: Use iommu_fwspec in .probe_/.attach_device()
Quoted 10 lines · Dmitry Osipenko
...
Quoted 7 lines
Quoted 2 lines
As I mentioned in another reply, I think
tegra_smmu_find()should be all you need in this case.This function is used by .probe_device() where its dev pointer is an SMMU client. IIUC,
tegra_smmu_find()needs np pointer of "mc". For a PCI device that doesn't have a DT node with iommus property, not sure how we can usetegra_smmu_find().Perhaps you could get np from struct pci_dev.bus?
Possibly yes...but I was hoping the solution would be potentially reused by other devices that don't have DT nodes, USB for example, though I haven't tested with current version.
PATCH 4/5 iommu/tegra-smmu: Add PCI support
In reply to Nicolin ChenThis patch simply adds support for PCI devices.
--- drivers/iommu/tegra-smmu.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)
drivers/iommu/tegra-smmu.c +16 -1
@@ -15,6 +15,7 @@#include <linux/spinlock.h> #include <linux/dma-iommu.h> #include <linux/dma-mapping.h> +#include <linux/pci.h> #include <soc/tegra/ahb.h> #include <soc/tegra/mc.h>
@@ -935,6 +936,7 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)const struct tegra_smmu_group_soc *soc;struct tegra_smmu_group *group;int swgroup = fwspec->ids[0 ]; +bool pci =dev_is_pci (dev);struct iommu_group *grp;/* Find group_soc associating with swgroup */
@@ -961,7 +963,7 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev) group->smmu = smmu; group->soc = soc; - group->group =iommu_group_alloc (); + group->group = pci ?pci_device_group (dev) :iommu_group_alloc ();if (IS_ERR (group->group)) {devm_kfree (smmu->dev, group);mutex_unlock (&smmu->lock);
@@ -1180,6 +1182,19 @@ struct tegra_smmu *tegra_smmu_probe(struct device *dev,return ERR_PTR (err); } +#ifdef CONFIG_PCI +if (!iommu_present (&pci_bus_type)) { +pci_request_acs (); + err =bus_set_iommu (&pci_bus_type, &tegra_smmu_ops); +if (err <0 ) { +bus_set_iommu (&platform_bus_type,NULL ); +iommu_device_unregister (&smmu->iommu); +iommu_device_sysfs_remove (&smmu->iommu); +return ERR_PTR (err); + } + } +#endif +if (IS_ENABLED (CONFIG_DEBUG_FS ))tegra_smmu_debugfs_init (smmu);
Signature · 6 lines
2.17.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 4/5] iommu/tegra-smmu: Add PCI support
26.09.2020 11:07, Nicolin Chen пишет:
...
drivers/iommu/tegra-smmu.c struct tegra_smmu *tegra_smmu_probe(struct device *dev, · line 1186
Quoted 2 lines
drivers/iommu/tegra-smmu.c +2
-0+#ifdef CONFIG_PCI +if (!iommu_present (&pci_bus_type)) {
Is this iommu_present() check really needed?
drivers/iommu/tegra-smmu.c struct tegra_smmu *tegra_smmu_probe(struct device *dev, · line 1187
Quoted 1 line
drivers/iommu/tegra-smmu.c +1
-0+pci_request_acs ();
Shouldn't pci_request_acs() be invoked *after* bus_set_iommu() succeeds?
Quoted 14 lines
drivers/iommu/tegra-smmu.c +10
-0+ err =bus_set_iommu (&pci_bus_type, &tegra_smmu_ops); +if (err <0 ) { +bus_set_iommu (&platform_bus_type,NULL ); +iommu_device_unregister (&smmu->iommu); +iommu_device_sysfs_remove (&smmu->iommu); +return ERR_PTR (err); + } + } +#endif +if (IS_ENABLED (CONFIG_DEBUG_FS ))tegra_smmu_debugfs_init (smmu);
Re: [PATCH 4/5] iommu/tegra-smmu: Add PCI support
Quoted 10 lines · Dmitry Osipenko
26.09.2020 11:07, Nicolin Chen пишет:
...drivers/iommu/tegra-smmu.c struct tegra_smmu *tegra_smmu_probe(struct device *dev, · line 1186
Quoted 2 lines
drivers/iommu/tegra-smmu.c +2
-0+#ifdef CONFIG_PCI +if (!iommu_present (&pci_bus_type)) {Is this
iommu_present()check really needed?drivers/iommu/tegra-smmu.c struct tegra_smmu *tegra_smmu_probe(struct device *dev, · line 1187
Quoted 1 line
drivers/iommu/tegra-smmu.c +1
-0+pci_request_acs ();Shouldn't
pci_request_acs()be invoked *after*bus_set_iommu()succeeds?
Will move it after that. Thanks!
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 4/5] iommu/tegra-smmu: Add PCI support
In reply to Nicolin ChenQuoted 39 lines · Nicolin Chen
This patch simply adds support for PCI devices.
Signed-off-by Nicolin Chen <[email protected]>--- drivers/iommu/tegra-smmu.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)drivers/iommu/tegra-smmu.c +3
-1@@ -15,6 +15,7 @@#include <linux/spinlock.h> #include <linux/dma-iommu.h> #include <linux/dma-mapping.h> +#include <linux/pci.h> #include <soc/tegra/ahb.h> #include <soc/tegra/mc.h> @@ -935,6 +936,7 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev)const struct tegra_smmu_group_soc *soc;struct tegra_smmu_group *group;int swgroup = fwspec->ids[0 ]; +bool pci =dev_is_pci (dev);struct iommu_group *grp;/* Find group_soc associating with swgroup */ @@ -961,7 +963,7 @@ static struct iommu_group *tegra_smmu_device_group(struct device *dev) group->smmu = smmu; group->soc = soc; - group->group =iommu_group_alloc (); + group->group = pci ?pci_device_group (dev) :iommu_group_alloc ();if (IS_ERR (group->group)) {devm_kfree (smmu->dev, group);mutex_unlock (&smmu->lock);@@ -1180,6 +1182,19 @@ struct tegra_smmu *tegra_smmu_probe(struct device *dev,return ERR_PTR (err); }
drivers/iommu/tegra-smmu.c struct tegra_smmu *tegra_smmu_probe(struct device *dev, · line 1193
Quoted 10 lines
drivers/iommu/tegra-smmu.c +9
-0+#ifdef CONFIG_PCI +if (!iommu_present (&pci_bus_type)) { +pci_request_acs (); + err =bus_set_iommu (&pci_bus_type, &tegra_smmu_ops); +if (err <0 ) { +bus_set_iommu (&platform_bus_type,NULL ); +iommu_device_unregister (&smmu->iommu); +iommu_device_sysfs_remove (&smmu->iommu); +return ERR_PTR (err);
It might be worth factoring out the cleanup code now that there are multiple failures from which we may need to clean up.
Also, it'd be great if somehow we could do this without the #ifdef, but I guess since we're using the pci_bus_type global variable directly, there isn't much we can do here?
Thierry
Re: [PATCH 4/5] iommu/tegra-smmu: Add PCI support
Quoted 13 lines · Thierry Reding
drivers/iommu/tegra-smmu.c struct tegra_smmu *tegra_smmu_probe(struct device *dev, · line 1193
Quoted 9 lines · Nicolin Chen
drivers/iommu/tegra-smmu.c +9
-0+#ifdef CONFIG_PCI +if (!iommu_present (&pci_bus_type)) { +pci_request_acs (); + err =bus_set_iommu (&pci_bus_type, &tegra_smmu_ops); +if (err <0 ) { +bus_set_iommu (&platform_bus_type,NULL ); +iommu_device_unregister (&smmu->iommu); +iommu_device_sysfs_remove (&smmu->iommu); +return ERR_PTR (err);It might be worth factoring out the cleanup code now that there are multiple failures from which we may need to clean up.
Will do.
Quoted 3 lines
Also, it'd be great if somehow we could do this without the #ifdef, but I guess since we're using the
pci_bus_typeglobal variable directly, there isn't much we can do here?
Probably no -- both pci_bus_type and pci_request_acs seem to depend on it.
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 5/5 iommu/tegra-smmu: Add pagetable mappings to debugfs
In reply to Nicolin ChenThis patch dumps all active mapping entries from pagetable to a debugfs directory named "mappings".
Ataching an example:
SWGROUP: hc
ASID: 0
reg: 0x250PTB_ASID: 0xe00bb880as->pd_dma: 0xbb880000
{
[1023] 0xf00bb882 (1)
{
PDE ATTR PHYS IOVA
#1023 0x5 0x159f5d000 0xfffff000
}
}Total PDE count: 1
Total PTE count: 1
--- drivers/iommu/tegra-smmu.c | 130 +++++++++++++++++++++++++++++++++++-- 1 file changed, 125 insertions(+), 5 deletions(-)
drivers/iommu/tegra-smmu.c +125 -5
@@ -20,6 +20,11 @@#include <soc/tegra/ahb.h> #include <soc/tegra/mc.h> +struct tegra_smmu_group_debug { +const struct tegra_smmu_swgroup *group; +void *priv; +}; +struct tegra_smmu_group {struct list_head list;struct tegra_smmu *smmu;
@@ -48,6 +53,8 @@ struct tegra_smmu {struct dentry *debugfs;struct iommu_device iommu;/* IOMMU Core code handle */ + +struct tegra_smmu_group_debug *group_debug; };struct tegra_smmu_as {
@@ -155,6 +162,9 @@ static inline u32 smmu_readl(struct tegra_smmu *smmu, unsigned long offset)#define SMMU_PDE_ATTR (SMMU_PDE_READABLE | SMMU_PDE_WRITABLE | \ SMMU_PDE_NONSECURE) +#define SMMU_PTE_ATTR (SMMU_PTE_READABLE | SMMU_PTE_WRITABLE | \ + SMMU_PTE_NONSECURE) +#define SMMU_PTE_ATTR_SHIFT (29)static unsigned int iova_pd_index (unsigned long iova) {
@@ -337,7 +347,7 @@ static void tegra_smmu_domain_free(struct iommu_domain *domain) }static const struct tegra_smmu_swgroup * -tegra_smmu_find_swgroup (struct tegra_smmu *smmu,unsigned int swgroup) +tegra_smmu_find_swgroup (struct tegra_smmu *smmu,unsigned int swgroup,int *index) {const struct tegra_smmu_swgroup *group =NULL ;unsigned int i;
@@ -345,6 +355,8 @@ tegra_smmu_find_swgroup(struct tegra_smmu *smmu, unsigned int swgroup)for (i =0 ; i < smmu->soc->num_swgroups; i++) {if (smmu->soc->swgroups[i].swgroup == swgroup) { group = &smmu->soc->swgroups[i]; +if (index) + *index = i;break ; } }
@@ -353,19 +365,22 @@ tegra_smmu_find_swgroup(struct tegra_smmu *smmu, unsigned int swgroup) }static void tegra_smmu_enable (struct tegra_smmu *smmu,unsigned int swgroup, -unsigned int asid) +struct tegra_smmu_as *as) {const struct tegra_smmu_swgroup *group; +unsigned int asid = as->id;unsigned int i;u32 value; - group =tegra_smmu_find_swgroup (smmu, swgroup); + group =tegra_smmu_find_swgroup (smmu, swgroup, &i);if (group) { value =smmu_readl (smmu, group->reg); value &= ~SMMU_ASID_MASK ; value |=SMMU_ASID_VALUE (asid); value |=SMMU_ASID_ENABLE ;smmu_writel (smmu, value, group->reg); +if (smmu->group_debug) + smmu->group_debug[i].priv = as; }else {pr_warn ("%s group from swgroup %u not found\n" , __func__, swgroup);
@@ -392,13 +407,15 @@ static void tegra_smmu_disable(struct tegra_smmu *smmu, unsigned int swgroup,unsigned int i;u32 value; - group =tegra_smmu_find_swgroup (smmu, swgroup); + group =tegra_smmu_find_swgroup (smmu, swgroup, &i);if (group) { value =smmu_readl (smmu, group->reg); value &= ~SMMU_ASID_MASK ; value |=SMMU_ASID_VALUE (asid); value &= ~SMMU_ASID_ENABLE ;smmu_writel (smmu, value, group->reg); +if (smmu->group_debug) + smmu->group_debug[i].priv =NULL ; }for (i =0 ; i < smmu->soc->num_clients; i++) {
@@ -501,7 +518,7 @@ static int tegra_smmu_attach_dev(struct iommu_domain *domain,if (err) goto err_disable; -tegra_smmu_enable (smmu, fwspec->ids[index], as->id); +tegra_smmu_enable (smmu, fwspec->ids[index], as); }if (index ==0 )
@@ -1078,8 +1095,96 @@ static int tegra_smmu_clients_show(struct seq_file *s, void *data)DEFINE_SHOW_ATTRIBUTE (tegra_smmu_clients); +static int tegra_smmu_mappings_show (struct seq_file *s,void *data) +{ +struct tegra_smmu_group_debug *group_debug = s->private; +const struct tegra_smmu_swgroup *group; +struct tegra_smmu_as *as; +struct tegra_smmu *smmu; +int pd_index, pt_index; +u64 pte_count =0 ; +u32 pde_count =0 ; +u32 val, ptb_reg; +u32 *pd; + +if (!group_debug || !group_debug->priv || !group_debug->group) +return 0 ; + + group = group_debug->group; + as = group_debug->priv; + smmu = as->smmu; + + val =smmu_readl (smmu, group->reg) &SMMU_ASID_ENABLE ; +if (!val) +return 0 ; + + pd =page_address (as->pd); +if (!pd) +return 0 ; + +seq_printf (s,"\nSWGROUP: %s\nASID: %d\nreg: 0x%x\n" , group->name, + as->id, group->reg); + +mutex_lock (&smmu->lock); +smmu_writel (smmu, as->id &0x7f ,SMMU_PTB_ASID ); + ptb_reg =smmu_readl (smmu,SMMU_PTB_DATA ); +mutex_unlock (&smmu->lock); + +seq_printf (s,"PTB_ASID: 0x%x\nas->pd_dma: 0x%llx\n" , ptb_reg, as->pd_dma); +seq_puts (s,"{\n" ); + +for (pd_index =0 ; pd_index <SMMU_NUM_PDE ; pd_index++) { +struct page *pt; +u32 *addr; + +if (!as->count[pd_index] || !pd[pd_index]) +continue ; + + pde_count++; + pte_count += as->count[pd_index]; +seq_printf (s,"\t[%d] 0x%x (%d)\n" , + pd_index, pd[pd_index], as->count[pd_index]); + pt = as->pts[pd_index]; + addr =page_address (pt); + +seq_puts (s,"\t{\n" ); +seq_printf (s,"\t\t%-5s %-4s %12s %12s\n" ,"PDE" ,"ATTR" ,"PHYS" ,"IOVA" ); +for (pt_index =0 ; pt_index <SMMU_NUM_PTE ; pt_index++) { +u64 iova; + +if (!addr[pt_index]) +continue ; + + iova = ((dma_addr_t )pd_index & (SMMU_NUM_PDE -1 )) <<SMMU_PDE_SHIFT ; + iova |= ((dma_addr_t )pt_index & (SMMU_NUM_PTE -1 )) <<SMMU_PTE_SHIFT ; + +seq_printf (s,"\t\t#%-4d 0x%-4x 0x%-12llx 0x%-12llx\n" , + pt_index, addr[pt_index] >>SMMU_PTE_ATTR_SHIFT , +SMMU_PFN_PHYS (addr[pt_index] & ~SMMU_PTE_ATTR ), iova); + } +seq_puts (s,"\t}\n" ); + } +seq_puts (s,"}\n" ); +seq_printf (s,"Total PDE count: %d\n" , pde_count); +seq_printf (s,"Total PTE count: %lld\n" , pte_count); + +return 0 ; +} + +DEFINE_SHOW_ATTRIBUTE (tegra_smmu_mappings); +static void tegra_smmu_debugfs_init (struct tegra_smmu *smmu) { +const struct tegra_smmu_soc *soc = smmu->soc; +struct tegra_smmu_group_debug *group_debug; +struct device *dev = smmu->dev; +struct dentry *d; +int i; + + group_debug =devm_kzalloc (dev,sizeof (*group_debug) * soc->num_swgroups,GFP_KERNEL ); +if (!group_debug) +return ; + smmu->debugfs =debugfs_create_dir ("smmu" ,NULL );if (!smmu->debugfs)return ;
@@ -1088,6 +1193,21 @@ static void tegra_smmu_debugfs_init(struct tegra_smmu *smmu) &tegra_smmu_swgroups_fops);debugfs_create_file ("clients" ,S_IRUGO , smmu->debugfs, smmu, &tegra_smmu_clients_fops); + d =debugfs_create_dir ("mappings" , smmu->debugfs); + +for (i =0 ; i < soc->num_swgroups; i++) { +const struct tegra_smmu_swgroup *group = &soc->swgroups[i]; + +if (!group->name) +continue ; + + group_debug[i].group = group; + +debugfs_create_file (group->name,0444 , d, &group_debug[i], + &tegra_smmu_mappings_fops); + } + + smmu->group_debug = group_debug; }static void tegra_smmu_debugfs_exit (struct tegra_smmu *smmu)
Signature · 6 lines
2.17.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 5/5] iommu/tegra-smmu: Add pagetable mappings to debugfs
26.09.2020 11:07, Nicolin Chen пишет:
...
Quoted 23 lines
drivers/iommu/tegra-smmu.c +23
-0+static int tegra_smmu_mappings_show (struct seq_file *s,void *data) +{ +struct tegra_smmu_group_debug *group_debug = s->private; +const struct tegra_smmu_swgroup *group; +struct tegra_smmu_as *as; +struct tegra_smmu *smmu; +int pd_index, pt_index; +u64 pte_count =0 ; +u32 pde_count =0 ; +u32 val, ptb_reg; +u32 *pd; + +if (!group_debug || !group_debug->priv || !group_debug->group) +return 0 ; + + group = group_debug->group; + as = group_debug->priv; + smmu = as->smmu; + + val =smmu_readl (smmu, group->reg) &SMMU_ASID_ENABLE ; +if (!val) +return 0 ; +
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1130
Quoted 10 lines
drivers/iommu/tegra-smmu.c +10
-0+ pd =page_address (as->pd); +if (!pd) +return 0 ; + +seq_printf (s,"\nSWGROUP: %s\nASID: %d\nreg: 0x%x\n" , group->name, + as->id, group->reg); + +mutex_lock (&smmu->lock); +smmu_writel (smmu, as->id &0x7f ,SMMU_PTB_ASID ); + ptb_reg =smmu_readl (smmu,SMMU_PTB_DATA );
I think the whole traverse needs a locking protection, doesn't it?
Quoted 8 lines
drivers/iommu/tegra-smmu.c +8
-0+mutex_unlock (&smmu->lock); + +seq_printf (s,"PTB_ASID: 0x%x\nas->pd_dma: 0x%llx\n" , ptb_reg, as->pd_dma); +seq_puts (s,"{\n" ); + +for (pd_index =0 ; pd_index <SMMU_NUM_PDE ; pd_index++) { +struct page *pt; +u32 *addr;
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1148
Quoted 10 lines
drivers/iommu/tegra-smmu.c +10
-0+ +if (!as->count[pd_index] || !pd[pd_index]) +continue ; + + pde_count++; + pte_count += as->count[pd_index]; +seq_printf (s,"\t[%d] 0x%x (%d)\n" , + pd_index, pd[pd_index], as->count[pd_index]); + pt = as->pts[pd_index]; + addr =page_address (pt);
This needs as->lock protection.
Quoted 25 lines
drivers/iommu/tegra-smmu.c +25
-0+seq_puts (s,"\t{\n" ); +seq_printf (s,"\t\t%-5s %-4s %12s %12s\n" ,"PDE" ,"ATTR" ,"PHYS" ,"IOVA" ); +for (pt_index =0 ; pt_index <SMMU_NUM_PTE ; pt_index++) { +u64 iova; + +if (!addr[pt_index]) +continue ; + + iova = ((dma_addr_t )pd_index & (SMMU_NUM_PDE -1 )) <<SMMU_PDE_SHIFT ; + iova |= ((dma_addr_t )pt_index & (SMMU_NUM_PTE -1 )) <<SMMU_PTE_SHIFT ; + +seq_printf (s,"\t\t#%-4d 0x%-4x 0x%-12llx 0x%-12llx\n" , + pt_index, addr[pt_index] >>SMMU_PTE_ATTR_SHIFT , +SMMU_PFN_PHYS (addr[pt_index] & ~SMMU_PTE_ATTR ), iova); + } +seq_puts (s,"\t}\n" ); + } +seq_puts (s,"}\n" ); +seq_printf (s,"Total PDE count: %d\n" , pde_count); +seq_printf (s,"Total PTE count: %lld\n" , pte_count); + +return 0 ; +} + +DEFINE_SHOW_ATTRIBUTE (tegra_smmu_mappings);
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1184
Quoted 10 lines
drivers/iommu/tegra-smmu.c +8
-0+static void tegra_smmu_debugfs_init (struct tegra_smmu *smmu) { +const struct tegra_smmu_soc *soc = smmu->soc; +struct tegra_smmu_group_debug *group_debug; +struct device *dev = smmu->dev; +struct dentry *d; +int i; + + group_debug =devm_kzalloc (dev,sizeof (*group_debug) * soc->num_swgroups,GFP_KERNEL );
devm_kcalloc()
Re: [PATCH 5/5] iommu/tegra-smmu: Add pagetable mappings to debugfs
Hi Dmitry,
Thank you for the review.
Quoted 29 lines · Dmitry Osipenko
26.09.2020 11:07, Nicolin Chen пишет:
...Quoted 27 lines
drivers/iommu/tegra-smmu.c +27
-0+static int tegra_smmu_mappings_show (struct seq_file *s,void *data) +{ +struct tegra_smmu_group_debug *group_debug = s->private; +const struct tegra_smmu_swgroup *group; +struct tegra_smmu_as *as; +struct tegra_smmu *smmu; +int pd_index, pt_index; +u64 pte_count =0 ; +u32 pde_count =0 ; +u32 val, ptb_reg; +u32 *pd; + +if (!group_debug || !group_debug->priv || !group_debug->group) +return 0 ; + + group = group_debug->group; + as = group_debug->priv; + smmu = as->smmu; + + val =smmu_readl (smmu, group->reg) &SMMU_ASID_ENABLE ; +if (!val) +return 0 ; + + pd =page_address (as->pd); +if (!pd) +return 0 ; +
Quoted 8 lines
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1130
Quoted 6 lines
drivers/iommu/tegra-smmu.c +6
-0+seq_printf (s,"\nSWGROUP: %s\nASID: %d\nreg: 0x%x\n" , group->name, + as->id, group->reg); + +mutex_lock (&smmu->lock); +smmu_writel (smmu, as->id &0x7f ,SMMU_PTB_ASID ); + ptb_reg =smmu_readl (smmu,SMMU_PTB_DATA );I think the whole traverse needs a locking protection, doesn't it?
Hmm..probably would be nicer. Will move the mutex to the top.
Quoted 12 lines
Quoted 12 lines
drivers/iommu/tegra-smmu.c +12
-0+mutex_unlock (&smmu->lock); + +seq_printf (s,"PTB_ASID: 0x%x\nas->pd_dma: 0x%llx\n" , ptb_reg, as->pd_dma); +seq_puts (s,"{\n" ); + +for (pd_index =0 ; pd_index <SMMU_NUM_PDE ; pd_index++) { +struct page *pt; +u32 *addr; + +if (!as->count[pd_index] || !pd[pd_index]) +continue ; +
Quoted 8 lines
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1148
Quoted 6 lines
drivers/iommu/tegra-smmu.c +6
-0+ pde_count++; + pte_count += as->count[pd_index]; +seq_printf (s,"\t[%d] 0x%x (%d)\n" , + pd_index, pd[pd_index], as->count[pd_index]); + pt = as->pts[pd_index]; + addr =page_address (pt);This needs
as->lockprotection.
Will add that.
Quoted 29 lines
Quoted 29 lines
drivers/iommu/tegra-smmu.c +27
-0+seq_puts (s,"\t{\n" ); +seq_printf (s,"\t\t%-5s %-4s %12s %12s\n" ,"PDE" ,"ATTR" ,"PHYS" ,"IOVA" ); +for (pt_index =0 ; pt_index <SMMU_NUM_PTE ; pt_index++) { +u64 iova; + +if (!addr[pt_index]) +continue ; + + iova = ((dma_addr_t )pd_index & (SMMU_NUM_PDE -1 )) <<SMMU_PDE_SHIFT ; + iova |= ((dma_addr_t )pt_index & (SMMU_NUM_PTE -1 )) <<SMMU_PTE_SHIFT ; + +seq_printf (s,"\t\t#%-4d 0x%-4x 0x%-12llx 0x%-12llx\n" , + pt_index, addr[pt_index] >>SMMU_PTE_ATTR_SHIFT , +SMMU_PFN_PHYS (addr[pt_index] & ~SMMU_PTE_ATTR ), iova); + } +seq_puts (s,"\t}\n" ); + } +seq_puts (s,"}\n" ); +seq_printf (s,"Total PDE count: %d\n" , pde_count); +seq_printf (s,"Total PTE count: %lld\n" , pte_count); + +return 0 ; +} + +DEFINE_SHOW_ATTRIBUTE (tegra_smmu_mappings); +static void tegra_smmu_debugfs_init (struct tegra_smmu *smmu) { +const struct tegra_smmu_soc *soc = smmu->soc;
Quoted 8 lines
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1184
Quoted 6 lines
drivers/iommu/tegra-smmu.c +6
-0+struct tegra_smmu_group_debug *group_debug; +struct device *dev = smmu->dev; +struct dentry *d; +int i; + + group_debug =devm_kzalloc (dev,sizeof (*group_debug) * soc->num_swgroups,GFP_KERNEL );
devm_kcalloc()
Will change it.
Thanks
Nic
Re: [PATCH 5/5] iommu/tegra-smmu: Add pagetable mappings to debugfs
26.09.2020 23:47, Nicolin Chen пишет:
...
Quoted 18 lines
Quoted 7 lines
Quoted 7 lines
drivers/iommu/tegra-smmu.c +7
-0+for (pd_index =0 ; pd_index <SMMU_NUM_PDE ; pd_index++) { +struct page *pt; +u32 *addr; + +if (!as->count[pd_index] || !pd[pd_index]) +continue ; +Quoted 8 lines
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1148
Quoted 6 lines
drivers/iommu/tegra-smmu.c +6
-0+ pde_count++; + pte_count += as->count[pd_index]; +seq_printf (s,"\t[%d] 0x%x (%d)\n" , + pd_index, pd[pd_index], as->count[pd_index]); + pt = as->pts[pd_index]; + addr =page_address (pt);This needs
as->lockprotection.Will add that.
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
drivers/iommu/tegra-smmu.c +6
-0+seq_puts (s,"\t{\n" ); +seq_printf (s,"\t\t%-5s %-4s %12s %12s\n" ,"PDE" ,"ATTR" ,"PHYS" ,"IOVA" ); +for (pt_index =0 ; pt_index <SMMU_NUM_PTE ; pt_index++) { +u64 iova; + +if (!addr[pt_index])Quoted 8 lines
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1163
Quoted 8 lines
drivers/iommu/tegra-smmu.c +8
-0+continue ; + + iova = ((dma_addr_t )pd_index & (SMMU_NUM_PDE -1 )) <<SMMU_PDE_SHIFT ; + iova |= ((dma_addr_t )pt_index & (SMMU_NUM_PTE -1 )) <<SMMU_PTE_SHIFT ; + +seq_printf (s,"\t\t#%-4d 0x%-4x 0x%-12llx 0x%-12llx\n" , + pt_index, addr[pt_index] >>SMMU_PTE_ATTR_SHIFT , +SMMU_PFN_PHYS (addr[pt_index] & ~SMMU_PTE_ATTR ), iova);
Please also note that the addr[pt_index] needs to be protected as well. Perhaps you could temporally bump the as->count.
Re: [PATCH 5/5] iommu/tegra-smmu: Add pagetable mappings to debugfs
In reply to Nicolin Chen26.09.2020 11:07, Nicolin Chen пишет:
...
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1141
Quoted 6 lines
drivers/iommu/tegra-smmu.c +6
-0+for (pd_index =0 ; pd_index <SMMU_NUM_PDE ; pd_index++) { +struct page *pt; +u32 *addr; + +if (!as->count[pd_index] || !pd[pd_index]) +continue ;
I guess the idea of this patch is to print out the hardware state, isn't it? Hence the as->count shouldn't be checked here.
Quoted 12 lines
drivers/iommu/tegra-smmu.c +11
-0+ pde_count++; + pte_count += as->count[pd_index]; +seq_printf (s,"\t[%d] 0x%x (%d)\n" , + pd_index, pd[pd_index], as->count[pd_index]); + pt = as->pts[pd_index]; + addr =page_address (pt); + +seq_puts (s,"\t{\n" ); +seq_printf (s,"\t\t%-5s %-4s %12s %12s\n" ,"PDE" ,"ATTR" ,"PHYS" ,"IOVA" ); +for (pt_index =0 ; pt_index <SMMU_NUM_PTE ; pt_index++) { +u64 iova;
drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1163
Quoted 10 lines
drivers/iommu/tegra-smmu.c +10
-0+ +if (!addr[pt_index]) +continue ; + + iova = ((dma_addr_t )pd_index & (SMMU_NUM_PDE -1 )) <<SMMU_PDE_SHIFT ; + iova |= ((dma_addr_t )pt_index & (SMMU_NUM_PTE -1 )) <<SMMU_PTE_SHIFT ; + +seq_printf (s,"\t\t#%-4d 0x%-4x 0x%-12llx 0x%-12llx\n" , + pt_index, addr[pt_index] >>SMMU_PTE_ATTR_SHIFT , +SMMU_PFN_PHYS (addr[pt_index] & ~SMMU_PTE_ATTR ), iova);
Would be nice if you could improve the output formatting by printing out contiguous ranges that have the same ATTRs, otherwise it could be a bit too large and unpractical output in a case if lots of pages are mapped.
Re: [PATCH 5/5] iommu/tegra-smmu: Add pagetable mappings to debugfs
27.09.2020 00:24, Dmitry Osipenko пишет:
Quoted 11 lines
26.09.2020 11:07, Nicolin Chen пишет:
...drivers/iommu/tegra-smmu.c static int tegra_smmu_clients_show(struct seq_file *s, void *data) · line 1141
Quoted 6 lines
drivers/iommu/tegra-smmu.c +6
-0+for (pd_index =0 ; pd_index <SMMU_NUM_PDE ; pd_index++) { +struct page *pt; +u32 *addr; + +if (!as->count[pd_index] || !pd[pd_index]) +continue ;I guess the idea of this patch is to print out the hardware state, isn't it? Hence the
as->countshouldn't be checked here.
Perhaps also will be good to check whether the state of !as->count[pd_index] matches state of !pd[pd_index].
WARN_ON_ONCE(!as->count[pd_index] ^ !pd[pd_index])