iommu/arm: Add support for non-coherent page tables
Subsystems (3)
cover iommu/arm: Add support for non-coherent page tables
As discussed in the Qcom system cache support thread [1], it is imperative that we enable the support for non-cacheable page tables for SMMU implementations for which removing snoop latency on walks by making mappings as non-cacheable, outweighs the cost of cache maintenance on PTE updates.
This series adds a new SMMU device tree option to let the particular SMMU configuration setup cacheable or non-cacheable mappings for page-tables out of box. We set a new quirk for i/o page tables - IO_PGTABLE_QUIRK_NON_COHERENT, that lets us set different TCR configurations.
This quirk enables the non-cacheable page tables for all masters sitting on SMMU. Should this control be available per smmu_domain as each master may have a different perf requirement? Enabling this for the entire SMMU may not be desirable for all masters.
[1] https://lore.kernel.org/patchwork/patch/1020906/
Vivek Gautam (2):
iommu/io-pgtable-arm: Add support for non-coherent page tables iommu/arm-smmu: Add support for non-coherent page table mappings
drivers/iommu/arm-smmu.c | 7 +++++++ drivers/iommu/io-pgtable-arm.c | 17 ++++++++++++----- drivers/iommu/io-pgtable.h | 6 ++++++ 3 files changed, 25 insertions(+), 5 deletions(-)
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
PATCH 1/2 iommu/io-pgtable-arm: Add support for non-coherent page tables
Quoted 1 line
From Robin's comment [1] about touching TCR configurations -
"TBH if we're going to touch the TCR attributes at all then we should probably correct that sloppiness first - there's an occasional argument for using non-cacheable pagetables even on a coherent SMMU if reducing snoop traffic/latency on walks outweighs the cost of cache maintenance on PTE updates, but anyone thinking they can get that by overriding dma-coherent silently gets the worst of both worlds thanks to this current TCR value."
We have IO_PGTABLE_QUIRK_NO_DMA quirk present, but we don't force anybody _not_ using dma-coherent smmu to have non-cacheable page table mappings.
Having another quirk flag can help in having non-cacheable memory for page tables once and for all.
[1] https://lore.kernel.org/patchwork/patch/1020906/
--- drivers/iommu/io-pgtable-arm.c | 17 ++++++++++++----- drivers/iommu/io-pgtable.h | 6 ++++++ 2 files changed, 18 insertions(+), 5 deletions(-)
drivers/iommu/io-pgtable-arm.c +12 -5
@@ -780,7 +780,8 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie)struct arm_lpae_io_pgtable *data;if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS |IO_PGTABLE_QUIRK_NO_DMA | -IO_PGTABLE_QUIRK_NON_STRICT )) +IO_PGTABLE_QUIRK_NON_STRICT | +IO_PGTABLE_QUIRK_NON_COHERENT ))return NULL ; data =arm_lpae_alloc_pgtable (cfg);
@@ -788,9 +789,14 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie)return NULL ;/* TCR */ - reg = (ARM_LPAE_TCR_SH_IS <<ARM_LPAE_TCR_SH0_SHIFT ) | - (ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_IRGN0_SHIFT ) | - (ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_ORGN0_SHIFT ); + reg =ARM_LPAE_TCR_SH_IS <<ARM_LPAE_TCR_SH0_SHIFT ; + +if (cfg->quirks &IO_PGTABLE_QUIRK_NON_COHERENT ) + reg |=ARM_LPAE_TCR_RGN_NC <<ARM_LPAE_TCR_IRGN0_SHIFT | +ARM_LPAE_TCR_RGN_NC <<ARM_LPAE_TCR_ORGN0_SHIFT ; +else + reg |=ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_IRGN0_SHIFT | +ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_ORGN0_SHIFT ;switch (ARM_LPAE_GRANULE (data)) {case SZ_4K :
@@ -873,7 +879,8 @@ arm_64_lpae_alloc_pgtable_s2(struct io_pgtable_cfg *cfg, void *cookie)/* The NS quirk doesn't apply at stage 2 */ if (cfg->quirks & ~(IO_PGTABLE_QUIRK_NO_DMA | -IO_PGTABLE_QUIRK_NON_STRICT )) +IO_PGTABLE_QUIRK_NON_STRICT | +IO_PGTABLE_QUIRK_NON_COHERENT ))return NULL ; data =arm_lpae_alloc_pgtable (cfg);
drivers/iommu/io-pgtable.h +6 -0
@@ -75,6 +75,11 @@ struct io_pgtable_cfg {* IO_PGTABLE_QUIRK_NON_STRICT: Skip issuing synchronous leaf TLBIs * on unmap, for DMA domains using the flush queue mechanism for * delayed invalidation. +* +* IO_PGTABLE_QUIRK_NON_COHERENT: Enforce non-cacheable mappings for +* pagetables even on a coherent SMMU for cases where reducing +* snoop traffic/latency on walks outweighs the cost of cache +* maintenance on PTE updates. */ #define IO_PGTABLE_QUIRK_ARM_NS BIT(0)#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)
@@ -82,6 +87,7 @@ struct io_pgtable_cfg {#define IO_PGTABLE_QUIRK_ARM_MTK_4GB BIT(3)#define IO_PGTABLE_QUIRK_NO_DMA BIT(4)#define IO_PGTABLE_QUIRK_NON_STRICT BIT(5) +#define IO_PGTABLE_QUIRK_NON_COHERENT BIT(6)unsigned long quirks;unsigned long pgsize_bitmap;unsigned int ias;
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Robin Murphy
reviewerRe: [PATCH 1/2] iommu/io-pgtable-arm: Add support for non-coherent page tables
Quoted 71 lines · Vivek Gautam
From Robin's comment [1] about touching TCR configurations -
"TBH if we're going to touch the TCR attributes at all then we should probably correct that sloppiness first - there's an occasional argument for using non-cacheable pagetables even on a coherent SMMU if reducing snoop traffic/latency on walks outweighs the cost of cache maintenance on PTE updates, but anyone thinking they can get that by overriding dma-coherent silently gets the worst of both worlds thanks to this current TCR value."
We have
IO_PGTABLE_QUIRK_NO_DMAquirk present, but we don't force anybody _not_ using dma-coherent smmu to have non-cacheable page table mappings.
Having another quirk flag can help in having non-cacheable memory for page tables once and for all.[1] https://lore.kernel.org/patchwork/patch/1020906/
Signed-off-by Vivek Gautam <[email protected]>--- drivers/iommu/io-pgtable-arm.c | 17 ++++++++++++----- drivers/iommu/io-pgtable.h | 6 ++++++ 2 files changed, 18 insertions(+), 5 deletions(-)drivers/iommu/io-pgtable-arm.c +12
-5@@ -780,7 +780,8 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie)struct arm_lpae_io_pgtable *data;if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS |IO_PGTABLE_QUIRK_NO_DMA | -IO_PGTABLE_QUIRK_NON_STRICT )) +IO_PGTABLE_QUIRK_NON_STRICT | +IO_PGTABLE_QUIRK_NON_COHERENT ))return NULL ; data =arm_lpae_alloc_pgtable (cfg);@@ -788,9 +789,14 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie)return NULL ;/* TCR */ - reg = (ARM_LPAE_TCR_SH_IS <<ARM_LPAE_TCR_SH0_SHIFT ) | - (ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_IRGN0_SHIFT ) | - (ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_ORGN0_SHIFT ); + reg =ARM_LPAE_TCR_SH_IS <<ARM_LPAE_TCR_SH0_SHIFT ; + +if (cfg->quirks &IO_PGTABLE_QUIRK_NON_COHERENT ) + reg |=ARM_LPAE_TCR_RGN_NC <<ARM_LPAE_TCR_IRGN0_SHIFT | +ARM_LPAE_TCR_RGN_NC <<ARM_LPAE_TCR_ORGN0_SHIFT ; +else + reg |=ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_IRGN0_SHIFT | +ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_ORGN0_SHIFT ;switch (ARM_LPAE_GRANULE (data)) {case SZ_4K :@@ -873,7 +879,8 @@ arm_64_lpae_alloc_pgtable_s2(struct io_pgtable_cfg *cfg, void *cookie)/* The NS quirk doesn't apply at stage 2 */ if (cfg->quirks & ~(IO_PGTABLE_QUIRK_NO_DMA | -IO_PGTABLE_QUIRK_NON_STRICT )) +IO_PGTABLE_QUIRK_NON_STRICT | +IO_PGTABLE_QUIRK_NON_COHERENT ))return NULL ; data =arm_lpae_alloc_pgtable (cfg);drivers/iommu/io-pgtable.h +0
-0
drivers/iommu/io-pgtable.h struct io_pgtable_cfg { · line 82
Quoted 8 lines
drivers/iommu/io-pgtable.h +5
-0@@ -75,6 +75,11 @@ struct io_pgtable_cfg {* IO_PGTABLE_QUIRK_NON_STRICT: Skip issuing synchronous leaf TLBIs * on unmap, for DMA domains using the flush queue mechanism for * delayed invalidation. +* +* IO_PGTABLE_QUIRK_NON_COHERENT: Enforce non-cacheable mappings for +* pagetables even on a coherent SMMU for cases where reducing +* snoop traffic/latency on walks outweighs the cost of cache +* maintenance on PTE updates.
Hmm, we can't actually "enforce" anything with this as-is - all we're doing is setting the attributes that the IOMMU will use for pagetable walks, and that has no impact on how the CPU actually writes PTEs to memory. In particular, in the case of a hardware-coherent IOMMU which is described as such, even if we make the dma_map/sync calls they still won't do anything since they 'know' that the IOMMU is coherent. Thus if we then set up a non-cacheable TCR we would have no proper means of making pagetables correctly visible to the walker.
Aw crap, this is turning out to be a microcosm of the PCIe no-snoop mess... :(
To start with, at least, what we want is to set a non-cacheable TCR if the IOMMU is *not* coherent (as far as Linux is concerned - that includes the firmware-lying-about-the-hardware situation I was alluding to before), but even that isn't necessarily as straightforward as it seems. AFAICS, if QUIRK_NO_DMA is set then we definitely have to use a cacheable TCR; we can't strictly rely on the inverse being true, but in practice we *might* get away with it since we already disallow most cases in which the DMA API calls would actually do anything for a known-coherent IOMMU device.
Robin.
Quoted 12 lines
drivers/iommu/io-pgtable.h +1
-0*/ #define IO_PGTABLE_QUIRK_ARM_NS BIT(0)#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)@@ -82,6 +87,7 @@ struct io_pgtable_cfg {#define IO_PGTABLE_QUIRK_ARM_MTK_4GB BIT(3)#define IO_PGTABLE_QUIRK_NO_DMA BIT(4)#define IO_PGTABLE_QUIRK_NON_STRICT BIT(5) +#define IO_PGTABLE_QUIRK_NON_COHERENT BIT(6)unsigned long quirks;unsigned long pgsize_bitmap;unsigned int ias;
Re: [PATCH 1/2] iommu/io-pgtable-arm: Add support for non-coherent page tables
Quoted 82 lines · Robin Murphy
Quoted 71 lines · Vivek Gautam
From Robin's comment [1] about touching TCR configurations -
"TBH if we're going to touch the TCR attributes at all then we should probably correct that sloppiness first - there's an occasional argument for using non-cacheable pagetables even on a coherent SMMU if reducing snoop traffic/latency on walks outweighs the cost of cache maintenance on PTE updates, but anyone thinking they can get that by overriding dma-coherent silently gets the worst of both worlds thanks to this current TCR value."
We have
IO_PGTABLE_QUIRK_NO_DMAquirk present, but we don't force anybody _not_ using dma-coherent smmu to have non-cacheable page table mappings.
Having another quirk flag can help in having non-cacheable memory for page tables once and for all.[1] https://lore.kernel.org/patchwork/patch/1020906/
Signed-off-by Vivek Gautam <[email protected]>--- drivers/iommu/io-pgtable-arm.c | 17 ++++++++++++----- drivers/iommu/io-pgtable.h | 6 ++++++ 2 files changed, 18 insertions(+), 5 deletions(-)drivers/iommu/io-pgtable-arm.c +12
-5@@ -780,7 +780,8 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie)struct arm_lpae_io_pgtable *data;if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS |IO_PGTABLE_QUIRK_NO_DMA | -IO_PGTABLE_QUIRK_NON_STRICT )) +IO_PGTABLE_QUIRK_NON_STRICT | +IO_PGTABLE_QUIRK_NON_COHERENT ))return NULL ; data =arm_lpae_alloc_pgtable (cfg);@@ -788,9 +789,14 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie)return NULL ;/* TCR */ - reg = (ARM_LPAE_TCR_SH_IS <<ARM_LPAE_TCR_SH0_SHIFT ) | - (ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_IRGN0_SHIFT ) | - (ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_ORGN0_SHIFT ); + reg =ARM_LPAE_TCR_SH_IS <<ARM_LPAE_TCR_SH0_SHIFT ; + +if (cfg->quirks &IO_PGTABLE_QUIRK_NON_COHERENT ) + reg |=ARM_LPAE_TCR_RGN_NC <<ARM_LPAE_TCR_IRGN0_SHIFT | +ARM_LPAE_TCR_RGN_NC <<ARM_LPAE_TCR_ORGN0_SHIFT ; +else + reg |=ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_IRGN0_SHIFT | +ARM_LPAE_TCR_RGN_WBWA <<ARM_LPAE_TCR_ORGN0_SHIFT ;switch (ARM_LPAE_GRANULE (data)) {case SZ_4K :@@ -873,7 +879,8 @@ arm_64_lpae_alloc_pgtable_s2(struct io_pgtable_cfg *cfg, void *cookie)/* The NS quirk doesn't apply at stage 2 */ if (cfg->quirks & ~(IO_PGTABLE_QUIRK_NO_DMA | -IO_PGTABLE_QUIRK_NON_STRICT )) +IO_PGTABLE_QUIRK_NON_STRICT | +IO_PGTABLE_QUIRK_NON_COHERENT ))return NULL ; data =arm_lpae_alloc_pgtable (cfg);drivers/iommu/io-pgtable.h +0
-0drivers/iommu/io-pgtable.h struct io_pgtable_cfg { · line 82
Quoted 8 lines
drivers/iommu/io-pgtable.h +5
-0@@ -75,6 +75,11 @@ struct io_pgtable_cfg {* IO_PGTABLE_QUIRK_NON_STRICT: Skip issuing synchronous leaf TLBIs * on unmap, for DMA domains using the flush queue mechanism for * delayed invalidation. +* +* IO_PGTABLE_QUIRK_NON_COHERENT: Enforce non-cacheable mappings for +* pagetables even on a coherent SMMU for cases where reducing +* snoop traffic/latency on walks outweighs the cost of cache +* maintenance on PTE updates.
Quoted 8 lines
Hmm, we can't actually "enforce" anything with this as-is - all we're doing is setting the attributes that the IOMMU will use for pagetable walks, and that has no impact on how the CPU actually writes PTEs to memory. In particular, in the case of a hardware-coherent IOMMU which is described as such, even if we make the dma_map/sync calls they still won't do anything since they 'know' that the IOMMU is coherent. Thus if we then set up a non-cacheable TCR we would have no proper means of making pagetables correctly visible to the walker.
Right, I get this point. With non-cacheable TCR, the PTW will always go to the main memory rather then snooping in CPU caches for the latest page tables.
Quoted 10 lines
Aw crap, this is turning out to be a microcosm of the PCIe no-snoop mess... :(
To start with, at least, what we want is to set a non-cacheable TCR if the IOMMU is *not* coherent (as far as Linux is concerned - that includes the firmware-lying-about-the-hardware situation I was alluding to before), but even that isn't necessarily as straightforward as it seems. AFAICS, if
QUIRK_NO_DMAis set then we definitely have to use a cacheable TCR;
Okay, so for QUIRK_NO_DMA we set IRGN0 and ORGN0 to WBWA in TCR, But, for SMMUs that omit 'dma-coherent' and thus QUIRK_NO_DMA is not set do we allow them to have a Non-Cacheable set to IRGN0 and ORGN0, as the PTW will anyways have to read from memory after the CPU flushes the PTEs to the memory (which we are already doing).
Regards
Vivek
Quoted 19 lines
we can't strictly rely on the inverse being true, but in practice we *might* get away with it since we already disallow most cases in which the DMA API calls would actually do anything for a known-coherent IOMMU device.
Robin.
Quoted 12 lines
drivers/iommu/io-pgtable.h +1
-0*/ #define IO_PGTABLE_QUIRK_ARM_NS BIT(0)#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)@@ -82,6 +87,7 @@ struct io_pgtable_cfg {#define IO_PGTABLE_QUIRK_ARM_MTK_4GB BIT(3)#define IO_PGTABLE_QUIRK_NO_DMA BIT(4)#define IO_PGTABLE_QUIRK_NON_STRICT BIT(5) +#define IO_PGTABLE_QUIRK_NON_COHERENT BIT(6)unsigned long quirks;unsigned long pgsize_bitmap;unsigned int ias;
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
PATCH 2/2 iommu/arm-smmu: Add support for non-coherent page table mappings
In reply to Vivek GautamAdding a device tree option for arm smmu to enable non-cacheable memory for page tables.
We already enable a smmu feature for coherent walk based on whether the smmu device is dma-coherent or not. Have an option to enable non-cacheable page table memory to force set it for particular smmu devices.
--- drivers/iommu/arm-smmu.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/iommu/arm-smmu.c +7 -0
@@ -188,6 +188,7 @@ struct arm_smmu_device {u32 features;#define ARM_SMMU_OPT_SECURE_CFG_ACCESS (1 << 0) +#define ARM_SMMU_OPT_PGTBL_NON_COHERENT (1 << 1)u32 options;enum arm_smmu_arch_version version;enum arm_smmu_implementation model;
@@ -273,6 +274,7 @@ static bool using_legacy_binding, using_generic_binding;static struct arm_smmu_option_prop arm_smmu_options[] = { {ARM_SMMU_OPT_SECURE_CFG_ACCESS ,"calxeda,smmu-secure-config-access" }, + {ARM_SMMU_OPT_PGTBL_NON_COHERENT ,"arm,smmu-pgtable-non-coherent" }, {0 ,NULL }, };
@@ -902,6 +904,11 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,if (smmu_domain->non_strict) pgtbl_cfg.quirks |=IO_PGTABLE_QUIRK_NON_STRICT ; +/* Non coherent page table mappings only for Stage-1 */ +if (smmu->options &ARM_SMMU_OPT_PGTBL_NON_COHERENT && + smmu_domain->stage ==ARM_SMMU_DOMAIN_S1 ) + pgtbl_cfg.quirks |=IO_PGTABLE_QUIRK_NON_COHERENT ; + smmu_domain->smmu = smmu; pgtbl_ops =alloc_io_pgtable_ops (fmt, &pgtbl_cfg, smmu_domain);if (!pgtbl_ops) {
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH 2/2] iommu/arm-smmu: Add support for non-coherent page table mappings
Quoted 6 lines · Vivek Gautam
Adding a device tree option for arm smmu to enable non-cacheable memory for page tables.
We already enable a smmu feature for coherent walk based on whether the smmu device is dma-coherent or not. Have an option to enable non-cacheable page table memory to force set it for particular smmu devices.
Hmm, I must be missing something here. What is the difference between this new property, and simply omitting dma-coherent on the SMMU?
Will
Re: [PATCH 2/2] iommu/arm-smmu: Add support for non-coherent page table mappings
Hi Will,
Quoted 11 lines · Will Deacon
Quoted 6 lines · Vivek Gautam
Adding a device tree option for arm smmu to enable non-cacheable memory for page tables.
We already enable a smmu feature for coherent walk based on whether the smmu device is dma-coherent or not. Have an option to enable non-cacheable page table memory to force set it for particular smmu devices.Hmm, I must be missing something here. What is the difference between this new property, and simply omitting dma-coherent on the SMMU?
So, this is what I understood from the email thread for Last level cache support -
Robin pointed to the fact that we may need to add support for setting non-cacheable
mappings in the TCR.
Currently, we don't do that for SMMUs that omit dma-coherent. We rely on the interconnect to handle the configuration set in TCR, and let interconnect
ignore the cacheability if it can't support.
Moreover, Robin suggested that we should take care of SMMUs, for which removing snoop latency on walks by making mappings as non-cacheable outweighs the cost of cache maintenance on PTE updates.
So, this change adds another property to do this non-cacheable mappings explicitly. As I pointed, omitting 'dma-coherent', and corresponding IO_PGTABLE_QUIRK_NO_DMA' does takes care of few things.
Should we handle the TCR settings too with this quirk?
Regards
Vivek
Quoted 6 lines
Will
_______________________________________________
iommu mailing list iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
--
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH 2/2] iommu/arm-smmu: Add support for non-coherent page table mappings
Quoted 12 lines · Vivek Gautam
Quoted 10 lines · Will Deacon
Quoted 6 lines · Vivek Gautam
Adding a device tree option for arm smmu to enable non-cacheable memory for page tables.
We already enable a smmu feature for coherent walk based on whether the smmu device is dma-coherent or not. Have an option to enable non-cacheable page table memory to force set it for particular smmu devices.Hmm, I must be missing something here. What is the difference between this new property, and simply omitting dma-coherent on the SMMU?
Quoted 9 lines
So, this is what I understood from the email thread for Last level cache support -
Robin pointed to the fact that we may need to add support for setting non-cacheable
mappings in the TCR.
Currently, we don't do that for SMMUs that omit dma-coherent. We rely on the interconnect to handle the configuration set in TCR, and let interconnect
ignore the cacheability if it can't support.
I think that's a bug. With that fixed, can you get what you want by omitting "dma-coherent"?
Will
Re: [PATCH 2/2] iommu/arm-smmu: Add support for non-coherent page table mappings
Hi Will,
Quoted 24 lines · Will Deacon
Quoted 12 lines · Vivek Gautam
Quoted 10 lines · Will Deacon
Quoted 6 lines · Vivek Gautam
Adding a device tree option for arm smmu to enable non-cacheable memory for page tables.
We already enable a smmu feature for coherent walk based on whether the smmu device is dma-coherent or not. Have an option to enable non-cacheable page table memory to force set it for particular smmu devices.Hmm, I must be missing something here. What is the difference between this new property, and simply omitting dma-coherent on the SMMU?
Quoted 9 lines
So, this is what I understood from the email thread for Last level cache support -
Robin pointed to the fact that we may need to add support for setting non-cacheable
mappings in the TCR.
Currently, we don't do that for SMMUs that omit dma-coherent. We rely on the interconnect to handle the configuration set in TCR, and let interconnect
ignore the cacheability if it can't support.
Quoted 2 lines
I think that's a bug. With that fixed, can you get what you want by omitting "dma-coherent"?
Based on the discussion on the first patch in this series [1], I can update the series.
First thing can be -
if QUIRK_NO_DMA is set (i.e. the IOMMU _is_ coherent) then we use a cacheable TCR;
So, we may need an additional check for this when setting the TCR.
For the second case -
IOMMUs that are *not* coherent, i.e ones that are omitting 'dma-coherent' property,
anyways have to access the page table directly from memory. We take care of the CPU
side of this by allocating non-coherent memory, and making sure that we sync the PTEs from map call.
Shouldn't we mark TCR for these IOMMUs as non-cacheable for inner and outer cacheability attribute?
[1] https://lore.kernel.org/patchwork/patch/1032939/
Regards
Vivek
Quoted 2 lines
Will
Signature · 2 lines
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation