iommu/io-pgtable-arm: Mali LPAE improvements
Subsystems (2)
Robin Murphy
reviewerv2 cover iommu/io-pgtable-arm: Mali LPAE improvements
Hi Will,
Although the rc1 tag is yet to appear, today's master seems close enough (the DRM and IOMMU pulls are landed at least) so here's the promised rebase of these patches with tags added, and some minor commit message improvements for good measure.
Robin.
Robin Murphy (3):
iommu/io-pgtable-arm: Correct Mali attributes iommu/io-pgtable-arm: Support all Mali configurations iommu/io-pgtable-arm: Allow coherent walks for Mali
drivers/iommu/io-pgtable-arm.c | 61 ++++++++++++++++++++++++++-------- 1 file changed, 48 insertions(+), 13 deletions(-)
Signature · 6 lines
2.21.0.dirty _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Robin Murphy
reviewerPATCH v2 1/3 iommu/io-pgtable-arm: Correct Mali attributes
Whilst Midgard's MEMATTR follows a similar principle to the VMSA MAIR, the actual attribute values differ, so although it currently appears to work to some degree, we probably shouldn't be using our standard stage 1 MAIR for that. Instead, generate a reasonable MEMATTR with attribute values borrowed from the kbase driver; at this point we'll be overriding or ignoring pretty much all of the LPAE config, so just implement these Mali details in a dedicated allocator instead of pretending to subclass the standard VMSA format.
--- drivers/iommu/io-pgtable-arm.c | 53 +++++++++++++++++++++++++--------- 1 file changed, 40 insertions(+), 13 deletions(-)
drivers/iommu/io-pgtable-arm.c +40 -13
@@ -166,6 +166,9 @@#define ARM_MALI_LPAE_TTBR_READ_INNER BIT(2)#define ARM_MALI_LPAE_TTBR_SHARE_OUTER BIT(4) +#define ARM_MALI_LPAE_MEMATTR_IMP_DEF 0x88ULL +#define ARM_MALI_LPAE_MEMATTR_WRITE_ALLOC 0x8DULL +/* IOPTE accessors */ #define iopte_deref (pte,d) __va(iopte_to_paddr(pte, d))
@@ -1015,27 +1018,51 @@ arm_32_lpae_alloc_pgtable_s2(struct io_pgtable_cfg *cfg, void *cookie)static struct io_pgtable *arm_mali_lpae_alloc_pgtable (struct io_pgtable_cfg *cfg,void *cookie) { -struct io_pgtable *iop; +struct arm_lpae_io_pgtable *data; + +/* No quirks for Mali (hopefully) */ +if (cfg->quirks) +return NULL ;if (cfg->ias !=48 || cfg->oas >40 )return NULL ; cfg->pgsize_bitmap &= (SZ_4K |SZ_2M |SZ_1G ); - iop =arm_64_lpae_alloc_pgtable_s1 (cfg, cookie); -if (iop) { -u64 mair, ttbr; -/* Copy values as union fields overlap */ - mair = cfg->arm_lpae_s1_cfg.mair[0 ]; - ttbr = cfg->arm_lpae_s1_cfg.ttbr[0 ]; + data =arm_lpae_alloc_pgtable (cfg); +if (!data) +return NULL ; - cfg->arm_mali_lpae_cfg.memattr = mair; - cfg->arm_mali_lpae_cfg.transtab = ttbr | -ARM_MALI_LPAE_TTBR_READ_INNER | -ARM_MALI_LPAE_TTBR_ADRMODE_TABLE ; - } +/* +* MEMATTR: Mali has no actual notion of a non-cacheable type, so the +* best we can do is mimic the out-of-tree driver and hope that the +* "implementation-defined caching policy" is good enough. Similarly, +* we'll use it for the sake of a valid attribute for our 'device' +* index, although callers should never request that in practice. +*/ + cfg->arm_mali_lpae_cfg.memattr = + (ARM_MALI_LPAE_MEMATTR_IMP_DEF + <<ARM_LPAE_MAIR_ATTR_SHIFT (ARM_LPAE_MAIR_ATTR_IDX_NC )) | + (ARM_MALI_LPAE_MEMATTR_WRITE_ALLOC + <<ARM_LPAE_MAIR_ATTR_SHIFT (ARM_LPAE_MAIR_ATTR_IDX_CACHE )) | + (ARM_MALI_LPAE_MEMATTR_IMP_DEF + <<ARM_LPAE_MAIR_ATTR_SHIFT (ARM_LPAE_MAIR_ATTR_IDX_DEV )); -return iop; + data->pgd =__arm_lpae_alloc_pages (data->pgd_size,GFP_KERNEL , cfg); +if (!data->pgd) + goto out_free_data; + +/* Ensure the empty pgd is visible before TRANSTAB can be written */ +wmb (); + + cfg->arm_mali_lpae_cfg.transtab =virt_to_phys (data->pgd) | +ARM_MALI_LPAE_TTBR_READ_INNER | +ARM_MALI_LPAE_TTBR_ADRMODE_TABLE ; +return &data->iop; + +out_free_data: +kfree (data); +return NULL ; }struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns = {
Signature · 6 lines
2.21.0.dirty _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 1/3] iommu/io-pgtable-arm: Correct Mali attributes
Quoted 10 lines · Robin Murphy
Whilst Midgard's MEMATTR follows a similar principle to the VMSA MAIR, the actual attribute values differ, so although it currently appears to work to some degree, we probably shouldn't be using our standard stage 1 MAIR for that. Instead, generate a reasonable MEMATTR with attribute values borrowed from the kbase driver; at this point we'll be overriding or ignoring pretty much all of the LPAE config, so just implement these Mali details in a dedicated allocator instead of pretending to subclass the standard VMSA format.
Quoted 7 lines
Fixes d08d42de6432 ("iommu: io-pgtable: Add ARM Mali midgard MMU page table format")Tested-by Neil Armstrong <[email protected]>Reviewed-by Steven Price <[email protected]>Signed-off-by Robin Murphy <[email protected]>--- drivers/iommu/io-pgtable-arm.c | 53 +++++++++++++++++++++++++--------- 1 file changed, 40 insertions(+), 13 deletions(-)
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Robin Murphy
reviewerPATCH v2 2/3 iommu/io-pgtable-arm: Support all Mali configurations
In reply to Robin MurphyIn principle, Midgard GPUs supporting smaller VA sizes should only require 3-level pagetables, since level 0 only resolves bits 48:40 of the address. However, the kbase driver does not appear to have any notion of a variable start level, and empirically T720 and T820 rapidly blow up with translation faults unless given a full 4-level table, despite only supporting a 33-bit VA size.
The 'real' IAS value is still valuable in terms of validating addresses on map/unmap, so tweak the allocator to allow smaller values while still forcing the resultant tables to the full 4 levels. As far as I can test, this should make all known Midgard variants happy.
--- drivers/iommu/io-pgtable-arm.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/iommu/io-pgtable-arm.c +6 -1
@@ -1024,7 +1024,7 @@ arm_mali_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie)if (cfg->quirks)return NULL ; -if (cfg->ias !=48 || cfg->oas >40 ) +if (cfg->ias >48 || cfg->oas >40 )return NULL ; cfg->pgsize_bitmap &= (SZ_4K |SZ_2M |SZ_1G );
@@ -1033,6 +1033,11 @@ arm_mali_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie)if (!data)return NULL ; +/* Mali seems to need a full 4-level table regardless of IAS */ +if (data->levels <ARM_LPAE_MAX_LEVELS ) { + data->levels =ARM_LPAE_MAX_LEVELS ; + data->pgd_size =sizeof (arm_lpae_iopte); + }/* * MEMATTR: Mali has no actual notion of a non-cacheable type, so the * best we can do is mimic the out-of-tree driver and hope that the
Signature · 6 lines
2.21.0.dirty _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 2/3] iommu/io-pgtable-arm: Support all Mali configurations
Quoted 13 lines · Robin Murphy
In principle, Midgard GPUs supporting smaller VA sizes should only require 3-level pagetables, since level 0 only resolves bits 48:40 of the address. However, the kbase driver does not appear to have any notion of a variable start level, and empirically T720 and T820 rapidly blow up with translation faults unless given a full 4-level table, despite only supporting a 33-bit VA size.
The 'real' IAS value is still valuable in terms of validating addresses on map/unmap, so tweak the allocator to allow smaller values while still forcing the resultant tables to the full 4 levels. As far as I can test, this should make all known Midgard variants happy.
Quoted 7 lines
Fixes d08d42de6432 ("iommu: io-pgtable: Add ARM Mali midgard MMU page table format")Tested-by Neil Armstrong <[email protected]>Reviewed-by Steven Price <[email protected]>Signed-off-by Robin Murphy <[email protected]>--- drivers/iommu/io-pgtable-arm.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Robin Murphy
reviewerPATCH v2 3/3 iommu/io-pgtable-arm: Allow coherent walks for Mali
In reply to Robin MurphyMidgard GPUs have ACE-Lite master interfaces which allows systems to integrate them in an I/O-coherent manner. It seems that from the GPU's viewpoint, the rest of the system is its outer shareable domain, and so even when snoop signals are wired up, they are only emitted for outer shareable accesses. As such, setting the TTBR_SHARE_OUTER bit does indeed get coherent pagetable walks working nicely for the coherent T620 in the Arm Juno SoC.
Exploiting coherency for data accesses is more of a challenge, since not only do we need to get the GPU MMU attributes right but we'd also have to avoid pgprot_writecombine creating an attribute mismatch on the CPU side, so we won't try wiring that up just yet.
--- drivers/iommu/io-pgtable-arm.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/iommu/io-pgtable-arm.c +3 -0
@@ -1063,6 +1063,9 @@ arm_mali_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie) cfg->arm_mali_lpae_cfg.transtab =virt_to_phys (data->pgd) |ARM_MALI_LPAE_TTBR_READ_INNER |ARM_MALI_LPAE_TTBR_ADRMODE_TABLE ; +if (cfg->coherent_walk) + cfg->arm_mali_lpae_cfg.transtab |=ARM_MALI_LPAE_TTBR_SHARE_OUTER ; +return &data->iop; out_free_data:
Signature · 6 lines
2.21.0.dirty _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 3/3] iommu/io-pgtable-arm: Allow coherent walks for Mali
Quoted 14 lines · Robin Murphy
Midgard GPUs have ACE-Lite master interfaces which allows systems to integrate them in an I/O-coherent manner. It seems that from the GPU's viewpoint, the rest of the system is its outer shareable domain, and so even when snoop signals are wired up, they are only emitted for outer shareable accesses. As such, setting the
TTBR_SHARE_OUTERbit does indeed get coherent pagetable walks working nicely for the coherent T620 in the Arm Juno SoC.Exploiting coherency for data accesses is more of a challenge, since not only do we need to get the GPU MMU attributes right but we'd also have to avoid
pgprot_writecombinecreating an attribute mismatch on the CPU side, so we won't try wiring that up just yet.
Quoted 5 lines
Reviewed-by Steven Price <[email protected]>Signed-off-by Robin Murphy <[email protected]>--- drivers/iommu/io-pgtable-arm.c | 3 +++ 1 file changed, 3 insertions(+)
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu