Skip to contentSkip to conversation

drm/panfrost: Add AArch64 page table format support

PATCH In review Rob Herring 4 participants 6 messages May 13, 2019 – Jun 27, 2019

Subsystems (3)
lore
Share & tools
lore

Rob Herring

author maintainer
#0 lore raw

PATCH 1/1 drm/panfrost: Add AArch64 page table format support

Bifrost GPUs use the standard format stage 1 LPAE page tables matching the io-pgtable ARM_64_LPAE_S1 format. The one difference is the TCR or TRANSCFG register as the Mali driver calls it has its own custom layout.

---
This and compatible strings should be enough for enabling bifrost. There's some other feature and issue differences, but I think they all either don't matter (because of differences in panfrost) or I've already handled them.

This is only compile tested as I don't have h/w. Running the existing IGT tests may be sufficient to test this. We should get an error with the command stream rather than a MMU fault if this is working correctly.

Rob

 drivers/gpu/drm/panfrost/panfrost_mmu.c  | 28 +++++++++++++++++++++++-
 drivers/gpu/drm/panfrost/panfrost_regs.h |  3 +++
 2 files changed, 30 insertions(+), 1 deletion(-)
drivers/gpu/drm/panfrost/panfrost_mmu.c +27 -1
index 762b1bd2a8c2..41d184fab07f 100644
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier:	GPL-2.0
 /* Copyright 2019 Linaro, Ltd, Rob Herring <[email protected]> */
+/* Copyright (C) 2019 Arm Ltd. */
 #include <linux/bitfield.h>
 #include <linux/delay.h>
 #include <linux/interrupt.h>
@@ -111,6 +112,14 @@ void panfrost_mmu_enable(struct panfrost_device *pfdev, u32 as_nr)
 	u64 transtab = cfg->arm_mali_lpae_cfg.transtab;
 	u64 memattr = cfg->arm_mali_lpae_cfg.memattr;
 
+	if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+		transtab = cfg->arm_lpae_s1_cfg.ttbr[0];
+		memattr = cfg->arm_lpae_s1_cfg.mair[0];
+	} else {
+		transtab = cfg->arm_mali_lpae_cfg.transtab;
+		memattr = cfg->arm_mali_lpae_cfg.memattr;
+	}
+
 	mmu_write(pfdev, MMU_INT_CLEAR, ~0);
 	mmu_write(pfdev, MMU_INT_MASK, ~0);
 
@@ -123,6 +132,14 @@ void panfrost_mmu_enable(struct panfrost_device *pfdev, u32 as_nr)
 	mmu_write(pfdev, AS_MEMATTR_LO(as_nr), memattr & 0xffffffffUL);
 	mmu_write(pfdev, AS_MEMATTR_HI(as_nr), memattr >> 32);
 
+	if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+		/* TODO: handle system coherency */
+		mmu_write(pfdev, AS_TRANSCFG_LO(as_nr),
+			AS_TRANSCFG_PTW_MEMATTR_WRITE_BACK |
+			AS_TRANSCFG_ADRMODE_AARCH64_4K);
+		mmu_write(pfdev, AS_TRANSCFG_HI(as_nr), 0);
+	}
+
 	write_cmd(pfdev, as_nr, AS_COMMAND_UPDATE);
 }
 
@@ -134,6 +151,11 @@ static void mmu_disable(struct panfrost_device *pfdev, u32 as_nr)
 	mmu_write(pfdev, AS_MEMATTR_LO(as_nr), 0);
 	mmu_write(pfdev, AS_MEMATTR_HI(as_nr), 0);
 
+	if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+		mmu_write(pfdev, AS_TRANSCFG_LO(as_nr), 0);
+		mmu_write(pfdev, AS_TRANSCFG_HI(as_nr), 0);
+	}
+
 	write_cmd(pfdev, as_nr, AS_COMMAND_UPDATE);
 }
 
@@ -335,6 +357,7 @@ static irqreturn_t panfrost_mmu_irq_handler(int irq, void *data)
 int panfrost_mmu_init(struct panfrost_device *pfdev)
 {
 	struct io_pgtable_ops *pgtbl_ops;
+	enum io_pgtable_fmt pgt_fmt = ARM_MALI_LPAE;
 	int err, irq;
 
 	pfdev->mmu = devm_kzalloc(pfdev->dev, sizeof(*pfdev->mmu), GFP_KERNEL);
@@ -365,7 +388,10 @@ int panfrost_mmu_init(struct panfrost_device *pfdev)
 		.iommu_dev	= pfdev->dev,
 	};
 
-	pgtbl_ops = alloc_io_pgtable_ops(ARM_MALI_LPAE, &pfdev->mmu->pgtbl_cfg,
+	if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU))
+		pgt_fmt = ARM_64_LPAE_S1;
+
+	pgtbl_ops = alloc_io_pgtable_ops(pgt_fmt, &pfdev->mmu->pgtbl_cfg,
 					 pfdev);
 	if (!pgtbl_ops)
 		return -ENOMEM;
drivers/gpu/drm/panfrost/panfrost_regs.h +3 -0
index 578c5fc2188b..31211df83c30 100644
@@ -287,6 +287,9 @@
 #define AS_TRANSTAB_LPAE_READ_INNER		BIT(2)
 #define AS_TRANSTAB_LPAE_SHARE_OUTER		BIT(4)
 
+#define AS_TRANSCFG_ADRMODE_AARCH64_4K		6
+#define AS_TRANSCFG_PTW_MEMATTR_WRITE_BACK	(2 << 24)
+
 #define AS_STATUS_AS_ACTIVE			0x01
 
 #define AS_FAULTSTATUS_ACCESS_TYPE_MASK		(0x3 << 8)
Signature · 6 lines
2.20.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#1 lore raw

Robin, Steven,

would you or someone else at Arm be able to run the IGT tests [0] on 5.2-rc2 with this patch on top?

I don't have any hw with Bifrost and am not planning to work on the userspace any time soon, but I think it would be good to at least check that the kernel doesn't have any obvious bugs.

[0] https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/tests/panfrost_submit.c

Thanks,

Tomeu

Quoted 112 lines · Rob Herring

Bifrost GPUs use the standard format stage 1 LPAE page tables matching the io-pgtable ARM_64_LPAE_S1 format. The one difference is the TCR or TRANSCFG register as the Mali driver calls it has its own custom layout.

---
This and compatible strings should be enough for enabling bifrost. There's some other feature and issue differences, but I think they all either don't matter (because of differences in panfrost) or I've already handled them.

This is only compile tested as I don't have h/w. Running the existing IGT tests may be sufficient to test this. We should get an error with the command stream rather than a MMU fault if this is working correctly.

Rob

 drivers/gpu/drm/panfrost/panfrost_mmu.c  | 28 +++++++++++++++++++++++-
 drivers/gpu/drm/panfrost/panfrost_regs.h |  3 +++
 2 files changed, 30 insertions(+), 1 deletion(-)
drivers/gpu/drm/panfrost/panfrost_mmu.c +27 -1
index 762b1bd2a8c2..41d184fab07f 100644
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier:    GPL-2.0
 /* Copyright 2019 Linaro, Ltd, Rob Herring <[email protected]> */
+/* Copyright (C) 2019 Arm Ltd. */
 #include <linux/bitfield.h>
 #include <linux/delay.h>
 #include <linux/interrupt.h>
@@ -111,6 +112,14 @@ void panfrost_mmu_enable(struct panfrost_device *pfdev, u32 as_nr)
        u64 transtab = cfg->arm_mali_lpae_cfg.transtab;
        u64 memattr = cfg->arm_mali_lpae_cfg.memattr;
 
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+               transtab = cfg->arm_lpae_s1_cfg.ttbr[0];
+               memattr = cfg->arm_lpae_s1_cfg.mair[0];
+       } else {
+               transtab = cfg->arm_mali_lpae_cfg.transtab;
+               memattr = cfg->arm_mali_lpae_cfg.memattr;
+       }
+
        mmu_write(pfdev, MMU_INT_CLEAR, ~0);
        mmu_write(pfdev, MMU_INT_MASK, ~0);
 
@@ -123,6 +132,14 @@ void panfrost_mmu_enable(struct panfrost_device *pfdev, u32 as_nr)
        mmu_write(pfdev, AS_MEMATTR_LO(as_nr), memattr & 0xffffffffUL);
        mmu_write(pfdev, AS_MEMATTR_HI(as_nr), memattr >> 32);
 
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+               /* TODO: handle system coherency */
+               mmu_write(pfdev, AS_TRANSCFG_LO(as_nr),
+                       AS_TRANSCFG_PTW_MEMATTR_WRITE_BACK |
+                       AS_TRANSCFG_ADRMODE_AARCH64_4K);
+               mmu_write(pfdev, AS_TRANSCFG_HI(as_nr), 0);
+       }
+
        write_cmd(pfdev, as_nr, AS_COMMAND_UPDATE);
 }
 
@@ -134,6 +151,11 @@ static void mmu_disable(struct panfrost_device *pfdev, u32 as_nr)
        mmu_write(pfdev, AS_MEMATTR_LO(as_nr), 0);
        mmu_write(pfdev, AS_MEMATTR_HI(as_nr), 0);
 
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+               mmu_write(pfdev, AS_TRANSCFG_LO(as_nr), 0);
+               mmu_write(pfdev, AS_TRANSCFG_HI(as_nr), 0);
+       }
+
        write_cmd(pfdev, as_nr, AS_COMMAND_UPDATE);
 }
 
@@ -335,6 +357,7 @@ static irqreturn_t panfrost_mmu_irq_handler(int irq, void *data)
 int panfrost_mmu_init(struct panfrost_device *pfdev)
 {
        struct io_pgtable_ops *pgtbl_ops;
+       enum io_pgtable_fmt pgt_fmt = ARM_MALI_LPAE;
        int err, irq;
 
        pfdev->mmu = devm_kzalloc(pfdev->dev, sizeof(*pfdev->mmu), GFP_KERNEL);
@@ -365,7 +388,10 @@ int panfrost_mmu_init(struct panfrost_device *pfdev)
                .iommu_dev      = pfdev->dev,
        };
 
-       pgtbl_ops = alloc_io_pgtable_ops(ARM_MALI_LPAE, &pfdev->mmu->pgtbl_cfg,
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU))
+               pgt_fmt = ARM_64_LPAE_S1;
+
+       pgtbl_ops = alloc_io_pgtable_ops(pgt_fmt, &pfdev->mmu->pgtbl_cfg,
                                         pfdev);
        if (!pgtbl_ops)
                return -ENOMEM;
drivers/gpu/drm/panfrost/panfrost_regs.h +3 -1
index 578c5fc2188b..31211df83c30 100644
@@ -287,6 +287,9 @@
 #define AS_TRANSTAB_LPAE_READ_INNER            BIT(2)
 #define AS_TRANSTAB_LPAE_SHARE_OUTER           BIT(4)
 
+#define AS_TRANSCFG_ADRMODE_AARCH64_4K         6
+#define AS_TRANSCFG_PTW_MEMATTR_WRITE_BACK     (2 << 24)
+
 #define AS_STATUS_AS_ACTIVE                    0x01
 
 #define AS_FAULTSTATUS_ACCESS_TYPE_MASK                (0x3 << 8)
--

2.20.1

Quoted 4 lines

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#2 lore raw

Hi Tomeu, Rob,

Quoted 8 lines · Tomeu Vizoso

Robin, Steven,

would you or someone else at Arm be able to run the IGT tests [0] on 5.2-rc2 with this patch on top?

I don't have any hw with Bifrost and am not planning to work on the userspace any time soon, but I think it would be good to at least check that the kernel doesn't have any obvious bugs.

I've managed to cobble something together which appears to sort-of-work, although I don't have the knowledge, time or patience to dive down the rabbit-hole of getting a working Arm DDK driver to actually prove the setup. The immediate observation is that I get a lot of this:

[  305.602001] panfrost 6e000000.gpu: error powering up gpu

Which appears to stem from the poking of STACK_PWRON_LO. Judging by CONFIG_MALI_CORESTACK in kbase, maybe we shouldn't always be going there anyway (Steve probably knows more, but is away for a few weeks ATM).

I can't make much sense of the IGT output, but trying "scripts/run-tests.sh -t panfrost" (because I also don't have the patience to watch it skip through all ~63,000 tests...) seems pretty much consistent with or without this patch. Same for trying kmscube with mesa/master; that produces lots of job errors:

[   42.409568] panfrost 6e000000.gpu: js fault, js=0, status=DATA_INVALID_FAULT, head=0x2400b00, tail=0x2400b00
[   42.419380] panfrost 6e000000.gpu: gpu sched timeout, js=0, status=0x58, head=0x2400b00, tail=0x2400b00, sched_job=00000000d17b91

rather than MMU faults either way, so at least this change doesn't seem to present any significant regression. It looks like without this patch we end up using AS_TRANSCFG_ADRMODE_LEGACY, which presumably means exactly what it sounds like, although whether that's sufficiently reliable I don't know; those kinds of backwards-compatibility features do have a habit of eventually disappearing, and what I've tried so far is certainly not the latest and greatest.

Robin.

Quoted 115 lines

[0] https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/tests/panfrost_submit.c

Thanks,

Tomeu

Quoted 108 lines · Rob Herring

Bifrost GPUs use the standard format stage 1 LPAE page tables matching the io-pgtable ARM_64_LPAE_S1 format. The one difference is the TCR or TRANSCFG register as the Mali driver calls it has its own custom layout.

---
This and compatible strings should be enough for enabling bifrost. There's some other feature and issue differences, but I think they all either don't matter (because of differences in panfrost) or I've already handled them.

This is only compile tested as I don't have h/w. Running the existing IGT tests may be sufficient to test this. We should get an error with the command stream rather than a MMU fault if this is working correctly.

Rob

  drivers/gpu/drm/panfrost/panfrost_mmu.c  | 28 +++++++++++++++++++++++-
  drivers/gpu/drm/panfrost/panfrost_regs.h |  3 +++
  2 files changed, 30 insertions(+), 1 deletion(-)
drivers/gpu/drm/panfrost/panfrost_mmu.c +27 -1
index 762b1bd2a8c2..41d184fab07f 100644
@@ -1,5 +1,6 @@
  // SPDX-License-Identifier:    GPL-2.0
  /* Copyright 2019 Linaro, Ltd, Rob Herring <[email protected]> */
+/* Copyright (C) 2019 Arm Ltd. */
  #include <linux/bitfield.h>
  #include <linux/delay.h>
  #include <linux/interrupt.h>
@@ -111,6 +112,14 @@ void panfrost_mmu_enable(struct panfrost_device *pfdev, u32 as_nr)
         u64 transtab = cfg->arm_mali_lpae_cfg.transtab;
         u64 memattr = cfg->arm_mali_lpae_cfg.memattr;
 
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+               transtab = cfg->arm_lpae_s1_cfg.ttbr[0];
+               memattr = cfg->arm_lpae_s1_cfg.mair[0];
+       } else {
+               transtab = cfg->arm_mali_lpae_cfg.transtab;
+               memattr = cfg->arm_mali_lpae_cfg.memattr;
+       }
+
         mmu_write(pfdev, MMU_INT_CLEAR, ~0);
         mmu_write(pfdev, MMU_INT_MASK, ~0);
 
@@ -123,6 +132,14 @@ void panfrost_mmu_enable(struct panfrost_device *pfdev, u32 as_nr)
         mmu_write(pfdev, AS_MEMATTR_LO(as_nr), memattr & 0xffffffffUL);
         mmu_write(pfdev, AS_MEMATTR_HI(as_nr), memattr >> 32);
 
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+               /* TODO: handle system coherency */
+               mmu_write(pfdev, AS_TRANSCFG_LO(as_nr),
+                       AS_TRANSCFG_PTW_MEMATTR_WRITE_BACK |
+                       AS_TRANSCFG_ADRMODE_AARCH64_4K);
+               mmu_write(pfdev, AS_TRANSCFG_HI(as_nr), 0);
+       }
+
         write_cmd(pfdev, as_nr, AS_COMMAND_UPDATE);
  }
 
@@ -134,6 +151,11 @@ static void mmu_disable(struct panfrost_device *pfdev, u32 as_nr)
         mmu_write(pfdev, AS_MEMATTR_LO(as_nr), 0);
         mmu_write(pfdev, AS_MEMATTR_HI(as_nr), 0);
 
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+               mmu_write(pfdev, AS_TRANSCFG_LO(as_nr), 0);
+               mmu_write(pfdev, AS_TRANSCFG_HI(as_nr), 0);
+       }
+
         write_cmd(pfdev, as_nr, AS_COMMAND_UPDATE);
  }
 
@@ -335,6 +357,7 @@ static irqreturn_t panfrost_mmu_irq_handler(int irq, void *data)
  int panfrost_mmu_init(struct panfrost_device *pfdev)
  {
         struct io_pgtable_ops *pgtbl_ops;
+       enum io_pgtable_fmt pgt_fmt = ARM_MALI_LPAE;
         int err, irq;
 
         pfdev->mmu = devm_kzalloc(pfdev->dev, sizeof(*pfdev->mmu), GFP_KERNEL);
@@ -365,7 +388,10 @@ int panfrost_mmu_init(struct panfrost_device *pfdev)
                 .iommu_dev      = pfdev->dev,
         };
 
-       pgtbl_ops = alloc_io_pgtable_ops(ARM_MALI_LPAE, &pfdev->mmu->pgtbl_cfg,
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU))
+               pgt_fmt = ARM_64_LPAE_S1;
+
+       pgtbl_ops = alloc_io_pgtable_ops(pgt_fmt, &pfdev->mmu->pgtbl_cfg,
                                          pfdev);
         if (!pgtbl_ops)
                 return -ENOMEM;
drivers/gpu/drm/panfrost/panfrost_regs.h +3 -0
index 578c5fc2188b..31211df83c30 100644
@@ -287,6 +287,9 @@
  #define AS_TRANSTAB_LPAE_READ_INNER            BIT(2)
  #define AS_TRANSTAB_LPAE_SHARE_OUTER           BIT(4)
 
+#define AS_TRANSCFG_ADRMODE_AARCH64_4K         6
+#define AS_TRANSCFG_PTW_MEMATTR_WRITE_BACK     (2 << 24)
+
  #define AS_STATUS_AS_ACTIVE                    0x01
 
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/panfrost/panfrost_regs.h +0 -1
  #define AS_FAULTSTATUS_ACCESS_TYPE_MASK                (0x3 << 8)
--

2.20.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#3 lore raw
Quoted 20 lines · Robin Murphy

Hi Tomeu, Rob,

Quoted 8 lines · Tomeu Vizoso

Robin, Steven,

would you or someone else at Arm be able to run the IGT tests [0] on 5.2-rc2 with this patch on top?

I don't have any hw with Bifrost and am not planning to work on the userspace any time soon, but I think it would be good to at least check that the kernel doesn't have any obvious bugs.

I've managed to cobble something together which appears to sort-of-work, although I don't have the knowledge, time or patience to dive down the rabbit-hole of getting a working Arm DDK driver to actually prove the setup. The immediate observation is that I get a lot of this:

[  305.602001] panfrost 6e000000.gpu: error powering up gpu
Quoted 8 lines

Which appears to stem from the poking of STACK_PWRON_LO. Judging by CONFIG_MALI_CORESTACK in kbase, maybe we shouldn't always be going there anyway (Steve probably knows more, but is away for a few weeks ATM).

I can't make much sense of the IGT output, but trying "scripts/run-tests.sh -t panfrost" (because I also don't have the patience to watch it skip through all ~63,000 tests...) seems pretty much consistent with or without this patch.

Oops, sorry about that. It would have been sufficient to directly run these executables:

tests/panfrost_gem_new
tests/panfrost_get_param
tests/panfrost_prime
tests/panfrost_submit

Quoted 10 lines

Same for trying kmscube with
mesa/master; that produces lots of job errors:

[   42.409568] panfrost 6e000000.gpu: js fault, js=0,
status=DATA_INVALID_FAULT, head=0x2400b00, tail=0x2400b00
[   42.419380] panfrost 6e000000.gpu: gpu sched timeout, js=0,

status=0x58, head=0x2400b00, tail=0x2400b00, sched_job=00000000d17b91

rather than MMU faults either way, so at least this change doesn't seem to present any significant regression.

That sounds pretty good to me. We know that the cmdstream and compiler aren't ready yet for Bifrost.

Quoted 6 lines

It looks like without this patch
we end up using AS_TRANSCFG_ADRMODE_LEGACY, which presumably means exactly what it sounds like, although whether that's sufficiently reliable I don't know; those kinds of backwards-compatibility features do have a habit of eventually disappearing, and what I've tried so far is certainly not the latest and greatest.

One for Rob when he's back, I think :)

Thanks a lot!

Tomeu

Quoted 117 lines

Robin.

Quoted 115 lines

[0] https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/tests/panfrost_submit.c

Thanks,

Tomeu

Quoted 108 lines · Rob Herring

Bifrost GPUs use the standard format stage 1 LPAE page tables matching the io-pgtable ARM_64_LPAE_S1 format. The one difference is the TCR or TRANSCFG register as the Mali driver calls it has its own custom layout.

---
This and compatible strings should be enough for enabling bifrost. There's some other feature and issue differences, but I think they all either don't matter (because of differences in panfrost) or I've already handled them.

This is only compile tested as I don't have h/w. Running the existing IGT tests may be sufficient to test this. We should get an error with the command stream rather than a MMU fault if this is working correctly.

Rob

  drivers/gpu/drm/panfrost/panfrost_mmu.c  | 28 +++++++++++++++++++++++-
  drivers/gpu/drm/panfrost/panfrost_regs.h |  3 +++
  2 files changed, 30 insertions(+), 1 deletion(-)
drivers/gpu/drm/panfrost/panfrost_mmu.c +27 -1
index 762b1bd2a8c2..41d184fab07f 100644
@@ -1,5 +1,6 @@
  // SPDX-License-Identifier:    GPL-2.0
  /* Copyright 2019 Linaro, Ltd, Rob Herring <[email protected]> */
+/* Copyright (C) 2019 Arm Ltd. */
  #include <linux/bitfield.h>
  #include <linux/delay.h>
  #include <linux/interrupt.h>
@@ -111,6 +112,14 @@ void panfrost_mmu_enable(struct panfrost_device *pfdev, u32 as_nr)
         u64 transtab = cfg->arm_mali_lpae_cfg.transtab;
         u64 memattr = cfg->arm_mali_lpae_cfg.memattr;
 
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+               transtab = cfg->arm_lpae_s1_cfg.ttbr[0];
+               memattr = cfg->arm_lpae_s1_cfg.mair[0];
+       } else {
+               transtab = cfg->arm_mali_lpae_cfg.transtab;
+               memattr = cfg->arm_mali_lpae_cfg.memattr;
+       }
+
         mmu_write(pfdev, MMU_INT_CLEAR, ~0);
         mmu_write(pfdev, MMU_INT_MASK, ~0);
 
@@ -123,6 +132,14 @@ void panfrost_mmu_enable(struct panfrost_device *pfdev, u32 as_nr)
         mmu_write(pfdev, AS_MEMATTR_LO(as_nr), memattr & 0xffffffffUL);
         mmu_write(pfdev, AS_MEMATTR_HI(as_nr), memattr >> 32);
 
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+               /* TODO: handle system coherency */
+               mmu_write(pfdev, AS_TRANSCFG_LO(as_nr),
+                       AS_TRANSCFG_PTW_MEMATTR_WRITE_BACK |
+                       AS_TRANSCFG_ADRMODE_AARCH64_4K);
+               mmu_write(pfdev, AS_TRANSCFG_HI(as_nr), 0);
+       }
+
         write_cmd(pfdev, as_nr, AS_COMMAND_UPDATE);
  }
 
@@ -134,6 +151,11 @@ static void mmu_disable(struct panfrost_device *pfdev, u32 as_nr)
         mmu_write(pfdev, AS_MEMATTR_LO(as_nr), 0);
         mmu_write(pfdev, AS_MEMATTR_HI(as_nr), 0);
 
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU)) {
+               mmu_write(pfdev, AS_TRANSCFG_LO(as_nr), 0);
+               mmu_write(pfdev, AS_TRANSCFG_HI(as_nr), 0);
+       }
+
         write_cmd(pfdev, as_nr, AS_COMMAND_UPDATE);
  }
 
@@ -335,6 +357,7 @@ static irqreturn_t panfrost_mmu_irq_handler(int irq, void *data)
  int panfrost_mmu_init(struct panfrost_device *pfdev)
  {
         struct io_pgtable_ops *pgtbl_ops;
+       enum io_pgtable_fmt pgt_fmt = ARM_MALI_LPAE;
         int err, irq;
 
         pfdev->mmu = devm_kzalloc(pfdev->dev, sizeof(*pfdev->mmu), GFP_KERNEL);
@@ -365,7 +388,10 @@ int panfrost_mmu_init(struct panfrost_device *pfdev)
                 .iommu_dev      = pfdev->dev,
         };
 
-       pgtbl_ops = alloc_io_pgtable_ops(ARM_MALI_LPAE, &pfdev->mmu->pgtbl_cfg,
+       if (panfrost_has_hw_feature(pfdev, HW_FEATURE_AARCH64_MMU))
+               pgt_fmt = ARM_64_LPAE_S1;
+
+       pgtbl_ops = alloc_io_pgtable_ops(pgt_fmt, &pfdev->mmu->pgtbl_cfg,
                                          pfdev);
         if (!pgtbl_ops)
                 return -ENOMEM;
drivers/gpu/drm/panfrost/panfrost_regs.h +3 -0
index 578c5fc2188b..31211df83c30 100644
@@ -287,6 +287,9 @@
  #define AS_TRANSTAB_LPAE_READ_INNER            BIT(2)
  #define AS_TRANSTAB_LPAE_SHARE_OUTER           BIT(4)
 
+#define AS_TRANSCFG_ADRMODE_AARCH64_4K         6
+#define AS_TRANSCFG_PTW_MEMATTR_WRITE_BACK     (2 << 24)
+
  #define AS_STATUS_AS_ACTIVE                    0x01
 
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/panfrost/panfrost_regs.h +0 -1
  #define AS_FAULTSTATUS_ACCESS_TYPE_MASK                (0x3 << 8)
--

2.20.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#4 lore raw
Quoted 53 lines · Tomeu Vizoso
Quoted 20 lines · Robin Murphy

Hi Tomeu, Rob,

Quoted 8 lines · Tomeu Vizoso

Robin, Steven,

would you or someone else at Arm be able to run the IGT tests [0] on 5.2-rc2 with this patch on top?

I don't have any hw with Bifrost and am not planning to work on the userspace any time soon, but I think it would be good to at least check that the kernel doesn't have any obvious bugs.

I've managed to cobble something together which appears to sort-of-work, although I don't have the knowledge, time or patience to dive down the rabbit-hole of getting a working Arm DDK driver to actually prove the setup. The immediate observation is that I get a lot of this:

[  305.602001] panfrost 6e000000.gpu: error powering up gpu
Quoted 8 lines

Which appears to stem from the poking of STACK_PWRON_LO. Judging by CONFIG_MALI_CORESTACK in kbase, maybe we shouldn't always be going there anyway (Steve probably knows more, but is away for a few weeks ATM).

I can't make much sense of the IGT output, but trying "scripts/run-tests.sh -t panfrost" (because I also don't have the patience to watch it skip through all ~63,000 tests...) seems pretty much consistent with or without this patch.

Oops, sorry about that. It would have been sufficient to directly run these executables:

tests/panfrost_gem_new
tests/panfrost_get_param
tests/panfrost_prime
tests/panfrost_submit

Quoted 10 lines

Same for trying kmscube with
mesa/master; that produces lots of job errors:

[   42.409568] panfrost 6e000000.gpu: js fault, js=0,
status=DATA_INVALID_FAULT, head=0x2400b00, tail=0x2400b00
[   42.419380] panfrost 6e000000.gpu: gpu sched timeout, js=0,

status=0x58, head=0x2400b00, tail=0x2400b00, sched_job=00000000d17b91

rather than MMU faults either way, so at least this change doesn't seem to present any significant regression.

That sounds pretty good to me. We know that the cmdstream and compiler aren't ready yet for Bifrost.

Quoted 8 lines
Quoted 6 lines

It looks like without this patch
we end up using AS_TRANSCFG_ADRMODE_LEGACY, which presumably means exactly what it sounds like, although whether that's sufficiently reliable I don't know; those kinds of backwards-compatibility features do have a habit of eventually disappearing, and what I've tried so far is certainly not the latest and greatest.

One for Rob when he's back, I think :)

I wouldn't have expected AS_TRANSCFG_ADRMODE_LEGACY to work and if it did it was by chance. So I don't think it is something we want to support.

Rob
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

maintainer
#5 lore raw

On 10/06/2019 14:20, Rob Herring wrote:
[...]

Quoted 3 lines

I wouldn't have expected AS_TRANSCFG_ADRMODE_LEGACY to work and if it did it was by chance. So I don't think it is something we want to support.

Actually legacy mode is supported on (most?) Bifrost GPUs. But best to follow the lead of kbase here as that will be what has been tested.

Steve
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel