Skip to contentSkip to conversation

Lima DRM driver

PATCH Superseded Qiang Yu 11 participants 42 messages Feb 6, 2019 – Mar 15, 2019

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Qiang Yu

author maintainer
#0 lore raw

cover Lima DRM driver

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.
- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

This patch serial is based on 5.0-rc5 and squash all the commits. For whole history of this driver's development, see:
https://gitlab.freedesktop.org/lima/linux/commits/lima-5.0-rc5
https://gitlab.freedesktop.org/lima/linux/commits/lima-4.17-rc4

Mesa driver is still in development and not ready for daily usage, but can run some simple tests like kmscube and glamrk2, and some single full screen application like kodi-gbm, see:
https://gitlab.freedesktop.org/lima/mesa

[rfc]
https://lists.freedesktop.org/archives/dri-devel/2018-May/177314.html

Lima Project Developers (1):
drm/lima: driver for ARM Mali4xx GPUs

Qiang Yu (1):
drm/fourcc: add ARM tiled format modifier

 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  22 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  46 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       | 124 +++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  33 ++
 drivers/gpu/drm/lima/lima_device.c    | 384 ++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 116 ++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  56 +++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 +
 drivers/gpu/drm/lima/lima_drv.c       | 459 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  59 ++++
 drivers/gpu/drm/lima/lima_gem.c       | 485 +++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c | 144 ++++++++
 drivers/gpu/drm/lima/lima_gem_prime.h |  18 +
 drivers/gpu/drm/lima/lima_gp.c        | 280 +++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  79 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 135 +++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 103 ++++++
 drivers/gpu/drm/lima/lima_object.h    |  72 ++++
 drivers/gpu/drm/lima/lima_pmu.c       |  61 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 419 ++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_pp.h        |  19 +
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 486 ++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 108 ++++++
 drivers/gpu/drm/lima/lima_ttm.c       | 319 +++++++++++++++++
 drivers/gpu/drm/lima/lima_ttm.h       |  24 ++
 drivers/gpu/drm/lima/lima_vm.c        | 354 +++++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  59 ++++
 include/uapi/drm/drm_fourcc.h         |   9 +
 include/uapi/drm/lima_drm.h           | 193 ++++++++++
 39 files changed, 5092 insertions(+)
 create mode 100644 drivers/gpu/drm/lima/Kconfig
 create mode 100644 drivers/gpu/drm/lima/Makefile
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.c
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.h
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.c
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.h
 create mode 100644 drivers/gpu/drm/lima/lima_device.c
 create mode 100644 drivers/gpu/drm/lima/lima_device.h
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h
 create mode 100644 drivers/gpu/drm/lima/lima_drv.c
 create mode 100644 drivers/gpu/drm/lima/lima_drv.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h
 create mode 100644 drivers/gpu/drm/lima/lima_gp.c
 create mode 100644 drivers/gpu/drm/lima/lima_gp.h
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_object.c
 create mode 100644 drivers/gpu/drm/lima/lima_object.h
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_pp.c
 create mode 100644 drivers/gpu/drm/lima/lima_pp.h
 create mode 100644 drivers/gpu/drm/lima/lima_regs.h
 create mode 100644 drivers/gpu/drm/lima/lima_sched.c
 create mode 100644 drivers/gpu/drm/lima/lima_sched.h
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.c
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.h
 create mode 100644 drivers/gpu/drm/lima/lima_vm.c
 create mode 100644 drivers/gpu/drm/lima/lima_vm.h
 create mode 100644 include/uapi/drm/lima_drm.h
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#1 lore raw

PATCH 1/2 drm/fourcc: add ARM tiled format modifier

---
 include/uapi/drm/drm_fourcc.h | 9 +++++++++
 1 file changed, 9 insertions(+)
include/uapi/drm/drm_fourcc.h +9 -0
index 0b44260a5ee9..953b59eb3fd2 100644
@@ -644,6 +644,15 @@ extern "C" {
  */
 #define AFBC_FORMAT_MOD_SC      (1ULL <<  9)
 
+/*
+ * ARM tiled format
+ *
+ * This is used by ARM Mali Utgard/Midgard GPU. It divides buffer into
+ * 16x16 pixel blocks. Blocks are stored linearly in order, but pixels
+ * in the block are reordered.
+ */
+#define DRM_FORMAT_MOD_ARM_TILED fourcc_mod_code(ARM, 1)
+
 #if defined(__cplusplus)
 }
 #endif
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#2 lore raw

Re: [PATCH 1/2] drm/fourcc: add ARM tiled format modifier

Hi,

Quoted 12 lines · Qiang Yu
---
 include/uapi/drm/drm_fourcc.h | 9 +++++++++
 1 file changed, 9 insertions(+)
include/uapi/drm/drm_fourcc.h +0 -0
index 0b44260a5ee9..953b59eb3fd2 100644
@@ -644,6 +644,15 @@ extern "C" {
  */
 #define AFBC_FORMAT_MOD_SC      (1ULL <<  9)

include/uapi/drm/drm_fourcc.h extern "C" { · line 655

Quoted 10 lines
include/uapi/drm/drm_fourcc.h +9 -0
 
+/*
+ * ARM tiled format
+ *
+ * This is used by ARM Mali Utgard/Midgard GPU. It divides buffer into
+ * 16x16 pixel blocks. Blocks are stored linearly in order, but pixels
+ * in the block are reordered.
+ */
+#define DRM_FORMAT_MOD_ARM_TILED fourcc_mod_code(ARM, 1)
+

This conflicts with the already-defined AFBC modifiers. It has the same value as:

DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_BLOCK_SIZE_16x16);

I would suggest we use the top bit(s) to namespace between the different types of tiling formats Arm has (e.g. AFBC vs non-compressed).

I think this would be an internal-only layout though. Do we need modifiers for internal-only formats? I thought they were mainly for sharing cross-device. I also didn't see this macro getting used anywhere in the driver; I suppose at a minimum you should validate the modifier value provided by userspace.

Thanks,
-Brian

Quoted 10 lines
include/uapi/drm/drm_fourcc.h +0 -0
 #if defined(__cplusplus)
 }
 #endif
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#3 lore raw

Re: [PATCH 1/2] drm/fourcc: add ARM tiled format modifier

Quoted 35 lines · Brian Starkey

Hi,

Quoted 12 lines · Qiang Yu
---
 include/uapi/drm/drm_fourcc.h | 9 +++++++++
 1 file changed, 9 insertions(+)
include/uapi/drm/drm_fourcc.h +0 -0
index 0b44260a5ee9..953b59eb3fd2 100644
@@ -644,6 +644,15 @@ extern "C" {
  */
 #define AFBC_FORMAT_MOD_SC      (1ULL <<  9)

include/uapi/drm/drm_fourcc.h extern "C" { · line 655

Quoted 10 lines
include/uapi/drm/drm_fourcc.h +9 -0
 
+/*
+ * ARM tiled format
+ *
+ * This is used by ARM Mali Utgard/Midgard GPU. It divides buffer into
+ * 16x16 pixel blocks. Blocks are stored linearly in order, but pixels
+ * in the block are reordered.
+ */
+#define DRM_FORMAT_MOD_ARM_TILED fourcc_mod_code(ARM, 1)
+

This conflicts with the already-defined AFBC modifiers. It has the same value as:

DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_BLOCK_SIZE_16x16);

I would suggest we use the top bit(s) to namespace between the different types of tiling formats Arm has (e.g. AFBC vs non-compressed).

Quoted 5 lines

I think this would be an internal-only layout though. Do we need modifiers for internal-only formats? I thought they were mainly for sharing cross-device. I also didn't see this macro getting used anywhere in the driver; I suppose at a minimum you should validate the modifier value provided by userspace.

Modifiers are also used in dri2/3 and wayland protocols, plus for buffer sharing on the gl side. And for render drivers it's pretty common to not validate at all what userspace is doing with it's buffers, as long as you can make sure it can't escape it's sandbox (iommu pagetables generally considered good enough, better if per-process).
-Daniel

Quoted 18 lines

Thanks,
-Brian

Quoted 10 lines
include/uapi/drm/drm_fourcc.h +0 -0
 #if defined(__cplusplus)
 }
 #endif
Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#4 lore raw

Re: [PATCH 1/2] drm/fourcc: add ARM tiled format modifier

In reply to Brian Starkey
Quoted 30 lines · Brian Starkey

Hi,

Quoted 12 lines · Qiang Yu
---
 include/uapi/drm/drm_fourcc.h | 9 +++++++++
 1 file changed, 9 insertions(+)
include/uapi/drm/drm_fourcc.h +0 -0
index 0b44260a5ee9..953b59eb3fd2 100644
@@ -644,6 +644,15 @@ extern "C" {
  */
 #define AFBC_FORMAT_MOD_SC      (1ULL <<  9)

include/uapi/drm/drm_fourcc.h extern "C" { · line 655

Quoted 10 lines
include/uapi/drm/drm_fourcc.h +9 -0
 
+/*
+ * ARM tiled format
+ *
+ * This is used by ARM Mali Utgard/Midgard GPU. It divides buffer into
+ * 16x16 pixel blocks. Blocks are stored linearly in order, but pixels
+ * in the block are reordered.
+ */
+#define DRM_FORMAT_MOD_ARM_TILED fourcc_mod_code(ARM, 1)
+

This conflicts with the already-defined AFBC modifiers. It has the same value as:

Quoted 5 lines

DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_BLOCK_SIZE_16x16);

I would suggest we use the top bit(s) to namespace between the different types of tiling formats Arm has (e.g. AFBC vs non-compressed).

Just for conformation, is the non-compressed tile format + BLOCK_SIZE_16x16 super block format same as mali gpu used tile format?

So what name would you suggest to name this mali gpu tile format? Or maybe it's better that you can send out a patch to add the tile format (better also other ones) and how it's assembled with super block format in the modifier, as I don't have a whole picture of this.

Quoted 7 lines

I think this would be an internal-only layout though. Do we need modifiers for internal-only formats? I thought they were mainly for sharing cross-device. I also didn't see this macro getting used anywhere in the driver; I suppose at a minimum you should validate the modifier value provided by userspace.

Yes we need it. It's used in the userspace mesa driver and applications which need to share mali driver allocated buffers across process. So that one process can tell another process which format to interpret the passed buffer.

Thanks,
Qiang
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#5 lore raw

Re: [PATCH 1/2] drm/fourcc: add ARM tiled format modifier

Quoted 36 lines · Qiang Yu
Quoted 30 lines · Brian Starkey

Hi,

Quoted 12 lines · Qiang Yu
---
 include/uapi/drm/drm_fourcc.h | 9 +++++++++
 1 file changed, 9 insertions(+)
include/uapi/drm/drm_fourcc.h +0 -0
index 0b44260a5ee9..953b59eb3fd2 100644
@@ -644,6 +644,15 @@ extern "C" {
  */
 #define AFBC_FORMAT_MOD_SC      (1ULL <<  9)

include/uapi/drm/drm_fourcc.h extern "C" { · line 655

Quoted 10 lines
include/uapi/drm/drm_fourcc.h +9 -0
 
+/*
+ * ARM tiled format
+ *
+ * This is used by ARM Mali Utgard/Midgard GPU. It divides buffer into
+ * 16x16 pixel blocks. Blocks are stored linearly in order, but pixels
+ * in the block are reordered.
+ */
+#define DRM_FORMAT_MOD_ARM_TILED fourcc_mod_code(ARM, 1)
+

This conflicts with the already-defined AFBC modifiers. It has the same value as:

Quoted 5 lines

DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_BLOCK_SIZE_16x16);

I would suggest we use the top bit(s) to namespace between the different types of tiling formats Arm has (e.g. AFBC vs non-compressed).

Quoted 7 lines

Just for conformation, is the non-compressed tile format + BLOCK_SIZE_16x16 super block format same as mali gpu used tile format?

So what name would you suggest to name this mali gpu tile format? Or maybe it's better that you can send out a patch to add the tile format (better also other ones) and how it's assembled with super block format in the modifier, as I don't have a whole picture of this.

Honestly, you know more about it than I do (ironic, I know). I'm not working on Mali GPUs and I've never run anything on Utgard. All I really know is that what you're describing here definitely _isn't_ AFBC.

I've asked internally, so if I have anything useful to add, I will update you. If we can get a patch from Arm, I agree that would be a good solution.

My (personal) main concern here is only that you don't conflict with AFBC.

Quoted 8 lines
Quoted 8 lines

I think this would be an internal-only layout though. Do we need modifiers for internal-only formats? I thought they were mainly for sharing cross-device. I also didn't see this macro getting used anywhere in the driver; I suppose at a minimum you should validate the modifier value provided by userspace.

Quoted 4 lines

Yes we need it. It's used in the userspace mesa driver and applications which need to share mali driver allocated buffers across process. So that one process can tell another process which format to interpret the passed buffer.

OK, thanks to both of you for educating me :-)

Cheers,
-Brian

Quoted 3 lines

Thanks,
Qiang

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#6 lore raw

PATCH 2/2 drm/lima: driver for ARM Mali4xx GPUs

In reply to Qiang Yu

From: Lima Project Developers <[email protected]>

---
 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  22 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  46 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       | 124 +++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  33 ++
 drivers/gpu/drm/lima/lima_device.c    | 384 ++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 116 ++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  56 +++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 +
 drivers/gpu/drm/lima/lima_drv.c       | 459 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  59 ++++
 drivers/gpu/drm/lima/lima_gem.c       | 485 +++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c | 144 ++++++++
 drivers/gpu/drm/lima/lima_gem_prime.h |  18 +
 drivers/gpu/drm/lima/lima_gp.c        | 280 +++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  79 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 135 +++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 103 ++++++
 drivers/gpu/drm/lima/lima_object.h    |  72 ++++
 drivers/gpu/drm/lima/lima_pmu.c       |  61 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 419 ++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_pp.h        |  19 +
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 486 ++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 108 ++++++
 drivers/gpu/drm/lima/lima_ttm.c       | 319 +++++++++++++++++
 drivers/gpu/drm/lima/lima_ttm.h       |  24 ++
 drivers/gpu/drm/lima/lima_vm.c        | 354 +++++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  59 ++++
 include/uapi/drm/lima_drm.h           | 193 ++++++++++
 38 files changed, 5083 insertions(+)
 create mode 100644 drivers/gpu/drm/lima/Kconfig
 create mode 100644 drivers/gpu/drm/lima/Makefile
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.c
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.h
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.c
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.h
 create mode 100644 drivers/gpu/drm/lima/lima_device.c
 create mode 100644 drivers/gpu/drm/lima/lima_device.h
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h
 create mode 100644 drivers/gpu/drm/lima/lima_drv.c
 create mode 100644 drivers/gpu/drm/lima/lima_drv.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h
 create mode 100644 drivers/gpu/drm/lima/lima_gp.c
 create mode 100644 drivers/gpu/drm/lima/lima_gp.h
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_object.c
 create mode 100644 drivers/gpu/drm/lima/lima_object.h
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_pp.c
 create mode 100644 drivers/gpu/drm/lima/lima_pp.h
 create mode 100644 drivers/gpu/drm/lima/lima_regs.h
 create mode 100644 drivers/gpu/drm/lima/lima_sched.c
 create mode 100644 drivers/gpu/drm/lima/lima_sched.h
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.c
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.h
 create mode 100644 drivers/gpu/drm/lima/lima_vm.c
 create mode 100644 drivers/gpu/drm/lima/lima_vm.h
 create mode 100644 include/uapi/drm/lima_drm.h
drivers/gpu/drm/Kconfig +2 -0
index 4385f00e1d05..dfefcb393858 100644
@@ -333,6 +333,8 @@ source "drivers/gpu/drm/tve200/Kconfig"
 
 source "drivers/gpu/drm/xen/Kconfig"
 
+source "drivers/gpu/drm/lima/Kconfig"
+
 # Keep legacy drivers last
 
 menuconfig DRM_LEGACY
drivers/gpu/drm/Makefile +1 -0
index ce8d1d384319..8d024b729902 100644
@@ -109,3 +109,4 @@ obj-$(CONFIG_DRM_TINYDRM) += tinydrm/
 obj-$(CONFIG_DRM_PL111) += pl111/
 obj-$(CONFIG_DRM_TVE200) += tve200/
 obj-$(CONFIG_DRM_XEN) += xen/
+obj-$(CONFIG_DRM_LIMA)  += lima/
drivers/gpu/drm/lima/Kconfig new file +10 -0
new file mode 100644
index 000000000000..89d63cca8a75
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0 OR MIT
+# Copyright 2017-2018 Qiang Yu <[email protected]>
+
+config DRM_LIMA
+       tristate "LIMA (DRM support for ARM Mali 400/450 GPU)"
+       depends on DRM
+       select DRM_SCHED
+       select DRM_TTM
+       help
+         DRM driver for ARM Mali 400/450 GPUs.
drivers/gpu/drm/lima/Makefile new file +22 -0
new file mode 100644
index 000000000000..8bb97410d961
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0 OR MIT
+# Copyright 2017-2018 Qiang Yu <[email protected]>
+
+lima-y := \
+	lima_drv.o \
+	lima_device.o \
+	lima_pmu.o \
+	lima_l2_cache.o \
+	lima_mmu.o \
+	lima_gp.o \
+	lima_pp.o \
+	lima_gem.o \
+	lima_vm.o \
+	lima_sched.o \
+	lima_ctx.o \
+	lima_gem_prime.o \
+	lima_dlbu.o \
+	lima_bcast.o \
+	lima_ttm.o \
+	lima_object.o
+
+obj-$(CONFIG_DRM_LIMA) += lima.o
drivers/gpu/drm/lima/lima_bcast.c new file +46 -0

46 lines not on this page · raw message

drivers/gpu/drm/lima/lima_bcast.h new file +14 -0
new file mode 100644
index 000000000000..345e3e809860
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_BCAST_H__
+#define __LIMA_BCAST_H__
+
+struct lima_ip;
+
+int lima_bcast_init(struct lima_ip *ip);
+void lima_bcast_fini(struct lima_ip *ip);
+
+void lima_bcast_enable(struct lima_device *dev, int num_pp);
+
+#endif
drivers/gpu/drm/lima/lima_ctx.c new file +124 -0

124 lines not on this page · raw message

drivers/gpu/drm/lima/lima_ctx.h new file +33 -0
new file mode 100644
index 000000000000..80e55e16619f
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_CTX_H__
+#define __LIMA_CTX_H__
+
+#include <linux/idr.h>
+
+#include "lima_device.h"
+
+struct lima_ctx {
+	struct kref refcnt;
+	struct lima_device *dev;
+	struct lima_sched_context context[lima_pipe_num];
+	atomic_t guilty;
+};
+
+struct lima_ctx_mgr {
+	spinlock_t lock;
+	struct idr handles;
+};
+
+int lima_ctx_create(struct lima_device *dev, struct lima_ctx_mgr *mgr, u32 *id);
+int lima_ctx_free(struct lima_ctx_mgr *mgr, u32 id);
+struct lima_ctx *lima_ctx_get(struct lima_ctx_mgr *mgr, u32 id);
+void lima_ctx_put(struct lima_ctx *ctx);
+void lima_ctx_mgr_init(struct lima_ctx_mgr *mgr);
+void lima_ctx_mgr_fini(struct lima_ctx_mgr *mgr);
+
+struct dma_fence *lima_ctx_get_native_fence(struct lima_ctx_mgr *mgr,
+					    u32 ctx, u32 pipe, u32 seq);
+
+#endif
drivers/gpu/drm/lima/lima_device.c new file +384 -0

384 lines not on this page · raw message

drivers/gpu/drm/lima/lima_device.h new file +116 -0

116 lines not on this page · raw message

drivers/gpu/drm/lima/lima_dlbu.c new file +56 -0

56 lines not on this page · raw message

drivers/gpu/drm/lima/lima_dlbu.h new file +18 -0
new file mode 100644
index 000000000000..60cba387cf30
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_DLBU_H__
+#define __LIMA_DLBU_H__
+
+struct lima_ip;
+struct lima_device;
+
+void lima_dlbu_enable(struct lima_device *dev, int num_pp);
+void lima_dlbu_disable(struct lima_device *dev);
+
+void lima_dlbu_set_reg(struct lima_ip *ip, u32 *reg);
+
+int lima_dlbu_init(struct lima_ip *ip);
+void lima_dlbu_fini(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_drv.c new file +459 -0

459 lines not on this page · raw message

drivers/gpu/drm/lima/lima_drv.h new file +59 -0

59 lines not on this page · raw message

drivers/gpu/drm/lima/lima_gem.c new file +485 -0

485 lines not on this page · raw message

drivers/gpu/drm/lima/lima_gem.h new file +25 -0
new file mode 100644
index 000000000000..da6968fab6bb
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_GEM_H__
+#define __LIMA_GEM_H__
+
+struct lima_bo;
+struct lima_submit;
+
+struct lima_bo *lima_gem_create_bo(struct drm_device *dev, u32 size, u32 flags);
+int lima_gem_create_handle(struct drm_device *dev, struct drm_file *file,
+			   u32 size, u32 flags, u32 *handle);
+void lima_gem_free_object(struct drm_gem_object *obj);
+int lima_gem_object_open(struct drm_gem_object *obj, struct drm_file *file);
+void lima_gem_object_close(struct drm_gem_object *obj, struct drm_file *file);
+int lima_gem_mmap_offset(struct drm_file *file, u32 handle, u64 *offset);
+int lima_gem_mmap(struct file *filp, struct vm_area_struct *vma);
+int lima_gem_va_map(struct drm_file *file, u32 handle, u32 flags, u32 va);
+int lima_gem_va_unmap(struct drm_file *file, u32 handle, u32 va);
+int lima_gem_submit(struct drm_file *file, struct lima_submit *submit);
+int lima_gem_wait(struct drm_file *file, u32 handle, u32 op, u64 timeout_ns);
+int lima_gem_get_modifier(struct drm_file *file, u32 handle, u64 *modifier);
+int lima_gem_set_modifier(struct drm_file *file, u32 handle, u64 modifier);
+
+#endif
drivers/gpu/drm/lima/lima_gem_prime.c new file +144 -0

144 lines not on this page · raw message

drivers/gpu/drm/lima/lima_gem_prime.h new file +18 -0
new file mode 100644
index 000000000000..05ddb59b237a
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_GEM_PRIME_H__
+#define __LIMA_GEM_PRIME_H__
+
+struct drm_gem_object *lima_gem_prime_import_sg_table(
+	struct drm_device *dev, struct dma_buf_attachment *attach,
+	struct sg_table *sgt);
+struct sg_table *lima_gem_prime_get_sg_table(struct drm_gem_object *obj);
+struct reservation_object *lima_gem_prime_res_obj(struct drm_gem_object *obj);
+void *lima_gem_prime_vmap(struct drm_gem_object *obj);
+void lima_gem_prime_vunmap(struct drm_gem_object *obj, void *vaddr);
+int lima_gem_prime_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma);
+
+int lima_gem_prime_dma_buf_mmap(struct file *filp, struct vm_area_struct *vma);
+
+#endif
drivers/gpu/drm/lima/lima_gp.c new file +280 -0

280 lines not on this page · raw message

drivers/gpu/drm/lima/lima_gp.h new file +16 -0
new file mode 100644
index 000000000000..55bc48ec7603
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_GP_H__
+#define __LIMA_GP_H__
+
+struct lima_ip;
+struct lima_device;
+
+int lima_gp_init(struct lima_ip *ip);
+void lima_gp_fini(struct lima_ip *ip);
+
+int lima_gp_pipe_init(struct lima_device *dev);
+void lima_gp_pipe_fini(struct lima_device *dev);
+
+#endif
drivers/gpu/drm/lima/lima_l2_cache.c new file +79 -0

79 lines not on this page · raw message

drivers/gpu/drm/lima/lima_l2_cache.h new file +14 -0
new file mode 100644
index 000000000000..2ff91eafefbe
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_L2_CACHE_H__
+#define __LIMA_L2_CACHE_H__
+
+struct lima_ip;
+
+int lima_l2_cache_init(struct lima_ip *ip);
+void lima_l2_cache_fini(struct lima_ip *ip);
+
+int lima_l2_cache_flush(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_mmu.c new file +135 -0

135 lines not on this page · raw message

drivers/gpu/drm/lima/lima_mmu.h new file +16 -0
new file mode 100644
index 000000000000..ca173b60fc73
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_MMU_H__
+#define __LIMA_MMU_H__
+
+struct lima_ip;
+struct lima_vm;
+
+int lima_mmu_init(struct lima_ip *ip);
+void lima_mmu_fini(struct lima_ip *ip);
+
+void lima_mmu_switch_vm(struct lima_ip *ip, struct lima_vm *vm);
+void lima_mmu_page_fault_resume(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_object.c new file +103 -0

103 lines not on this page · raw message

drivers/gpu/drm/lima/lima_object.h new file +72 -0

72 lines not on this page · raw message

drivers/gpu/drm/lima/lima_pmu.c new file +61 -0

61 lines not on this page · raw message

drivers/gpu/drm/lima/lima_pmu.h new file +12 -0
new file mode 100644
index 000000000000..1cf94a35bdf9
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_PMU_H__
+#define __LIMA_PMU_H__
+
+struct lima_ip;
+
+int lima_pmu_init(struct lima_ip *ip);
+void lima_pmu_fini(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_pp.c new file +419 -0

187 lines not on this page · raw message

drivers/gpu/drm/lima/lima_pp.h new file +19 -0
new file mode 100644
index 000000000000..f83f8cb4d30a

Lines of this file are not shown. View the raw message.

drivers/gpu/drm/lima/lima_regs.h new file +298 -0

298 lines not on this page · raw message

drivers/gpu/drm/lima/lima_sched.c new file +486 -0

486 lines not on this page · raw message

drivers/gpu/drm/lima/lima_sched.h new file +108 -0

108 lines not on this page · raw message

drivers/gpu/drm/lima/lima_ttm.c new file +319 -0

319 lines not on this page · raw message

drivers/gpu/drm/lima/lima_ttm.h new file +24 -0
new file mode 100644
index 000000000000..c2759a43f06b

Lines of this file are not shown. View the raw message.

drivers/gpu/drm/lima/lima_vm.c new file +354 -0

354 lines not on this page · raw message

drivers/gpu/drm/lima/lima_vm.h new file +59 -0

59 lines not on this page · raw message

include/uapi/drm/lima_drm.h new file +193 -0

193 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Part of the diff is not shown here. View the raw message.

#7 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 11 lines · Qiang Yu

From: Lima Project Developers <[email protected]>

---

Some comments to follow. Of them, the integer overflow and flags checks definitely need fixing, I strongly recommend changing your timeout handling, and would not block on any of my other suggestions.

Quoted 28 lines
drivers/gpu/drm/lima/lima_ctx.c new file +22 -0
new file mode 100644
index 000000000000..724ac4051f7a
@@ -0,0 +1,124 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#include <linux/slab.h>
+
+#include "lima_device.h"
+#include "lima_ctx.h"
+
+int lima_ctx_create(struct lima_device *dev, struct lima_ctx_mgr *mgr, u32 *id)
+{
+	struct lima_ctx *ctx;
+	int i, err;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+	ctx->dev = dev;
+	kref_init(&ctx->refcnt);
+
+	for (i = 0; i < lima_pipe_num; i++) {
+		err = lima_sched_context_init(dev->pipe + i, ctx->context + i, &ctx->guilty);
+		if (err)

drivers/gpu/drm/lima/lima_ctx.c line 32

Quoted 10 lines
drivers/gpu/drm/lima/lima_ctx.c new file +10 -0
+			goto err_out0;
+	}
+
+	idr_preload(GFP_KERNEL);
+	spin_lock(&mgr->lock);
+	err = idr_alloc(&mgr->handles, ctx, 1, 0, GFP_ATOMIC);
+	spin_unlock(&mgr->lock);
+	idr_preload_end();
+	if (err < 0)
+		goto err_out0;

You might enjoy using the new xa_alloc() api instead of idrs.

Quoted 17 lines
drivers/gpu/drm/lima/lima_ctx.c new file +17 -0
+static int lima_ioctl_gem_submit(struct drm_device *dev, void *data, struct drm_file *file)
+{
+	struct drm_lima_gem_submit_in *args = data;
+	struct lima_device *ldev = to_lima_dev(dev);
+	struct lima_drm_priv *priv = file->driver_priv;
+	struct drm_lima_gem_submit_bo *bos;
+	struct ttm_validate_buffer *vbs;
+	union drm_lima_gem_submit_dep *deps = NULL;
+	struct lima_sched_pipe *pipe;
+	struct lima_sched_task *task;
+	struct lima_ctx *ctx;
+	struct lima_submit submit = {0};
+	int err = 0, size;
+
+	if (args->pipe >= lima_pipe_num || args->nr_bos == 0)
+		return -EINVAL;
+

drivers/gpu/drm/lima/lima_ctx.c line 59

Quoted 10 lines
drivers/gpu/drm/lima/lima_ctx.c new file +10 -0
+	if (args->flags & ~(LIMA_SUBMIT_FLAG_EXPLICIT_FENCE |
+			    LIMA_SUBMIT_FLAG_SYNC_FD_OUT))
+		return -EINVAL;
+
+	pipe = ldev->pipe + args->pipe;
+	if (args->frame_size != pipe->frame_size)
+		return -EINVAL;
+
+	size = args->nr_bos * (sizeof(*submit.bos) + sizeof(*submit.vbs)) +
+		args->nr_deps * sizeof(*submit.deps);

Needs checking for integer overflow with user-submitted args here.

(Having done overflow math for the equivalent in vc4, I'd say: don't bother, do more kmallocs.)

Quoted 18 lines
drivers/gpu/drm/lima/lima_ctx.c new file +18 -0
+	bos = kzalloc(size, GFP_KERNEL);
+	if (!bos)
+		return -ENOMEM;
+
+	size = args->nr_bos * sizeof(*submit.bos);
+	if (copy_from_user(bos, u64_to_user_ptr(args->bos), size)) {
+		err = -EFAULT;
+		goto out0;
+	}
+
+	vbs = (void *)bos + size;
+
+	if (args->nr_deps) {
+		deps = (void *)vbs + args->nr_bos * sizeof(*submit.vbs);
+		size = args->nr_deps * sizeof(*submit.deps);
+		if (copy_from_user(deps, u64_to_user_ptr(args->deps), size)) {
+			err = -EFAULT;
+			goto out0;

drivers/gpu/drm/lima/lima_ctx.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_ctx.c new file +8 -0
+		}
+	}
 
 
+static int lima_ioctl_gem_wait(struct drm_device *dev, void *data, struct drm_file *file)
+{
+	struct drm_lima_gem_wait *args = data;
+
+	if (!(args->op & (LIMA_GEM_WAIT_READ|LIMA_GEM_WAIT_WRITE)))
+	    return -EINVAL;

I think you want if (args->op & ~(LIMA_GEM_WAIT_READ|LIMA_GEM_WAIT_WRITE)) so that you can be sure userspace is doing the right thing today so that you can extend with other flags in the future.

drivers/gpu/drm/lima/lima_ctx.c

Quoted 13 lines
drivers/gpu/drm/lima/lima_ctx.c new file +13 -0
+static int lima_ioctl_ctx(struct drm_device *dev, void *data, struct drm_file *file)
+{
+	struct drm_lima_ctx *args = data;
+	struct lima_drm_priv *priv = file->driver_priv;
+	struct lima_device *ldev = to_lima_dev(dev);
+
+	if (args->op == LIMA_CTX_OP_CREATE)
+		return lima_ctx_create(ldev, &priv->ctx_mgr, &args->id);
+	else if (args->op == LIMA_CTX_OP_FREE)
+		return lima_ctx_free(&priv->ctx_mgr, args->id);
+
+	return -EINVAL;
+}

Overall UAPI suggestion: Having these muxing ioctls means that your debug logs are harder to parse. ioctls already are already a mux based on the command number, so just have separate ctx_create/ctx_free, gem_get/gem_set ioctls, etc.

Quoted 7 lines
drivers/gpu/drm/lima/lima_ctx.c new file +7 -0
+static int lima_drm_driver_open(struct drm_device *dev, struct drm_file *file)
+{
+	int err;
+	struct lima_drm_priv *priv;
+	struct lima_device *ldev = to_lima_dev(dev);
+
+	priv = kzalloc(sizeof(*priv), GFP_KERNEL);

drivers/gpu/drm/lima/lima_ctx.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_ctx.c new file +10 -0
+	if (!priv)
+		return -ENOMEM;
+
+	priv->vm = lima_vm_create(ldev);
+	if (!priv->vm) {
+		err = -ENOMEM;
+		goto err_out0;
+	}
+
+        lima_ctx_mgr_init(&priv->ctx_mgr);

whitespace issue

Quoted 24 lines
drivers/gpu/drm/lima/lima_ctx.c new file +24 -0
+int lima_gem_wait(struct drm_file *file, u32 handle, u32 op, u64 timeout_ns)
+{
+	bool write = op & LIMA_GEM_WAIT_WRITE;
+	struct drm_gem_object *obj;
+	struct lima_bo *bo;
+	signed long ret;
+	unsigned long timeout;
+
+	obj = drm_gem_object_lookup(file, handle);
+	if (!obj)
+		return -ENOENT;
+
+	bo = to_lima_bo(obj);
+
+	timeout = timeout_ns ? lima_timeout_to_jiffies(timeout_ns) : 0;
+
+	ret = lima_bo_reserve(bo, true);
+	if (ret)
+		goto out;
+
+	/* must use long for result check because in 64bit arch int
+	 * will overflow if timeout is too large and get <0 result
+	 */
+	ret = reservation_object_wait_timeout_rcu(bo->tbo.resv, write, true, timeout);

drivers/gpu/drm/lima/lima_ctx.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_ctx.c new file +10 -0
+	if (ret == 0)
+		ret = timeout ? -ETIMEDOUT : -EBUSY;
+	else if (ret > 0)
+		ret = 0;
+
+	lima_bo_unreserve(bo);
+out:
+	drm_gem_object_put_unlocked(obj);
+	return ret;
+}

From Documentation/botching-up-ioctls.txt:

* For timeouts, use absolute times. If you're a good fellow and made your ioctl restartable relative timeouts tend to be too coarse and can indefinitely extend your wait time due to rounding on each restart. Especially if your reference clock is something really slow like the display frame counter. With a spec lawyer hat on this isn't a bug since timeouts can always be extended - but users will surely hate you if their neat animations starts to stutter due to this.

(I made v3d's timeouts relative, but decrement the timeout value the user passed by how much I waited so that the timeout probably gets reduced after a restartable signal. I should have done absolute.)

Quoted 17 lines
include/uapi/drm/lima_drm.h new file +11 -0
new file mode 100644
index 000000000000..c44757b4be39
 
+
+#define LIMA_SUBMIT_DEP_FENCE   0x00
+#define LIMA_SUBMIT_DEP_SYNC_FD 0x01
+
+struct drm_lima_gem_submit_dep_fence {
+	__u32 type;
+	__u32 ctx;
+	__u32 pipe;
+	__u32 seq;
+};
+

include/uapi/drm/lima_drm.h line 73

Quoted 10 lines
include/uapi/drm/lima_drm.h new file +10 -0
+struct drm_lima_gem_submit_dep_sync_fd {
+	__u32 type;
+	__u32 fd;
+};
+
+union drm_lima_gem_submit_dep {
+	__u32 type;
+	struct drm_lima_gem_submit_dep_fence fence;
+	struct drm_lima_gem_submit_dep_sync_fd sync_fd;
+};

I've been using gem sync objects for exposing my fences in v3d. You can import/export fences from sync files into syncobjs, and then you don't need a custom driver fence type in the uapi or your own ioctls for it if the submit just takes syncobjs in and out.

include/uapi/drm/lima_drm.h line 167

Quoted 8 lines
include/uapi/drm/lima_drm.h new file +8 -0
+#define LIMA_GEM_MOD_OP_GET 0
+#define LIMA_GEM_MOD_OP_SET 1
+
+struct drm_lima_gem_mod {
+	__u32 handle;      /* in */
+	__u32 op;          /* in */
+	__u64 modifier;    /* in/out */
+};

I thought the whole idea with the DRI3 modifiers stuff was that the kernel didn't need to store modifier metadata on buffers? (And this gets in the way of Vulkan modifiers support, from what I understand). Do you actually need this ABI?

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#8 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 15 lines · Eric Anholt
Quoted 11 lines · Qiang Yu

From: Lima Project Developers <[email protected]>

---

Quoted 3 lines

Some comments to follow. Of them, the integer overflow and flags checks definitely need fixing, I strongly recommend changing your timeout handling, and would not block on any of my other suggestions.

Thanks for your kind and valuable suggestion, I'll fix the args check and left some of suggestions as future improvement.

Quoted 45 lines
Quoted 24 lines
drivers/gpu/drm/lima/lima_gem.c new file +24 -0
+int lima_gem_wait(struct drm_file *file, u32 handle, u32 op, u64 timeout_ns)
+{
+     bool write = op & LIMA_GEM_WAIT_WRITE;
+     struct drm_gem_object *obj;
+     struct lima_bo *bo;
+     signed long ret;
+     unsigned long timeout;
+
+     obj = drm_gem_object_lookup(file, handle);
+     if (!obj)
+             return -ENOENT;
+
+     bo = to_lima_bo(obj);
+
+     timeout = timeout_ns ? lima_timeout_to_jiffies(timeout_ns) : 0;
+
+     ret = lima_bo_reserve(bo, true);
+     if (ret)
+             goto out;
+
+     /* must use long for result check because in 64bit arch int
+      * will overflow if timeout is too large and get <0 result
+      */
+     ret = reservation_object_wait_timeout_rcu(bo->tbo.resv, write, true, timeout);

drivers/gpu/drm/lima/lima_gem.c line 453

Quoted 10 lines
drivers/gpu/drm/lima/lima_gem.c new file +10 -0
+     if (ret == 0)
+             ret = timeout ? -ETIMEDOUT : -EBUSY;
+     else if (ret > 0)
+             ret = 0;
+
+     lima_bo_unreserve(bo);
+out:
+     drm_gem_object_put_unlocked(obj);
+     return ret;
+}

From Documentation/botching-up-ioctls.txt:

* For timeouts, use absolute times. If you're a good fellow and made your ioctl restartable relative timeouts tend to be too coarse and can indefinitely extend your wait time due to rounding on each restart. Especially if your reference clock is something really slow like the display frame counter. With a spec lawyer hat on this isn't a bug since timeouts can always be extended - but users will surely hate you if their neat animations starts to stutter due to this.

Quoted 3 lines

(I made v3d's timeouts relative, but decrement the timeout value the user passed by how much I waited so that the timeout probably gets reduced after a restartable signal. I should have done absolute.)

timeout_ns in lima is already an absolute one which will be converted to relative one in lima_timeout_to_jiffies, is this what you want or I miss understand?

Quoted 25 lines
Quoted 25 lines
include/uapi/drm/lima_drm.h new file +18 -0
new file mode 100644
index 000000000000..c44757b4be39
 
+
+#define LIMA_SUBMIT_DEP_FENCE   0x00
+#define LIMA_SUBMIT_DEP_SYNC_FD 0x01
+
+struct drm_lima_gem_submit_dep_fence {
+     __u32 type;
+     __u32 ctx;
+     __u32 pipe;
+     __u32 seq;
+};
+
+struct drm_lima_gem_submit_dep_sync_fd {
+     __u32 type;
+     __u32 fd;
+};
+
+union drm_lima_gem_submit_dep {
+     __u32 type;
Quoted 8 lines

include/uapi/drm/lima_drm.h line 73

Quoted 3 lines
include/uapi/drm/lima_drm.h new file +3 -0
+     struct drm_lima_gem_submit_dep_fence fence;
+     struct drm_lima_gem_submit_dep_sync_fd sync_fd;
+};

I've been using gem sync objects for exposing my fences in v3d. You can import/export fences from sync files into syncobjs, and then you don't need a custom driver fence type in the uapi or your own ioctls for it if the submit just takes syncobjs in and out.

Sounds good, I'll consider about this way.

Quoted 14 lines

include/uapi/drm/lima_drm.h line 167

Quoted 9 lines
include/uapi/drm/lima_drm.h new file +8 -0
+#define LIMA_GEM_MOD_OP_GET 0
+#define LIMA_GEM_MOD_OP_SET 1
+
+struct drm_lima_gem_mod {
+     __u32 handle;      /* in */
+     __u32 op;          /* in */
+     __u64 modifier;    /* in/out */
+};

I thought the whole idea with the DRI3 modifiers stuff was that the kernel didn't need to store modifier metadata on buffers? (And this gets in the way of Vulkan modifiers support, from what I understand). Do you actually need this ABI?

Just for old apps when there's no user space modifier sharing method like the DRI3 modifiers, like old xserver.

Regards,
Qiang
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#9 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 48 lines · Qiang Yu
Quoted 47 lines · Eric Anholt
Quoted 24 lines · Qiang Yu
drivers/gpu/drm/lima/lima_gem.c new file +24 -0
+int lima_gem_wait(struct drm_file *file, u32 handle, u32 op, u64 timeout_ns)
+{
+     bool write = op & LIMA_GEM_WAIT_WRITE;
+     struct drm_gem_object *obj;
+     struct lima_bo *bo;
+     signed long ret;
+     unsigned long timeout;
+
+     obj = drm_gem_object_lookup(file, handle);
+     if (!obj)
+             return -ENOENT;
+
+     bo = to_lima_bo(obj);
+
+     timeout = timeout_ns ? lima_timeout_to_jiffies(timeout_ns) : 0;
+
+     ret = lima_bo_reserve(bo, true);
+     if (ret)
+             goto out;
+
+     /* must use long for result check because in 64bit arch int
+      * will overflow if timeout is too large and get <0 result
+      */
+     ret = reservation_object_wait_timeout_rcu(bo->tbo.resv, write, true, timeout);

drivers/gpu/drm/lima/lima_gem.c line 453

Quoted 10 lines
drivers/gpu/drm/lima/lima_gem.c new file +10 -0
+     if (ret == 0)
+             ret = timeout ? -ETIMEDOUT : -EBUSY;
+     else if (ret > 0)
+             ret = 0;
+
+     lima_bo_unreserve(bo);
+out:
+     drm_gem_object_put_unlocked(obj);
+     return ret;
+}

From Documentation/botching-up-ioctls.txt:

* For timeouts, use absolute times. If you're a good fellow and made your ioctl restartable relative timeouts tend to be too coarse and can indefinitely extend your wait time due to rounding on each restart. Especially if your reference clock is something really slow like the display frame counter. With a spec lawyer hat on this isn't a bug since timeouts can always be extended - but users will surely hate you if their neat animations starts to stutter due to this.

Quoted 6 lines
Quoted 3 lines

(I made v3d's timeouts relative, but decrement the timeout value the user passed by how much I waited so that the timeout probably gets reduced after a restartable signal. I should have done absolute.)

timeout_ns in lima is already an absolute one which will be converted to relative one in lima_timeout_to_jiffies, is this what you want or I miss understand?

Yeah, not sure how I missed the lima function, wich is clearly doing absolute. Sorry!

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#10 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

In reply to Qiang Yu
Quoted 2 lines · Qiang Yu

From: Lima Project Developers <[email protected]>

This should be a person (you).

Quoted 6 lines

Being the submitter, your S-o-b should be last.

Quoted 22 lines

---

drivers/gpu/drm/Kconfig +2 -0
index 4385f00e1d05..dfefcb393858 100644
@@ -333,6 +333,8 @@ source "drivers/gpu/drm/tve200/Kconfig"
 
 source "drivers/gpu/drm/xen/Kconfig"
 
+source "drivers/gpu/drm/lima/Kconfig"
+
 # Keep legacy drivers last
 
 menuconfig DRM_LEGACY
drivers/gpu/drm/Makefile +0 -0
index ce8d1d384319..8d024b729902 100644

drivers/gpu/drm/Makefile obj-$(CONFIG_DRM_TINYDRM) += tinydrm/ · line 112

Quoted 4 lines
drivers/gpu/drm/Makefile +1 -0
@@ -109,3 +109,4 @@ obj-$(CONFIG_DRM_TINYDRM) += tinydrm/
 obj-$(CONFIG_DRM_PL111) += pl111/
 obj-$(CONFIG_DRM_TVE200) += tve200/
 obj-$(CONFIG_DRM_XEN) += xen/
+obj-$(CONFIG_DRM_LIMA)  += lima/

Not sure about this file, but normally these should be kept sorted.

Quoted 8 lines
drivers/gpu/drm/lima/lima_bcast.c new file +2 -0
new file mode 100644
index 000000000000..63754f6465ea
@@ -0,0 +1,46 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2018 Qiang Yu <[email protected]> */

drivers/gpu/drm/lima/lima_bcast.c line 12

Quoted 10 lines
drivers/gpu/drm/lima/lima_bcast.c new file +10 -0
+
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_bcast.h"
+#include "lima_regs.h"
+
+#define bcast_write(reg, data) writel(data, ip->iomem + LIMA_BCAST_##reg)
+#define bcast_read(reg) readl(ip->iomem + LIMA_BCAST_##reg)

There are 2 things about this I would change. Just pass in 'ip' to the macro so it is clear in calling functions that ip is actually used. Second, don't do token pasting. It is generally avoided in the kernel. It makes grepping the source code harder and is a pointless indirection.

If you do both of those, then these can be static inline functions instead which are preferred because you get type checking.

Same comment applies to all the other register accessors.

drivers/gpu/drm/lima/lima_bcast.c line 25

Quoted 13 lines
drivers/gpu/drm/lima/lima_bcast.c new file +13 -0
+struct lima_ip {
+       struct lima_device *dev;
+       enum lima_ip_id id;
+       bool present;
+
+       void __iomem *iomem;
+       int irq;
+
+       union {
+               /* gp/pp */
+               bool async_reset;
+               /* l2 cache */
+               spinlock_t lock;

What happens when you need 2 elements for a sub-block. I'd make this a struct pointer for each IP sub-block.

Quoted 213 lines
drivers/gpu/drm/lima/lima_bcast.c new file +54 -0
+       } data;
+};
+
+enum lima_pipe_id {
+       lima_pipe_gp,
+       lima_pipe_pp,
+       lima_pipe_num,
+};
+
+struct lima_device {
+       struct device *dev;
+       struct drm_device *ddev;
+       struct platform_device *pdev;
+
+       enum lima_gpu_id id;
+       int num_pp;
+
+       void __iomem *iomem;
+       struct clk *clk_bus;
+       struct clk *clk_gpu;
+       struct reset_control *reset;
+       struct regulator *regulator;
+
+       struct lima_ip ip[lima_ip_num];
+       struct lima_sched_pipe pipe[lima_pipe_num];
+
+       struct lima_mman mman;
+
+       struct lima_vm *empty_vm;
+       uint64_t va_start;
+       uint64_t va_end;
+
+       u32 *dlbu_cpu;
+       dma_addr_t dlbu_dma;
+};
+
+static inline struct lima_device *
+to_lima_dev(struct drm_device *dev)
+{
+       return dev->dev_private;
+}
+
+static inline struct lima_device *
+ttm_to_lima_dev(struct ttm_bo_device *dev)
+{
+       return container_of(dev, struct lima_device, mman.bdev);
+}
+
+int lima_device_init(struct lima_device *ldev);
+void lima_device_fini(struct lima_device *ldev);
+
+const char *lima_ip_name(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_dlbu.c new file +56 -0
new file mode 100644
index 000000000000..6697d4ddd887
@@ -0,0 +1,56 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_dlbu.h"
+#include "lima_vm.h"
+#include "lima_regs.h"
+
+#define dlbu_write(reg, data) writel(data, ip->iomem + LIMA_DLBU_##reg)
+#define dlbu_read(reg) readl(ip->iomem + LIMA_DLBU_##reg)
+
+void lima_dlbu_enable(struct lima_device *dev, int num_pp)
+{
+       struct lima_sched_pipe *pipe = dev->pipe + lima_pipe_pp;
+       struct lima_ip *ip = dev->ip + lima_ip_dlbu;
+       int i, mask = 0;
+
+       for (i = 0; i < num_pp; i++) {
+               struct lima_ip *pp = pipe->processor[i];
+               mask |= 1 << (pp->id - lima_ip_pp0);
+       }
+
+       dlbu_write(PP_ENABLE_MASK, mask);
+}
+
+void lima_dlbu_disable(struct lima_device *dev)
+{
+       struct lima_ip *ip = dev->ip + lima_ip_dlbu;
+       dlbu_write(PP_ENABLE_MASK, 0);
+}
+
+void lima_dlbu_set_reg(struct lima_ip *ip, u32 *reg)
+{
+       dlbu_write(TLLIST_VBASEADDR, reg[0]);
+       dlbu_write(FB_DIM, reg[1]);
+       dlbu_write(TLLIST_CONF, reg[2]);
+       dlbu_write(START_TILE_POS, reg[3]);
+}
+
+int lima_dlbu_init(struct lima_ip *ip)
+{
+       struct lima_device *dev = ip->dev;
+
+       dlbu_write(MASTER_TLLIST_PHYS_ADDR, dev->dlbu_dma | 1);
+       dlbu_write(MASTER_TLLIST_VADDR, LIMA_VA_RESERVE_DLBU);
+
+       return 0;
+}
+
+void lima_dlbu_fini(struct lima_ip *ip)
+{
+
+}
drivers/gpu/drm/lima/lima_dlbu.h new file +18 -0
new file mode 100644
index 000000000000..60cba387cf30
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_DLBU_H__
+#define __LIMA_DLBU_H__
+
+struct lima_ip;
+struct lima_device;
+
+void lima_dlbu_enable(struct lima_device *dev, int num_pp);
+void lima_dlbu_disable(struct lima_device *dev);
+
+void lima_dlbu_set_reg(struct lima_ip *ip, u32 *reg);
+
+int lima_dlbu_init(struct lima_ip *ip);
+void lima_dlbu_fini(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_drv.c new file +67 -0
new file mode 100644
index 000000000000..132071b9be9b
@@ -0,0 +1,459 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/log2.h>
+#include <drm/drm_prime.h>
+#include <drm/lima_drm.h>
+
+#include "lima_drv.h"
+#include "lima_gem.h"
+#include "lima_gem_prime.h"
+#include "lima_vm.h"
+
+int lima_sched_timeout_ms = 0;
+int lima_sched_max_tasks = 32;
+int lima_max_mem = -1;
+
+MODULE_PARM_DESC(sched_timeout_ms, "task run timeout in ms (0 = no timeout (default))");
+module_param_named(sched_timeout_ms, lima_sched_timeout_ms, int, 0444);
+
+MODULE_PARM_DESC(sched_max_tasks, "max queued task num in a context (default 32)");
+module_param_named(sched_max_tasks, lima_sched_max_tasks, int, 0444);
+
+MODULE_PARM_DESC(max_mem, "Max memory size in MB can be used (<0 = auto)");
+module_param_named(max_mem, lima_max_mem, int, 0444);
+
+static int lima_ioctl_info(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_info *info = data;
+       struct lima_device *ldev = to_lima_dev(dev);
+
+       switch (ldev->id) {
+       case lima_gpu_mali400:
+               info->gpu_id = LIMA_INFO_GPU_MALI400;
+               break;
+       case lima_gpu_mali450:
+               info->gpu_id = LIMA_INFO_GPU_MALI450;
+               break;
+       default:
+               return -ENODEV;
+       }
+       info->num_pp = ldev->pipe[lima_pipe_pp].num_processor;
+       info->va_start = ldev->va_start;
+       info->va_end = ldev->va_end;
+       return 0;
+}
+
+static int lima_ioctl_gem_create(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_create *args = data;
+
+       if (args->flags)
+               return -EINVAL;
+
+       if (args->size == 0)
+               return -EINVAL;
+
+       return lima_gem_create_handle(dev, file, args->size, args->flags, &args->handle);
+}
+
+static int lima_ioctl_gem_info(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_info *args = data;
+
+       return lima_gem_mmap_offset(file, args->handle, &args->offset);
+}

drivers/gpu/drm/lima/lima_drv.c line 77

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

Quoted 98 lines
drivers/gpu/drm/lima/lima_drv.c new file +98 -0
+       default:
+               return -EINVAL;
+       }
+}
+
+static int lima_ioctl_gem_submit(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_submit_in *args = data;
+       struct lima_device *ldev = to_lima_dev(dev);
+       struct lima_drm_priv *priv = file->driver_priv;
+       struct drm_lima_gem_submit_bo *bos;
+       struct ttm_validate_buffer *vbs;
+       union drm_lima_gem_submit_dep *deps = NULL;
+       struct lima_sched_pipe *pipe;
+       struct lima_sched_task *task;
+       struct lima_ctx *ctx;
+       struct lima_submit submit = {0};
+       int err = 0, size;
+
+       if (args->pipe >= lima_pipe_num || args->nr_bos == 0)
+               return -EINVAL;
+
+       if (args->flags & ~(LIMA_SUBMIT_FLAG_EXPLICIT_FENCE |
+                           LIMA_SUBMIT_FLAG_SYNC_FD_OUT))
+               return -EINVAL;
+
+       pipe = ldev->pipe + args->pipe;
+       if (args->frame_size != pipe->frame_size)
+               return -EINVAL;
+
+       size = args->nr_bos * (sizeof(*submit.bos) + sizeof(*submit.vbs)) +
+               args->nr_deps * sizeof(*submit.deps);
+       bos = kzalloc(size, GFP_KERNEL);
+       if (!bos)
+               return -ENOMEM;
+
+       size = args->nr_bos * sizeof(*submit.bos);
+       if (copy_from_user(bos, u64_to_user_ptr(args->bos), size)) {
+               err = -EFAULT;
+               goto out0;
+       }
+
+       vbs = (void *)bos + size;
+
+       if (args->nr_deps) {
+               deps = (void *)vbs + args->nr_bos * sizeof(*submit.vbs);
+               size = args->nr_deps * sizeof(*submit.deps);
+               if (copy_from_user(deps, u64_to_user_ptr(args->deps), size)) {
+                       err = -EFAULT;
+                       goto out0;
+               }
+       }
+
+       task = kmem_cache_zalloc(pipe->task_slab, GFP_KERNEL);
+       if (!task) {
+               err = -ENOMEM;
+               goto out0;
+       }
+
+       task->frame = task + 1;
+       if (copy_from_user(task->frame, u64_to_user_ptr(args->frame), args->frame_size)) {
+               err = -EFAULT;
+               goto out1;
+       }
+
+       err = pipe->task_validate(pipe, task);
+       if (err)
+               goto out1;
+
+       ctx = lima_ctx_get(&priv->ctx_mgr, args->ctx);
+       if (!ctx) {
+               err = -ENOENT;
+               goto out1;
+       }
+
+       submit.pipe = args->pipe;
+       submit.bos = bos;
+       submit.vbs = vbs;
+       submit.nr_bos = args->nr_bos;
+       submit.task = task;
+       submit.ctx = ctx;
+       submit.deps = deps;
+       submit.nr_deps = args->nr_deps;
+       submit.flags = args->flags;
+
+       err = lima_gem_submit(file, &submit);
+       if (!err) {
+               struct drm_lima_gem_submit_out *out = data;
+               out->fence = submit.fence;
+               out->done = submit.done;
+               out->sync_fd = submit.sync_fd;
+       }
+
+       lima_ctx_put(ctx);
+out1:
+       if (err)
+               kmem_cache_free(pipe->task_slab, task);
+out0:

drivers/gpu/drm/lima/lima_drv.c line 185

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+       kfree(bos);
+       return err;
+}
+
+static int lima_wait_fence(struct dma_fence *fence, u64 timeout_ns)
+{
+       signed long ret;
+
+       if (!timeout_ns)
+               ret = dma_fence_is_signaled(fence) ? 0 : -EBUSY;

I think you can just call dma_fence_wait_timeout with a 0 timeout below and remove this clause.

drivers/gpu/drm/lima/lima_drv.c line 197

Quoted 12 lines
drivers/gpu/drm/lima/lima_drv.c new file +12 -0
+       else {
+               unsigned long timeout = lima_timeout_to_jiffies(timeout_ns);
+
+               /* must use long for result check because in 64bit arch int
+                * will overflow if timeout is too large and get <0 result
+                */
+               ret = dma_fence_wait_timeout(fence, true, timeout);
+               if (ret == 0)
+                       ret = timeout ? -ETIMEDOUT : -EBUSY;
+               else if (ret > 0)
+                       ret = 0;
+       }

I suspect this could be common like reservation object waits. However, I'm curious why lima needs this ioctl in the first place when I don't see the same for other drivers.

Quoted 124 lines
drivers/gpu/drm/lima/lima_drv.c new file +124 -0
+
+       return ret;
+}
+
+static int lima_ioctl_wait_fence(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_wait_fence *args = data;
+       struct lima_drm_priv *priv = file->driver_priv;
+       struct dma_fence *fence;
+       int err = 0;
+
+       fence = lima_ctx_get_native_fence(&priv->ctx_mgr, args->ctx,
+                                         args->pipe, args->seq);
+       if (IS_ERR(fence))
+               return PTR_ERR(fence);
+
+       if (fence) {
+               err = lima_wait_fence(fence, args->timeout_ns);
+               args->error = fence->error;
+               dma_fence_put(fence);
+       }
+       else
+               args->error = 0;
+
+       return err;
+}
+
+static int lima_ioctl_gem_wait(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_wait *args = data;
+
+       if (!(args->op & (LIMA_GEM_WAIT_READ|LIMA_GEM_WAIT_WRITE)))
+           return -EINVAL;
+
+       return lima_gem_wait(file, args->handle, args->op, args->timeout_ns);
+}
+
+static int lima_ioctl_ctx(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_ctx *args = data;
+       struct lima_drm_priv *priv = file->driver_priv;
+       struct lima_device *ldev = to_lima_dev(dev);
+
+       if (args->op == LIMA_CTX_OP_CREATE)
+               return lima_ctx_create(ldev, &priv->ctx_mgr, &args->id);
+       else if (args->op == LIMA_CTX_OP_FREE)
+               return lima_ctx_free(&priv->ctx_mgr, args->id);
+
+       return -EINVAL;
+}
+
+static int lima_ioctl_gem_mod(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_mod *args = data;
+
+       if (args->op == LIMA_GEM_MOD_OP_GET)
+               return lima_gem_get_modifier(file, args->handle, &args->modifier);
+       else if (args->op == LIMA_GEM_MOD_OP_SET)
+               return lima_gem_set_modifier(file, args->handle, args->modifier);
+
+       return -EINVAL;
+}
+
+static int lima_drm_driver_open(struct drm_device *dev, struct drm_file *file)
+{
+       int err;
+       struct lima_drm_priv *priv;
+       struct lima_device *ldev = to_lima_dev(dev);
+
+       priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+       if (!priv)
+               return -ENOMEM;
+
+       priv->vm = lima_vm_create(ldev);
+       if (!priv->vm) {
+               err = -ENOMEM;
+               goto err_out0;
+       }
+
+        lima_ctx_mgr_init(&priv->ctx_mgr);
+
+       file->driver_priv = priv;
+       return 0;
+
+err_out0:
+       kfree(priv);
+       return err;
+}
+
+static void lima_drm_driver_postclose(struct drm_device *dev, struct drm_file *file)
+{
+       struct lima_drm_priv *priv = file->driver_priv;
+
+       lima_ctx_mgr_fini(&priv->ctx_mgr);
+       lima_vm_put(priv->vm);
+       kfree(priv);
+}
+
+static const struct drm_ioctl_desc lima_drm_driver_ioctls[] = {
+       DRM_IOCTL_DEF_DRV(LIMA_INFO, lima_ioctl_info, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_CREATE, lima_ioctl_gem_create, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_INFO, lima_ioctl_gem_info, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_VA, lima_ioctl_gem_va, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_SUBMIT, lima_ioctl_gem_submit, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_WAIT_FENCE, lima_ioctl_wait_fence, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_WAIT, lima_ioctl_gem_wait, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_CTX, lima_ioctl_ctx, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_MOD, lima_ioctl_gem_mod, DRM_AUTH|DRM_RENDER_ALLOW),
+};
+
+static const struct file_operations lima_drm_driver_fops = {
+       .owner              = THIS_MODULE,
+       .open               = drm_open,
+       .release            = drm_release,
+       .unlocked_ioctl     = drm_ioctl,
+#ifdef CONFIG_COMPAT
+       .compat_ioctl       = drm_compat_ioctl,
+#endif
+       .mmap               = lima_gem_mmap,
+};
+
+static struct drm_driver lima_drm_driver = {
+       .driver_features    = DRIVER_RENDER | DRIVER_GEM | DRIVER_PRIME,
+       .open               = lima_drm_driver_open,

drivers/gpu/drm/lima/lima_drv.c line 331

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+       .postclose          = lima_drm_driver_postclose,
+       .ioctls             = lima_drm_driver_ioctls,
+       .num_ioctls         = ARRAY_SIZE(lima_drm_driver_ioctls),
+       .fops               = &lima_drm_driver_fops,
+       .gem_free_object_unlocked = lima_gem_free_object,
+       .gem_open_object    = lima_gem_object_open,
+       .gem_close_object   = lima_gem_object_close,
+       .name               = "lima",
+       .desc               = "lima DRM",
+       .date               = "20170325",

Perhaps this should be updated? TBH, I don't know why this is even useful.

drivers/gpu/drm/lima/lima_drv.c line 340

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +9 -0
+       .major              = 1,
+       .minor              = 0,
+       .patchlevel         = 0,
+
+       .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
+       .gem_prime_import   = drm_gem_prime_import,
+       .gem_prime_import_sg_table = lima_gem_prime_import_sg_table,
+       .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
+       .gem_prime_export   = drm_gem_prime_export,

import and export don't have to be set if you use the defaults.

Quoted 26 lines
drivers/gpu/drm/lima/lima_drv.c new file +25 -0
+       .gem_prime_res_obj  = lima_gem_prime_res_obj,
+       .gem_prime_get_sg_table = lima_gem_prime_get_sg_table,
+       .gem_prime_vmap = lima_gem_prime_vmap,
+       .gem_prime_vunmap = lima_gem_prime_vunmap,
+       .gem_prime_mmap = lima_gem_prime_mmap,
+};
+
 
+int lima_gem_wait(struct drm_file *file, u32 handle, u32 op, u64 timeout_ns)
+{
+       bool write = op & LIMA_GEM_WAIT_WRITE;
+       struct drm_gem_object *obj;
+       struct lima_bo *bo;
+       signed long ret;
+       unsigned long timeout;
+
+       obj = drm_gem_object_lookup(file, handle);
+       if (!obj)
+               return -ENOENT;
+
+       bo = to_lima_bo(obj);
+
+       timeout = timeout_ns ? lima_timeout_to_jiffies(timeout_ns) : 0;
+
+       ret = lima_bo_reserve(bo, true);
+       if (ret)

drivers/gpu/drm/lima/lima_drv.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+               goto out;
+
+       /* must use long for result check because in 64bit arch int
+        * will overflow if timeout is too large and get <0 result
+        */
+       ret = reservation_object_wait_timeout_rcu(bo->tbo.resv, write, true, timeout);
+       if (ret == 0)
+               ret = timeout ? -ETIMEDOUT : -EBUSY;
+       else if (ret > 0)
+               ret = 0;

There's a helper I added for all this that should land in 5.1.

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +8 -0
+
+       lima_bo_unreserve(bo);
+out:
+       drm_gem_object_put_unlocked(obj);
+       return ret;
+}
+
 
+static int lima_gp_soft_reset_async_wait(struct lima_ip *ip)

drivers/gpu/drm/lima/lima_drv.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+{
+       struct lima_device *dev = ip->dev;
+       int timeout;
+
+       if (!ip->data.async_reset)
+               return 0;
+
+       for (timeout = 1000; timeout > 0; timeout--) {
+               if (gp_read(INT_RAWSTAT) & LIMA_GP_IRQ_RESET_COMPLETED)
+                       break;

Use readl_poll_timeout instead of writing your own. At least add a udelay to the loop so the timing is fixed and not dependent on how fast the code can run.

Quoted 30 lines
drivers/gpu/drm/lima/lima_drv.c new file +12 -0
+       }
+       if (!timeout) {
+               dev_err(dev->dev, "gp soft reset time out\n");
+               return -ETIMEDOUT;
+       }
+
+       gp_write(INT_CLEAR, LIMA_GP_IRQ_MASK_ALL);
+       gp_write(INT_MASK, LIMA_GP_IRQ_MASK_USED);
+
+       ip->data.async_reset = false;
+       return 0;
+}
drivers/gpu/drm/lima/lima_l2_cache.c new file +11 -0
new file mode 100644
index 000000000000..e7cdec720e5d
@@ -0,0 +1,79 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_l2_cache.h"
+#include "lima_regs.h"
+
+#define l2_cache_write(reg, data) writel(data, ip->iomem + LIMA_L2_CACHE_##reg)

drivers/gpu/drm/lima/lima_l2_cache.c line 21

Quoted 10 lines
drivers/gpu/drm/lima/lima_l2_cache.c new file +10 -0
+#define l2_cache_read(reg) readl(ip->iomem + LIMA_L2_CACHE_##reg)
+
+static int lima_l2_cache_wait_idle(struct lima_ip *ip)
+{
+       int timeout;
+       struct lima_device *dev = ip->dev;
+
+       for (timeout = 100000; timeout > 0; timeout--) {
+           if (!(l2_cache_read(STATUS) & LIMA_L2_CACHE_STATUS_COMMAND_BUSY))
+               break;

Use readl_poll_timeout or variant.

Quoted 115 lines
drivers/gpu/drm/lima/lima_l2_cache.c new file +58 -0
+       }
+       if (!timeout) {
+           dev_err(dev->dev, "l2 cache wait command timeout\n");
+           return -ETIMEDOUT;
+       }
+       return 0;
+}
+
+int lima_l2_cache_flush(struct lima_ip *ip)
+{
+       int ret;
+
+       spin_lock(&ip->data.lock);
+       l2_cache_write(COMMAND, LIMA_L2_CACHE_COMMAND_CLEAR_ALL);
+       ret = lima_l2_cache_wait_idle(ip);
+       spin_unlock(&ip->data.lock);
+       return ret;
+}
+
+int lima_l2_cache_init(struct lima_ip *ip)
+{
+       int i, err;
+       u32 size;
+       struct lima_device *dev = ip->dev;
+
+       /* l2_cache2 only exists when one of PP4-7 present */
+       if (ip->id == lima_ip_l2_cache2) {
+               for (i = lima_ip_pp4; i <= lima_ip_pp7; i++) {
+                       if (dev->ip[i].present)
+                               break;
+               }
+               if (i > lima_ip_pp7)
+                       return -ENODEV;
+       }
+
+       spin_lock_init(&ip->data.lock);
+
+       size = l2_cache_read(SIZE);
+       dev_info(dev->dev, "l2 cache %uK, %u-way, %ubyte cache line, %ubit external bus\n",
+                1 << (((size >> 16) & 0xff) - 10),
+                1 << ((size >> 8) & 0xff),
+                1 << (size & 0xff),
+                1 << ((size >> 24) & 0xff));
+
+       err = lima_l2_cache_flush(ip);
+       if (err)
+               return err;
+
+       l2_cache_write(ENABLE, LIMA_L2_CACHE_ENABLE_ACCESS | LIMA_L2_CACHE_ENABLE_READ_ALLOCATE);
+       l2_cache_write(MAX_READS, 0x1c);
+
+       return 0;
+}
+
+void lima_l2_cache_fini(struct lima_ip *ip)
+{
+
+}
drivers/gpu/drm/lima/lima_l2_cache.h new file +14 -0
new file mode 100644
index 000000000000..2ff91eafefbe
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_L2_CACHE_H__
+#define __LIMA_L2_CACHE_H__
+
+struct lima_ip;
+
+int lima_l2_cache_init(struct lima_ip *ip);
+void lima_l2_cache_fini(struct lima_ip *ip);
+
+int lima_l2_cache_flush(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_mmu.c new file +31 -0
new file mode 100644
index 000000000000..234fb90a4285
@@ -0,0 +1,135 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_mmu.h"
+#include "lima_vm.h"
+#include "lima_object.h"
+#include "lima_regs.h"
+
+#define mmu_write(reg, data) writel(data, ip->iomem + LIMA_MMU_##reg)
+#define mmu_read(reg) readl(ip->iomem + LIMA_MMU_##reg)
+
+#define lima_mmu_send_command(command, condition)           \
+({                                                          \
+       int __timeout, __ret = 0;                            \
+                                                            \
+       mmu_write(COMMAND, command);                         \
+       for (__timeout = 1000; __timeout > 0; __timeout--) { \
+               if (condition)                               \
+                       break;                               \
+       }                                                    \
+       if (!__timeout) {                                    \
+               dev_err(dev->dev, "mmu command %x timeout\n", command); \
+               __ret = -ETIMEDOUT;                          \
+       }                                                    \
+       __ret;                                               \
+})

drivers/gpu/drm/lima/lima_mmu.c line 41

Quoted 10 lines
drivers/gpu/drm/lima/lima_mmu.c new file +10 -0
+
+static irqreturn_t lima_mmu_irq_handler(int irq, void *data)
+{
+       struct lima_ip *ip = data;
+       struct lima_device *dev = ip->dev;
+       u32 status = mmu_read(INT_STATUS);
+       struct lima_sched_pipe *pipe;
+
+       /* for shared irq case */
+       if (!status)

Can status have masked irq's? If so, you should be masking out the disabled irq bits.

Quoted 17 lines
drivers/gpu/drm/lima/lima_mmu.c new file +17 -0
+               return IRQ_NONE;
+
+       if (status & LIMA_MMU_INT_PAGE_FAULT) {
+               u32 fault = mmu_read(PAGE_FAULT_ADDR);
+               dev_err(dev->dev, "mmu page fault at 0x%x from bus id %d of type %s on %s\n",
+                       fault, LIMA_MMU_STATUS_BUS_ID(status),
+                       status & LIMA_MMU_STATUS_PAGE_FAULT_IS_WRITE ? "write" : "read",
+                       lima_ip_name(ip));
+       }
+
+       if (status & LIMA_MMU_INT_READ_BUS_ERROR) {
+               dev_err(dev->dev, "mmu %s irq bus error\n", lima_ip_name(ip));
+       }
+
+       /* mask all interrupts before resume */
+       mmu_write(INT_MASK, 0);
+       mmu_write(INT_CLEAR, status);

drivers/gpu/drm/lima/lima_mmu.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_mmu.c new file +8 -0
+
+       pipe = dev->pipe + (ip->id == lima_ip_gpmmu ? lima_pipe_gp : lima_pipe_pp);
+       lima_sched_pipe_mmu_error(pipe);
+
+       return IRQ_HANDLED;
+}
 
 
+
+unsigned long lima_timeout_to_jiffies(u64 timeout_ns)

Create a common helper instead of copy-n-pasting this from other drivers (etnaviv).

Quoted 116 lines
drivers/gpu/drm/lima/lima_mmu.c new file +31 -0
+{
+       unsigned long timeout_jiffies;
+       ktime_t timeout;
+
+       /* clamp timeout if it's to large */
+       if (((s64)timeout_ns) < 0)
+               return MAX_SCHEDULE_TIMEOUT;
+
+       timeout = ktime_sub(ns_to_ktime(timeout_ns), ktime_get());
+       if (ktime_to_ns(timeout) < 0)
+               return 0;
+
+       timeout_jiffies = nsecs_to_jiffies(ktime_to_ns(timeout));
+       /*  clamp timeout to avoid unsigned-> signed overflow */
+       if (timeout_jiffies > MAX_SCHEDULE_TIMEOUT )
+               return MAX_SCHEDULE_TIMEOUT;
+
+       return timeout_jiffies;
+}
+
+void lima_sched_pipe_task_done(struct lima_sched_pipe *pipe)
+{
+       if (pipe->error)
+               schedule_work(&pipe->error_work);
+       else {
+               struct lima_sched_task *task = pipe->current_task;
+
+               pipe->task_fini(pipe);
+               dma_fence_signal(task->fence);
+       }
+}
drivers/gpu/drm/lima/lima_vm.c new file +78 -0
new file mode 100644
index 000000000000..a264f3ae83fe
@@ -0,0 +1,354 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/slab.h>
+#include <linux/dma-mapping.h>
+#include <linux/interval_tree_generic.h>
+
+#include "lima_device.h"
+#include "lima_vm.h"
+#include "lima_object.h"
+#include "lima_regs.h"
+
+struct lima_bo_va_mapping {
+       struct list_head list;
+       struct rb_node rb;
+       uint32_t start;
+       uint32_t last;
+       uint32_t __subtree_last;
+};
+
+struct lima_bo_va {
+       struct list_head list;
+       unsigned ref_count;
+
+       struct list_head mapping;
+
+       struct lima_vm *vm;
+};
+
+#define LIMA_VM_PD_SHIFT 22
+#define LIMA_VM_PT_SHIFT 12
+#define LIMA_VM_PB_SHIFT (LIMA_VM_PD_SHIFT + LIMA_VM_NUM_PT_PER_BT_SHIFT)
+#define LIMA_VM_BT_SHIFT LIMA_VM_PT_SHIFT
+
+#define LIMA_VM_PT_MASK ((1 << LIMA_VM_PD_SHIFT) - 1)
+#define LIMA_VM_BT_MASK ((1 << LIMA_VM_PB_SHIFT) - 1)
+
+#define LIMA_PDE(va) (va >> LIMA_VM_PD_SHIFT)
+#define LIMA_PTE(va) ((va & LIMA_VM_PT_MASK) >> LIMA_VM_PT_SHIFT)
+#define LIMA_PBE(va) (va >> LIMA_VM_PB_SHIFT)
+#define LIMA_BTE(va) ((va & LIMA_VM_BT_MASK) >> LIMA_VM_BT_SHIFT)
+
+#define START(node) ((node)->start)
+#define LAST(node) ((node)->last)
+
+INTERVAL_TREE_DEFINE(struct lima_bo_va_mapping, rb, uint32_t, __subtree_last,
+                    START, LAST, static, lima_vm_it)
+
+#undef START
+#undef LAST
+
+static void lima_vm_unmap_page_table(struct lima_vm *vm, u32 start, u32 end)
+{
+       u32 addr;
+
+       for (addr = start; addr <= end; addr += LIMA_PAGE_SIZE) {
+               u32 pbe = LIMA_PBE(addr);
+               u32 bte = LIMA_BTE(addr);
+               u32 *bt;
+
+               bt = lima_bo_kmap(vm->bts[pbe]);
+               bt[bte] = 0;
+       }
+}
+
+static int lima_vm_map_page_table(struct lima_vm *vm, dma_addr_t *dma,
+                                 u32 start, u32 end)
+{
+       u64 addr;
+       int err, i = 0;
+
+       for (addr = start; addr <= end; addr += LIMA_PAGE_SIZE) {
+               u32 pbe = LIMA_PBE(addr);
+               u32 bte = LIMA_BTE(addr);
+               u32 *bt;
+
+               if (vm->bts[pbe])
+                       bt = lima_bo_kmap(vm->bts[pbe]);

drivers/gpu/drm/lima/lima_vm.c line 88

Quoted 10 lines
drivers/gpu/drm/lima/lima_vm.c new file +10 -0
+               else {
+                       struct lima_bo *bt_bo;
+                       dma_addr_t *pts;
+                       u32 *pd;
+                       int j;
+
+                       bt_bo = lima_bo_create(
+                               vm->dev, LIMA_PAGE_SIZE << LIMA_VM_NUM_PT_PER_BT_SHIFT,
+                               0, ttm_bo_type_kernel,
+                               NULL, vm->pd->tbo.resv);

I don't think using BOs for page tables buys you anything. You could just use the kernel DMA API directly. See io-pgtable-arm-v7s.c for inspiration. For panfrost, it's standard ARM format page tables so we can just use the io-pgtable library.

Quoted 57 lines
drivers/gpu/drm/lima/lima_vm.c new file +57 -0
+                       if (IS_ERR(bt_bo)) {
+                               err = PTR_ERR(bt_bo);
+                               goto err_out;
+                       }
+
+                       bt = lima_bo_kmap(bt_bo);
+                       if (IS_ERR(bt)) {
+                               lima_bo_unref(bt_bo);
+                               err = PTR_ERR(bt);
+                               goto err_out;
+                       }
+                       memset(bt, 0, LIMA_PAGE_SIZE << LIMA_VM_NUM_PT_PER_BT_SHIFT);
+
+                       vm->bts[pbe] = bt_bo;
+                       pd = lima_bo_kmap(vm->pd);
+                       pd += pbe << LIMA_VM_NUM_PT_PER_BT_SHIFT;
+                       pts = lima_bo_get_pages(bt_bo);
+                       for (j = 0; j < LIMA_VM_NUM_PT_PER_BT; j++)
+                               *pd++ = *pts++ | LIMA_VM_FLAG_PRESENT;
+               }
+
+               bt[bte] = dma[i++] | LIMA_VM_FLAGS_CACHE;
+       }
+
+       return 0;
+
+err_out:
+       if (addr != start)
+               lima_vm_unmap_page_table(vm, start, addr - 1);
+       return err;
+}
+
+static struct lima_bo_va *
+lima_vm_bo_find(struct lima_vm *vm, struct lima_bo *bo)
+{
+       struct lima_bo_va *bo_va, *ret = NULL;
+
+       list_for_each_entry(bo_va, &bo->va, list) {
+               if (bo_va->vm == vm) {
+                       ret = bo_va;
+                       break;
+               }
+       }
+
+       return ret;
+}
+
+int lima_vm_bo_map(struct lima_vm *vm, struct lima_bo *bo, u32 start)
+{
+       int err;
+       struct lima_bo_va_mapping *it, *mapping;
+       u32 end = start + bo->gem.size - 1;
+       dma_addr_t *pages_dma = lima_bo_get_pages(bo);
+       struct lima_bo_va *bo_va;
+
+       it = lima_vm_it_iter_first(&vm->va, start, end);
+       if (it) {

drivers/gpu/drm/lima/lima_vm.c line 155

Quoted 10 lines
drivers/gpu/drm/lima/lima_vm.c new file +10 -0
+               dev_dbg(bo->gem.dev->dev, "lima vm map va overlap %x-%x %x-%x\n",
+                       start, end, it->start, it->last);
+               return -EINVAL;
+       }
+
+       mapping = kmalloc(sizeof(*mapping), GFP_KERNEL);
+       if (!mapping)
+               return -ENOMEM;
+       mapping->start = start;
+       mapping->last = end;

Why don't you use the drm_mm_XX APIs instead of writing your own?

Quoted 4 lines
drivers/gpu/drm/lima/lima_vm.c new file +4 -0
+
+       err = lima_vm_map_page_table(vm, pages_dma, start, end);
+       if (err) {
+               kfree(mapping);

drivers/gpu/drm/lima/lima_vm.c line 169

Quoted 10 lines
drivers/gpu/drm/lima/lima_vm.c new file +10 -0
+               return err;
+       }
+
+       lima_vm_it_insert(mapping, &vm->va);
+
+       bo_va = lima_vm_bo_find(vm, bo);
+       list_add_tail(&mapping->list, &bo_va->mapping);
+
+       return 0;
+}

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#11 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 321 lines · Rob Herring via dri-devel
Quoted 2 lines · Qiang Yu

From: Lima Project Developers <[email protected]>

This should be a person (you).

Quoted 6 lines

Being the submitter, your S-o-b should be last.

Quoted 22 lines

---

drivers/gpu/drm/Kconfig +2 -0
index 4385f00e1d05..dfefcb393858 100644
@@ -333,6 +333,8 @@ source "drivers/gpu/drm/tve200/Kconfig"
 
 source "drivers/gpu/drm/xen/Kconfig"
 
+source "drivers/gpu/drm/lima/Kconfig"
+
 # Keep legacy drivers last
 
 menuconfig DRM_LEGACY
drivers/gpu/drm/Makefile +0 -0
index ce8d1d384319..8d024b729902 100644

drivers/gpu/drm/Makefile obj-$(CONFIG_DRM_TINYDRM) += tinydrm/ · line 112

Quoted 4 lines
drivers/gpu/drm/Makefile +1 -0
@@ -109,3 +109,4 @@ obj-$(CONFIG_DRM_TINYDRM) += tinydrm/
 obj-$(CONFIG_DRM_PL111) += pl111/
 obj-$(CONFIG_DRM_TVE200) += tve200/
 obj-$(CONFIG_DRM_XEN) += xen/
+obj-$(CONFIG_DRM_LIMA)  += lima/

Not sure about this file, but normally these should be kept sorted.

Quoted 8 lines
drivers/gpu/drm/lima/lima_bcast.c new file +2 -0
new file mode 100644
index 000000000000..63754f6465ea
@@ -0,0 +1,46 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2018 Qiang Yu <[email protected]> */

drivers/gpu/drm/lima/lima_bcast.c line 12

Quoted 10 lines
drivers/gpu/drm/lima/lima_bcast.c new file +10 -0
+
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_bcast.h"
+#include "lima_regs.h"
+
+#define bcast_write(reg, data) writel(data, ip->iomem + LIMA_BCAST_##reg)
+#define bcast_read(reg) readl(ip->iomem + LIMA_BCAST_##reg)

There are 2 things about this I would change. Just pass in 'ip' to the macro so it is clear in calling functions that ip is actually used. Second, don't do token pasting. It is generally avoided in the kernel. It makes grepping the source code harder and is a pointless indirection.

If you do both of those, then these can be static inline functions instead which are preferred because you get type checking.

Same comment applies to all the other register accessors.

drivers/gpu/drm/lima/lima_bcast.c line 25

Quoted 13 lines
drivers/gpu/drm/lima/lima_bcast.c new file +13 -0
+struct lima_ip {
+       struct lima_device *dev;
+       enum lima_ip_id id;
+       bool present;
+
+       void __iomem *iomem;
+       int irq;
+
+       union {
+               /* gp/pp */
+               bool async_reset;
+               /* l2 cache */
+               spinlock_t lock;

What happens when you need 2 elements for a sub-block. I'd make this a struct pointer for each IP sub-block.

Quoted 213 lines
drivers/gpu/drm/lima/lima_bcast.c new file +54 -0
+       } data;
+};
+
+enum lima_pipe_id {
+       lima_pipe_gp,
+       lima_pipe_pp,
+       lima_pipe_num,
+};
+
+struct lima_device {
+       struct device *dev;
+       struct drm_device *ddev;
+       struct platform_device *pdev;
+
+       enum lima_gpu_id id;
+       int num_pp;
+
+       void __iomem *iomem;
+       struct clk *clk_bus;
+       struct clk *clk_gpu;
+       struct reset_control *reset;
+       struct regulator *regulator;
+
+       struct lima_ip ip[lima_ip_num];
+       struct lima_sched_pipe pipe[lima_pipe_num];
+
+       struct lima_mman mman;
+
+       struct lima_vm *empty_vm;
+       uint64_t va_start;
+       uint64_t va_end;
+
+       u32 *dlbu_cpu;
+       dma_addr_t dlbu_dma;
+};
+
+static inline struct lima_device *
+to_lima_dev(struct drm_device *dev)
+{
+       return dev->dev_private;
+}
+
+static inline struct lima_device *
+ttm_to_lima_dev(struct ttm_bo_device *dev)
+{
+       return container_of(dev, struct lima_device, mman.bdev);
+}
+
+int lima_device_init(struct lima_device *ldev);
+void lima_device_fini(struct lima_device *ldev);
+
+const char *lima_ip_name(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_dlbu.c new file +56 -0
new file mode 100644
index 000000000000..6697d4ddd887
@@ -0,0 +1,56 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_dlbu.h"
+#include "lima_vm.h"
+#include "lima_regs.h"
+
+#define dlbu_write(reg, data) writel(data, ip->iomem + LIMA_DLBU_##reg)
+#define dlbu_read(reg) readl(ip->iomem + LIMA_DLBU_##reg)
+
+void lima_dlbu_enable(struct lima_device *dev, int num_pp)
+{
+       struct lima_sched_pipe *pipe = dev->pipe + lima_pipe_pp;
+       struct lima_ip *ip = dev->ip + lima_ip_dlbu;
+       int i, mask = 0;
+
+       for (i = 0; i < num_pp; i++) {
+               struct lima_ip *pp = pipe->processor[i];
+               mask |= 1 << (pp->id - lima_ip_pp0);
+       }
+
+       dlbu_write(PP_ENABLE_MASK, mask);
+}
+
+void lima_dlbu_disable(struct lima_device *dev)
+{
+       struct lima_ip *ip = dev->ip + lima_ip_dlbu;
+       dlbu_write(PP_ENABLE_MASK, 0);
+}
+
+void lima_dlbu_set_reg(struct lima_ip *ip, u32 *reg)
+{
+       dlbu_write(TLLIST_VBASEADDR, reg[0]);
+       dlbu_write(FB_DIM, reg[1]);
+       dlbu_write(TLLIST_CONF, reg[2]);
+       dlbu_write(START_TILE_POS, reg[3]);
+}
+
+int lima_dlbu_init(struct lima_ip *ip)
+{
+       struct lima_device *dev = ip->dev;
+
+       dlbu_write(MASTER_TLLIST_PHYS_ADDR, dev->dlbu_dma | 1);
+       dlbu_write(MASTER_TLLIST_VADDR, LIMA_VA_RESERVE_DLBU);
+
+       return 0;
+}
+
+void lima_dlbu_fini(struct lima_ip *ip)
+{
+
+}
drivers/gpu/drm/lima/lima_dlbu.h new file +18 -0
new file mode 100644
index 000000000000..60cba387cf30
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_DLBU_H__
+#define __LIMA_DLBU_H__
+
+struct lima_ip;
+struct lima_device;
+
+void lima_dlbu_enable(struct lima_device *dev, int num_pp);
+void lima_dlbu_disable(struct lima_device *dev);
+
+void lima_dlbu_set_reg(struct lima_ip *ip, u32 *reg);
+
+int lima_dlbu_init(struct lima_ip *ip);
+void lima_dlbu_fini(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_drv.c new file +67 -0
new file mode 100644
index 000000000000..132071b9be9b
@@ -0,0 +1,459 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/log2.h>
+#include <drm/drm_prime.h>
+#include <drm/lima_drm.h>
+
+#include "lima_drv.h"
+#include "lima_gem.h"
+#include "lima_gem_prime.h"
+#include "lima_vm.h"
+
+int lima_sched_timeout_ms = 0;
+int lima_sched_max_tasks = 32;
+int lima_max_mem = -1;
+
+MODULE_PARM_DESC(sched_timeout_ms, "task run timeout in ms (0 = no timeout (default))");
+module_param_named(sched_timeout_ms, lima_sched_timeout_ms, int, 0444);
+
+MODULE_PARM_DESC(sched_max_tasks, "max queued task num in a context (default 32)");
+module_param_named(sched_max_tasks, lima_sched_max_tasks, int, 0444);
+
+MODULE_PARM_DESC(max_mem, "Max memory size in MB can be used (<0 = auto)");
+module_param_named(max_mem, lima_max_mem, int, 0444);
+
+static int lima_ioctl_info(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_info *info = data;
+       struct lima_device *ldev = to_lima_dev(dev);
+
+       switch (ldev->id) {
+       case lima_gpu_mali400:
+               info->gpu_id = LIMA_INFO_GPU_MALI400;
+               break;
+       case lima_gpu_mali450:
+               info->gpu_id = LIMA_INFO_GPU_MALI450;
+               break;
+       default:
+               return -ENODEV;
+       }
+       info->num_pp = ldev->pipe[lima_pipe_pp].num_processor;
+       info->va_start = ldev->va_start;
+       info->va_end = ldev->va_end;
+       return 0;
+}
+
+static int lima_ioctl_gem_create(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_create *args = data;
+
+       if (args->flags)
+               return -EINVAL;
+
+       if (args->size == 0)
+               return -EINVAL;
+
+       return lima_gem_create_handle(dev, file, args->size, args->flags, &args->handle);
+}
+
+static int lima_ioctl_gem_info(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_info *args = data;
+
+       return lima_gem_mmap_offset(file, args->handle, &args->offset);
+}

drivers/gpu/drm/lima/lima_drv.c line 77

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Alex

Quoted 757 lines
Quoted 99 lines
drivers/gpu/drm/lima/lima_drv.c new file +98 -0
+       default:
+               return -EINVAL;
+       }
+}
+
+static int lima_ioctl_gem_submit(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_submit_in *args = data;
+       struct lima_device *ldev = to_lima_dev(dev);
+       struct lima_drm_priv *priv = file->driver_priv;
+       struct drm_lima_gem_submit_bo *bos;
+       struct ttm_validate_buffer *vbs;
+       union drm_lima_gem_submit_dep *deps = NULL;
+       struct lima_sched_pipe *pipe;
+       struct lima_sched_task *task;
+       struct lima_ctx *ctx;
+       struct lima_submit submit = {0};
+       int err = 0, size;
+
+       if (args->pipe >= lima_pipe_num || args->nr_bos == 0)
+               return -EINVAL;
+
+       if (args->flags & ~(LIMA_SUBMIT_FLAG_EXPLICIT_FENCE |
+                           LIMA_SUBMIT_FLAG_SYNC_FD_OUT))
+               return -EINVAL;
+
+       pipe = ldev->pipe + args->pipe;
+       if (args->frame_size != pipe->frame_size)
+               return -EINVAL;
+
+       size = args->nr_bos * (sizeof(*submit.bos) + sizeof(*submit.vbs)) +
+               args->nr_deps * sizeof(*submit.deps);
+       bos = kzalloc(size, GFP_KERNEL);
+       if (!bos)
+               return -ENOMEM;
+
+       size = args->nr_bos * sizeof(*submit.bos);
+       if (copy_from_user(bos, u64_to_user_ptr(args->bos), size)) {
+               err = -EFAULT;
+               goto out0;
+       }
+
+       vbs = (void *)bos + size;
+
+       if (args->nr_deps) {
+               deps = (void *)vbs + args->nr_bos * sizeof(*submit.vbs);
+               size = args->nr_deps * sizeof(*submit.deps);
+               if (copy_from_user(deps, u64_to_user_ptr(args->deps), size)) {
+                       err = -EFAULT;
+                       goto out0;
+               }
+       }
+
+       task = kmem_cache_zalloc(pipe->task_slab, GFP_KERNEL);
+       if (!task) {
+               err = -ENOMEM;
+               goto out0;
+       }
+
+       task->frame = task + 1;
+       if (copy_from_user(task->frame, u64_to_user_ptr(args->frame), args->frame_size)) {
+               err = -EFAULT;
+               goto out1;
+       }
+
+       err = pipe->task_validate(pipe, task);
+       if (err)
+               goto out1;
+
+       ctx = lima_ctx_get(&priv->ctx_mgr, args->ctx);
+       if (!ctx) {
+               err = -ENOENT;
+               goto out1;
+       }
+
+       submit.pipe = args->pipe;
+       submit.bos = bos;
+       submit.vbs = vbs;
+       submit.nr_bos = args->nr_bos;
+       submit.task = task;
+       submit.ctx = ctx;
+       submit.deps = deps;
+       submit.nr_deps = args->nr_deps;
+       submit.flags = args->flags;
+
+       err = lima_gem_submit(file, &submit);
+       if (!err) {
+               struct drm_lima_gem_submit_out *out = data;
+               out->fence = submit.fence;
+               out->done = submit.done;
+               out->sync_fd = submit.sync_fd;
+       }
+
+       lima_ctx_put(ctx);
+out1:
+       if (err)
+               kmem_cache_free(pipe->task_slab, task);
+out0:

drivers/gpu/drm/lima/lima_drv.c line 185

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+       kfree(bos);
+       return err;
+}
+
+static int lima_wait_fence(struct dma_fence *fence, u64 timeout_ns)
+{
+       signed long ret;
+
+       if (!timeout_ns)
+               ret = dma_fence_is_signaled(fence) ? 0 : -EBUSY;

I think you can just call dma_fence_wait_timeout with a 0 timeout below and remove this clause.

drivers/gpu/drm/lima/lima_drv.c line 197

Quoted 12 lines
drivers/gpu/drm/lima/lima_drv.c new file +12 -0
+       else {
+               unsigned long timeout = lima_timeout_to_jiffies(timeout_ns);
+
+               /* must use long for result check because in 64bit arch int
+                * will overflow if timeout is too large and get <0 result
+                */
+               ret = dma_fence_wait_timeout(fence, true, timeout);
+               if (ret == 0)
+                       ret = timeout ? -ETIMEDOUT : -EBUSY;
+               else if (ret > 0)
+                       ret = 0;
+       }

I suspect this could be common like reservation object waits. However, I'm curious why lima needs this ioctl in the first place when I don't see the same for other drivers.

Quoted 124 lines
drivers/gpu/drm/lima/lima_drv.c new file +124 -0
+
+       return ret;
+}
+
+static int lima_ioctl_wait_fence(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_wait_fence *args = data;
+       struct lima_drm_priv *priv = file->driver_priv;
+       struct dma_fence *fence;
+       int err = 0;
+
+       fence = lima_ctx_get_native_fence(&priv->ctx_mgr, args->ctx,
+                                         args->pipe, args->seq);
+       if (IS_ERR(fence))
+               return PTR_ERR(fence);
+
+       if (fence) {
+               err = lima_wait_fence(fence, args->timeout_ns);
+               args->error = fence->error;
+               dma_fence_put(fence);
+       }
+       else
+               args->error = 0;
+
+       return err;
+}
+
+static int lima_ioctl_gem_wait(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_wait *args = data;
+
+       if (!(args->op & (LIMA_GEM_WAIT_READ|LIMA_GEM_WAIT_WRITE)))
+           return -EINVAL;
+
+       return lima_gem_wait(file, args->handle, args->op, args->timeout_ns);
+}
+
+static int lima_ioctl_ctx(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_ctx *args = data;
+       struct lima_drm_priv *priv = file->driver_priv;
+       struct lima_device *ldev = to_lima_dev(dev);
+
+       if (args->op == LIMA_CTX_OP_CREATE)
+               return lima_ctx_create(ldev, &priv->ctx_mgr, &args->id);
+       else if (args->op == LIMA_CTX_OP_FREE)
+               return lima_ctx_free(&priv->ctx_mgr, args->id);
+
+       return -EINVAL;
+}
+
+static int lima_ioctl_gem_mod(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_mod *args = data;
+
+       if (args->op == LIMA_GEM_MOD_OP_GET)
+               return lima_gem_get_modifier(file, args->handle, &args->modifier);
+       else if (args->op == LIMA_GEM_MOD_OP_SET)
+               return lima_gem_set_modifier(file, args->handle, args->modifier);
+
+       return -EINVAL;
+}
+
+static int lima_drm_driver_open(struct drm_device *dev, struct drm_file *file)
+{
+       int err;
+       struct lima_drm_priv *priv;
+       struct lima_device *ldev = to_lima_dev(dev);
+
+       priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+       if (!priv)
+               return -ENOMEM;
+
+       priv->vm = lima_vm_create(ldev);
+       if (!priv->vm) {
+               err = -ENOMEM;
+               goto err_out0;
+       }
+
+        lima_ctx_mgr_init(&priv->ctx_mgr);
+
+       file->driver_priv = priv;
+       return 0;
+
+err_out0:
+       kfree(priv);
+       return err;
+}
+
+static void lima_drm_driver_postclose(struct drm_device *dev, struct drm_file *file)
+{
+       struct lima_drm_priv *priv = file->driver_priv;
+
+       lima_ctx_mgr_fini(&priv->ctx_mgr);
+       lima_vm_put(priv->vm);
+       kfree(priv);
+}
+
+static const struct drm_ioctl_desc lima_drm_driver_ioctls[] = {
+       DRM_IOCTL_DEF_DRV(LIMA_INFO, lima_ioctl_info, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_CREATE, lima_ioctl_gem_create, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_INFO, lima_ioctl_gem_info, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_VA, lima_ioctl_gem_va, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_SUBMIT, lima_ioctl_gem_submit, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_WAIT_FENCE, lima_ioctl_wait_fence, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_WAIT, lima_ioctl_gem_wait, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_CTX, lima_ioctl_ctx, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_MOD, lima_ioctl_gem_mod, DRM_AUTH|DRM_RENDER_ALLOW),
+};
+
+static const struct file_operations lima_drm_driver_fops = {
+       .owner              = THIS_MODULE,
+       .open               = drm_open,
+       .release            = drm_release,
+       .unlocked_ioctl     = drm_ioctl,
+#ifdef CONFIG_COMPAT
+       .compat_ioctl       = drm_compat_ioctl,
+#endif
+       .mmap               = lima_gem_mmap,
+};
+
+static struct drm_driver lima_drm_driver = {
+       .driver_features    = DRIVER_RENDER | DRIVER_GEM | DRIVER_PRIME,
+       .open               = lima_drm_driver_open,

drivers/gpu/drm/lima/lima_drv.c line 331

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+       .postclose          = lima_drm_driver_postclose,
+       .ioctls             = lima_drm_driver_ioctls,
+       .num_ioctls         = ARRAY_SIZE(lima_drm_driver_ioctls),
+       .fops               = &lima_drm_driver_fops,
+       .gem_free_object_unlocked = lima_gem_free_object,
+       .gem_open_object    = lima_gem_object_open,
+       .gem_close_object   = lima_gem_object_close,
+       .name               = "lima",
+       .desc               = "lima DRM",
+       .date               = "20170325",

Perhaps this should be updated? TBH, I don't know why this is even useful.

drivers/gpu/drm/lima/lima_drv.c line 340

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +9 -0
+       .major              = 1,
+       .minor              = 0,
+       .patchlevel         = 0,
+
+       .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
+       .gem_prime_import   = drm_gem_prime_import,
+       .gem_prime_import_sg_table = lima_gem_prime_import_sg_table,
+       .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
+       .gem_prime_export   = drm_gem_prime_export,

import and export don't have to be set if you use the defaults.

Quoted 26 lines
drivers/gpu/drm/lima/lima_drv.c new file +25 -0
+       .gem_prime_res_obj  = lima_gem_prime_res_obj,
+       .gem_prime_get_sg_table = lima_gem_prime_get_sg_table,
+       .gem_prime_vmap = lima_gem_prime_vmap,
+       .gem_prime_vunmap = lima_gem_prime_vunmap,
+       .gem_prime_mmap = lima_gem_prime_mmap,
+};
+
 
+int lima_gem_wait(struct drm_file *file, u32 handle, u32 op, u64 timeout_ns)
+{
+       bool write = op & LIMA_GEM_WAIT_WRITE;
+       struct drm_gem_object *obj;
+       struct lima_bo *bo;
+       signed long ret;
+       unsigned long timeout;
+
+       obj = drm_gem_object_lookup(file, handle);
+       if (!obj)
+               return -ENOENT;
+
+       bo = to_lima_bo(obj);
+
+       timeout = timeout_ns ? lima_timeout_to_jiffies(timeout_ns) : 0;
+
+       ret = lima_bo_reserve(bo, true);
+       if (ret)

drivers/gpu/drm/lima/lima_drv.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+               goto out;
+
+       /* must use long for result check because in 64bit arch int
+        * will overflow if timeout is too large and get <0 result
+        */
+       ret = reservation_object_wait_timeout_rcu(bo->tbo.resv, write, true, timeout);
+       if (ret == 0)
+               ret = timeout ? -ETIMEDOUT : -EBUSY;
+       else if (ret > 0)
+               ret = 0;

There's a helper I added for all this that should land in 5.1.

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +8 -0
+
+       lima_bo_unreserve(bo);
+out:
+       drm_gem_object_put_unlocked(obj);
+       return ret;
+}
+
 
+static int lima_gp_soft_reset_async_wait(struct lima_ip *ip)

drivers/gpu/drm/lima/lima_drv.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+{
+       struct lima_device *dev = ip->dev;
+       int timeout;
+
+       if (!ip->data.async_reset)
+               return 0;
+
+       for (timeout = 1000; timeout > 0; timeout--) {
+               if (gp_read(INT_RAWSTAT) & LIMA_GP_IRQ_RESET_COMPLETED)
+                       break;

Use readl_poll_timeout instead of writing your own. At least add a udelay to the loop so the timing is fixed and not dependent on how fast the code can run.

Quoted 30 lines
drivers/gpu/drm/lima/lima_drv.c new file +12 -0
+       }
+       if (!timeout) {
+               dev_err(dev->dev, "gp soft reset time out\n");
+               return -ETIMEDOUT;
+       }
+
+       gp_write(INT_CLEAR, LIMA_GP_IRQ_MASK_ALL);
+       gp_write(INT_MASK, LIMA_GP_IRQ_MASK_USED);
+
+       ip->data.async_reset = false;
+       return 0;
+}
drivers/gpu/drm/lima/lima_l2_cache.c new file +11 -0
new file mode 100644
index 000000000000..e7cdec720e5d
@@ -0,0 +1,79 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_l2_cache.h"
+#include "lima_regs.h"
+
+#define l2_cache_write(reg, data) writel(data, ip->iomem + LIMA_L2_CACHE_##reg)

drivers/gpu/drm/lima/lima_l2_cache.c line 21

Quoted 10 lines
drivers/gpu/drm/lima/lima_l2_cache.c new file +10 -0
+#define l2_cache_read(reg) readl(ip->iomem + LIMA_L2_CACHE_##reg)
+
+static int lima_l2_cache_wait_idle(struct lima_ip *ip)
+{
+       int timeout;
+       struct lima_device *dev = ip->dev;
+
+       for (timeout = 100000; timeout > 0; timeout--) {
+           if (!(l2_cache_read(STATUS) & LIMA_L2_CACHE_STATUS_COMMAND_BUSY))
+               break;

Use readl_poll_timeout or variant.

Quoted 115 lines
drivers/gpu/drm/lima/lima_l2_cache.c new file +58 -0
+       }
+       if (!timeout) {
+           dev_err(dev->dev, "l2 cache wait command timeout\n");
+           return -ETIMEDOUT;
+       }
+       return 0;
+}
+
+int lima_l2_cache_flush(struct lima_ip *ip)
+{
+       int ret;
+
+       spin_lock(&ip->data.lock);
+       l2_cache_write(COMMAND, LIMA_L2_CACHE_COMMAND_CLEAR_ALL);
+       ret = lima_l2_cache_wait_idle(ip);
+       spin_unlock(&ip->data.lock);
+       return ret;
+}
+
+int lima_l2_cache_init(struct lima_ip *ip)
+{
+       int i, err;
+       u32 size;
+       struct lima_device *dev = ip->dev;
+
+       /* l2_cache2 only exists when one of PP4-7 present */
+       if (ip->id == lima_ip_l2_cache2) {
+               for (i = lima_ip_pp4; i <= lima_ip_pp7; i++) {
+                       if (dev->ip[i].present)
+                               break;
+               }
+               if (i > lima_ip_pp7)
+                       return -ENODEV;
+       }
+
+       spin_lock_init(&ip->data.lock);
+
+       size = l2_cache_read(SIZE);
+       dev_info(dev->dev, "l2 cache %uK, %u-way, %ubyte cache line, %ubit external bus\n",
+                1 << (((size >> 16) & 0xff) - 10),
+                1 << ((size >> 8) & 0xff),
+                1 << (size & 0xff),
+                1 << ((size >> 24) & 0xff));
+
+       err = lima_l2_cache_flush(ip);
+       if (err)
+               return err;
+
+       l2_cache_write(ENABLE, LIMA_L2_CACHE_ENABLE_ACCESS | LIMA_L2_CACHE_ENABLE_READ_ALLOCATE);
+       l2_cache_write(MAX_READS, 0x1c);
+
+       return 0;
+}
+
+void lima_l2_cache_fini(struct lima_ip *ip)
+{
+
+}
drivers/gpu/drm/lima/lima_l2_cache.h new file +14 -0
new file mode 100644
index 000000000000..2ff91eafefbe
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_L2_CACHE_H__
+#define __LIMA_L2_CACHE_H__
+
+struct lima_ip;
+
+int lima_l2_cache_init(struct lima_ip *ip);
+void lima_l2_cache_fini(struct lima_ip *ip);
+
+int lima_l2_cache_flush(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_mmu.c new file +31 -0
new file mode 100644
index 000000000000..234fb90a4285
@@ -0,0 +1,135 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_mmu.h"
+#include "lima_vm.h"
+#include "lima_object.h"
+#include "lima_regs.h"
+
+#define mmu_write(reg, data) writel(data, ip->iomem + LIMA_MMU_##reg)
+#define mmu_read(reg) readl(ip->iomem + LIMA_MMU_##reg)
+
+#define lima_mmu_send_command(command, condition)           \
+({                                                          \
+       int __timeout, __ret = 0;                            \
+                                                            \
+       mmu_write(COMMAND, command);                         \
+       for (__timeout = 1000; __timeout > 0; __timeout--) { \
+               if (condition)                               \
+                       break;                               \
+       }                                                    \
+       if (!__timeout) {                                    \
+               dev_err(dev->dev, "mmu command %x timeout\n", command); \
+               __ret = -ETIMEDOUT;                          \
+       }                                                    \
+       __ret;                                               \
+})

drivers/gpu/drm/lima/lima_mmu.c line 41

Quoted 10 lines
drivers/gpu/drm/lima/lima_mmu.c new file +10 -0
+
+static irqreturn_t lima_mmu_irq_handler(int irq, void *data)
+{
+       struct lima_ip *ip = data;
+       struct lima_device *dev = ip->dev;
+       u32 status = mmu_read(INT_STATUS);
+       struct lima_sched_pipe *pipe;
+
+       /* for shared irq case */
+       if (!status)

Can status have masked irq's? If so, you should be masking out the disabled irq bits.

Quoted 17 lines
drivers/gpu/drm/lima/lima_mmu.c new file +17 -0
+               return IRQ_NONE;
+
+       if (status & LIMA_MMU_INT_PAGE_FAULT) {
+               u32 fault = mmu_read(PAGE_FAULT_ADDR);
+               dev_err(dev->dev, "mmu page fault at 0x%x from bus id %d of type %s on %s\n",
+                       fault, LIMA_MMU_STATUS_BUS_ID(status),
+                       status & LIMA_MMU_STATUS_PAGE_FAULT_IS_WRITE ? "write" : "read",
+                       lima_ip_name(ip));
+       }
+
+       if (status & LIMA_MMU_INT_READ_BUS_ERROR) {
+               dev_err(dev->dev, "mmu %s irq bus error\n", lima_ip_name(ip));
+       }
+
+       /* mask all interrupts before resume */
+       mmu_write(INT_MASK, 0);
+       mmu_write(INT_CLEAR, status);

drivers/gpu/drm/lima/lima_mmu.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_mmu.c new file +8 -0
+
+       pipe = dev->pipe + (ip->id == lima_ip_gpmmu ? lima_pipe_gp : lima_pipe_pp);
+       lima_sched_pipe_mmu_error(pipe);
+
+       return IRQ_HANDLED;
+}
 
 
+
+unsigned long lima_timeout_to_jiffies(u64 timeout_ns)

Create a common helper instead of copy-n-pasting this from other drivers (etnaviv).

Quoted 116 lines
drivers/gpu/drm/lima/lima_mmu.c new file +31 -0
+{
+       unsigned long timeout_jiffies;
+       ktime_t timeout;
+
+       /* clamp timeout if it's to large */
+       if (((s64)timeout_ns) < 0)
+               return MAX_SCHEDULE_TIMEOUT;
+
+       timeout = ktime_sub(ns_to_ktime(timeout_ns), ktime_get());
+       if (ktime_to_ns(timeout) < 0)
+               return 0;
+
+       timeout_jiffies = nsecs_to_jiffies(ktime_to_ns(timeout));
+       /*  clamp timeout to avoid unsigned-> signed overflow */
+       if (timeout_jiffies > MAX_SCHEDULE_TIMEOUT )
+               return MAX_SCHEDULE_TIMEOUT;
+
+       return timeout_jiffies;
+}
+
+void lima_sched_pipe_task_done(struct lima_sched_pipe *pipe)
+{
+       if (pipe->error)
+               schedule_work(&pipe->error_work);
+       else {
+               struct lima_sched_task *task = pipe->current_task;
+
+               pipe->task_fini(pipe);
+               dma_fence_signal(task->fence);
+       }
+}
drivers/gpu/drm/lima/lima_vm.c new file +78 -0
new file mode 100644
index 000000000000..a264f3ae83fe
@@ -0,0 +1,354 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/slab.h>
+#include <linux/dma-mapping.h>
+#include <linux/interval_tree_generic.h>
+
+#include "lima_device.h"
+#include "lima_vm.h"
+#include "lima_object.h"
+#include "lima_regs.h"
+
+struct lima_bo_va_mapping {
+       struct list_head list;
+       struct rb_node rb;
+       uint32_t start;
+       uint32_t last;
+       uint32_t __subtree_last;
+};
+
+struct lima_bo_va {
+       struct list_head list;
+       unsigned ref_count;
+
+       struct list_head mapping;
+
+       struct lima_vm *vm;
+};
+
+#define LIMA_VM_PD_SHIFT 22
+#define LIMA_VM_PT_SHIFT 12
+#define LIMA_VM_PB_SHIFT (LIMA_VM_PD_SHIFT + LIMA_VM_NUM_PT_PER_BT_SHIFT)
+#define LIMA_VM_BT_SHIFT LIMA_VM_PT_SHIFT
+
+#define LIMA_VM_PT_MASK ((1 << LIMA_VM_PD_SHIFT) - 1)
+#define LIMA_VM_BT_MASK ((1 << LIMA_VM_PB_SHIFT) - 1)
+
+#define LIMA_PDE(va) (va >> LIMA_VM_PD_SHIFT)
+#define LIMA_PTE(va) ((va & LIMA_VM_PT_MASK) >> LIMA_VM_PT_SHIFT)
+#define LIMA_PBE(va) (va >> LIMA_VM_PB_SHIFT)
+#define LIMA_BTE(va) ((va & LIMA_VM_BT_MASK) >> LIMA_VM_BT_SHIFT)
+
+#define START(node) ((node)->start)
+#define LAST(node) ((node)->last)
+
+INTERVAL_TREE_DEFINE(struct lima_bo_va_mapping, rb, uint32_t, __subtree_last,
+                    START, LAST, static, lima_vm_it)
+
+#undef START
+#undef LAST
+
+static void lima_vm_unmap_page_table(struct lima_vm *vm, u32 start, u32 end)
+{
+       u32 addr;
+
+       for (addr = start; addr <= end; addr += LIMA_PAGE_SIZE) {
+               u32 pbe = LIMA_PBE(addr);
+               u32 bte = LIMA_BTE(addr);
+               u32 *bt;
+
+               bt = lima_bo_kmap(vm->bts[pbe]);
+               bt[bte] = 0;
+       }
+}
+
+static int lima_vm_map_page_table(struct lima_vm *vm, dma_addr_t *dma,
+                                 u32 start, u32 end)
+{
+       u64 addr;
+       int err, i = 0;
+
+       for (addr = start; addr <= end; addr += LIMA_PAGE_SIZE) {
+               u32 pbe = LIMA_PBE(addr);
+               u32 bte = LIMA_BTE(addr);
+               u32 *bt;
+
+               if (vm->bts[pbe])
+                       bt = lima_bo_kmap(vm->bts[pbe]);

drivers/gpu/drm/lima/lima_vm.c line 88

Quoted 10 lines
drivers/gpu/drm/lima/lima_vm.c new file +10 -0
+               else {
+                       struct lima_bo *bt_bo;
+                       dma_addr_t *pts;
+                       u32 *pd;
+                       int j;
+
+                       bt_bo = lima_bo_create(
+                               vm->dev, LIMA_PAGE_SIZE << LIMA_VM_NUM_PT_PER_BT_SHIFT,
+                               0, ttm_bo_type_kernel,
+                               NULL, vm->pd->tbo.resv);

I don't think using BOs for page tables buys you anything. You could just use the kernel DMA API directly. See io-pgtable-arm-v7s.c for inspiration. For panfrost, it's standard ARM format page tables so we can just use the io-pgtable library.

Quoted 57 lines
drivers/gpu/drm/lima/lima_vm.c new file +57 -0
+                       if (IS_ERR(bt_bo)) {
+                               err = PTR_ERR(bt_bo);
+                               goto err_out;
+                       }
+
+                       bt = lima_bo_kmap(bt_bo);
+                       if (IS_ERR(bt)) {
+                               lima_bo_unref(bt_bo);
+                               err = PTR_ERR(bt);
+                               goto err_out;
+                       }
+                       memset(bt, 0, LIMA_PAGE_SIZE << LIMA_VM_NUM_PT_PER_BT_SHIFT);
+
+                       vm->bts[pbe] = bt_bo;
+                       pd = lima_bo_kmap(vm->pd);
+                       pd += pbe << LIMA_VM_NUM_PT_PER_BT_SHIFT;
+                       pts = lima_bo_get_pages(bt_bo);
+                       for (j = 0; j < LIMA_VM_NUM_PT_PER_BT; j++)
+                               *pd++ = *pts++ | LIMA_VM_FLAG_PRESENT;
+               }
+
+               bt[bte] = dma[i++] | LIMA_VM_FLAGS_CACHE;
+       }
+
+       return 0;
+
+err_out:
+       if (addr != start)
+               lima_vm_unmap_page_table(vm, start, addr - 1);
+       return err;
+}
+
+static struct lima_bo_va *
+lima_vm_bo_find(struct lima_vm *vm, struct lima_bo *bo)
+{
+       struct lima_bo_va *bo_va, *ret = NULL;
+
+       list_for_each_entry(bo_va, &bo->va, list) {
+               if (bo_va->vm == vm) {
+                       ret = bo_va;
+                       break;
+               }
+       }
+
+       return ret;
+}
+
+int lima_vm_bo_map(struct lima_vm *vm, struct lima_bo *bo, u32 start)
+{
+       int err;
+       struct lima_bo_va_mapping *it, *mapping;
+       u32 end = start + bo->gem.size - 1;
+       dma_addr_t *pages_dma = lima_bo_get_pages(bo);
+       struct lima_bo_va *bo_va;
+
+       it = lima_vm_it_iter_first(&vm->va, start, end);
+       if (it) {

drivers/gpu/drm/lima/lima_vm.c line 155

Quoted 10 lines
drivers/gpu/drm/lima/lima_vm.c new file +10 -0
+               dev_dbg(bo->gem.dev->dev, "lima vm map va overlap %x-%x %x-%x\n",
+                       start, end, it->start, it->last);
+               return -EINVAL;
+       }
+
+       mapping = kmalloc(sizeof(*mapping), GFP_KERNEL);
+       if (!mapping)
+               return -ENOMEM;
+       mapping->start = start;
+       mapping->last = end;

Why don't you use the drm_mm_XX APIs instead of writing your own?

Quoted 10 lines
drivers/gpu/drm/lima/lima_vm.c new file +10 -0
+
+       err = lima_vm_map_page_table(vm, pages_dma, start, end);
+       if (err) {
+               kfree(mapping);
+               return err;
+       }
+
+       lima_vm_it_insert(mapping, &vm->va);
+
+       bo_va = lima_vm_bo_find(vm, bo);
Quoted 8 lines

drivers/gpu/drm/lima/lima_vm.c line 169

Quoted 4 lines
drivers/gpu/drm/lima/lima_vm.c new file +4 -0
+       list_add_tail(&mapping->list, &bo_va->mapping);
+
+       return 0;
+}

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#12 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 7 lines · Alex Deucher
Quoted 4 lines · Rob Herring via dri-devel
Quoted 2 lines · Qiang Yu

From: Lima Project Developers <[email protected]>

[...]

Quoted 20 lines
Quoted 14 lines

drivers/gpu/drm/lima/lima_drv.c line 77

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

Quoted 3 lines

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

With any new driver in the kernel, the question is always which existing one is the best model to follow. I don't think Intel, AMD or Nouveau are good examples to follow because they have a lot of history and legacy, are both GPU and DC, and have separate graphics memory (except Intel I guess). The GPUs in ARM land have none of those really. Looking thru freedreno, etnaviv, and v3d mostly, I see they all have similar user ABIs. But they are all different based on what driver they copied and how they've evolved. I know it's a big can of worms, but it would be nice to have some alignment of ABIs. I know the reasons why there isn't, but it's frustrating that 11 out of 60K IGT tests will run. I don't think a common ABI matters much for the big 3, but in the ARM zoo I think it does. At least if the interfaces are kept similar, then having common code shared among the embedded GPUs would be easier and writing some IGT shim for each driver would be easier.

Rob
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#13 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 43 lines · Rob Herring
Quoted 7 lines · Alex Deucher
Quoted 4 lines · Rob Herring via dri-devel
Quoted 2 lines · Qiang Yu

From: Lima Project Developers <[email protected]>

[...]

Quoted 20 lines
Quoted 14 lines

drivers/gpu/drm/lima/lima_drv.c line 77

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

Quoted 3 lines

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

Quoted 15 lines

With any new driver in the kernel, the question is always which existing one is the best model to follow. I don't think Intel, AMD or Nouveau are good examples to follow because they have a lot of history and legacy, are both GPU and DC, and have separate graphics memory (except Intel I guess). The GPUs in ARM land have none of those really. Looking thru freedreno, etnaviv, and v3d mostly, I see they all have similar user ABIs. But they are all different based on what driver they copied and how they've evolved. I know it's a big can of worms, but it would be nice to have some alignment of ABIs. I know the reasons why there isn't, but it's frustrating that 11 out of 60K IGT tests will run. I don't think a common ABI matters much for the big 3, but in the ARM zoo I think it does. At least if the interfaces are kept similar, then having common code shared among the embedded GPUs would be easier and writing some IGT shim for each driver would be easier.

I admit the userspace VA management is not suitable and overfit to GPU like Mali4xx which does not have complicated SW/HW like AMD GPU. It needs some extra effort to make it work but bings no seen benefit to lima driver.

So I do VA map when BO creation in v2 now. And you may find the v2 driver is closer to the ARM world GPU driver which also uses GEM+shmem way instead of TTM:
https://gitlab.freedesktop.org/lima/linux/commits/lima-5.0-rc6

But due to the change is huge, I'll send it here for review latter this week for more testing and address other of your review comments.

Thanks,
Qiang

Quoted 3 lines

Rob

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#14 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

In reply to Rob Herring via dri-devel

I typed this up yesterday, but it looks like I never hit send.

Quoted 36 lines · Rob Herring
Quoted 7 lines · Alex Deucher
Quoted 4 lines · Rob Herring via dri-devel
Quoted 2 lines · Qiang Yu

From: Lima Project Developers <[email protected]>

[...]

Quoted 20 lines
Quoted 14 lines

drivers/gpu/drm/lima/lima_drv.c line 77

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

Quoted 3 lines

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Quoted 6 lines

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

Having it in userspace provides a lot more flexibility and makes it easier to support things like unified address space between CPU and GPU. I guess it depends on the hw as to what is the right choice.

Quoted 16 lines

With any new driver in the kernel, the question is always which existing one is the best model to follow. I don't think Intel, AMD or Nouveau are good examples to follow because they have a lot of history and legacy, are both GPU and DC, and have separate graphics memory (except Intel I guess). The GPUs in ARM land have none of those really. Looking thru freedreno, etnaviv, and v3d mostly, I see they all have similar user ABIs. But they are all different based on what driver they copied and how they've evolved. I know it's a big can of worms, but it would be nice to have some alignment of ABIs. I know the reasons why there isn't, but it's frustrating that 11 out of 60K IGT tests will run. I don't think a common ABI matters much for the big 3, but in the ARM zoo I think it does. At least if the interfaces are kept similar, then having common code shared among the embedded GPUs would be easier and writing some IGT shim for each driver would be easier.

It also depends on what you are familiar with. Qiang is familiar with amdgpu so it makes sense to use that as a model for his own projects. I don't know that similarity between drivers for hw that is mostly part of ARM SoCs is any more important than the applicability to the hw or what the maintainer likes. I mean you can use amdgpu or nouveau on ARM too.

Alex
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#15 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Am 14.02.19 um 03:52 schrieb Alex Deucher via dri-devel:

Quoted 28 lines

[SNIP]

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines

drivers/gpu/drm/lima/lima_drv.c line 77

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 8 lines
Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Quoted 6 lines

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

Quoted 3 lines

Having it in userspace provides a lot more flexibility and makes it easier to support things like unified address space between CPU and GPU. I guess it depends on the hw as to what is the right choice.

To summarize we actually have tried this approach with the radeon and it turned out to be a really bad mistake.

To implement features like partial residential textures and shared virtual address space you absolutely need userspace to be in charge of allocating virtual addresses.

Regards,
Christian.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#16 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 34 lines · Christian König via dri-devel

Am 14.02.19 um 03:52 schrieb Alex Deucher via dri-devel:

Quoted 28 lines

[SNIP]

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines

drivers/gpu/drm/lima/lima_drv.c line 77

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 8 lines
Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Quoted 6 lines

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

Quoted 3 lines

Having it in userspace provides a lot more flexibility and makes it easier to support things like unified address space between CPU and GPU. I guess it depends on the hw as to what is the right choice.

Quoted 6 lines

To summarize we actually have tried this approach with the radeon and it turned out to be a really bad mistake.

To implement features like partial residential textures and shared virtual address space you absolutely need userspace to be in charge of allocating virtual addresses.

Yeah same here, as soon as you have per-process address spaces you want your userspace to control where buffers are placed. All new intel drivers use softpin to control the layout fully (anv and iris). Of course if you also have hw without per-process virtual address space on the gpu in some form, then the kernel needs to assign addresses, which means lots of relocs. i965_dri.so still works like that, even with the rewritten buffer/batch manager. But I'd really only do that if you can't avoid it.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
maintainer
#17 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

In reply to Christian König via dri-devel
Quoted 34 lines · Christian König via dri-devel

Am 14.02.19 um 03:52 schrieb Alex Deucher via dri-devel:

Quoted 28 lines

[SNIP]

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines

drivers/gpu/drm/lima/lima_drv.c line 77

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 8 lines
Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Quoted 6 lines

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

Quoted 3 lines

Having it in userspace provides a lot more flexibility and makes it easier to support things like unified address space between CPU and GPU. I guess it depends on the hw as to what is the right choice.

Quoted 7 lines

To summarize we actually have tried this approach with the radeon and it turned out to be a really bad mistake.

To implement features like partial residential textures and shared virtual address space you absolutely need userspace to be in charge of allocating virtual addresses.

I think for lima not having this is fine, but for panfrost it really should have it.

If you can implement vulkan you probably want this, nouveau hasn't a vulkan driver because of exactly this problem in their uapi, so maybe adjust panfrost to do user-space managed vma.

Dave.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#18 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 45 lines · Dave Airlie
Quoted 34 lines · Christian König via dri-devel

Am 14.02.19 um 03:52 schrieb Alex Deucher via dri-devel:

Quoted 28 lines

[SNIP]

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines

Review

Quoted 9 lines
(quoted diff) +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 8 lines
Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Quoted 6 lines

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

Quoted 3 lines

Having it in userspace provides a lot more flexibility and makes it easier to support things like unified address space between CPU and GPU. I guess it depends on the hw as to what is the right choice.

Quoted 7 lines

To summarize we actually have tried this approach with the radeon and it turned out to be a really bad mistake.

To implement features like partial residential textures and shared virtual address space you absolutely need userspace to be in charge of allocating virtual addresses.

Quoted 6 lines

I think for lima not having this is fine, but for panfrost it really should have it.

If you can implement vulkan you probably want this, nouveau hasn't a vulkan driver because of exactly this problem in their uapi, so maybe adjust panfrost to do user-space managed vma.

Wouldn't this just require an allocation flag to not map the BO up front and then new ioctl's like above to map and unmap at specified VAs? Seems like we could add that when we get there.

Rob
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#19 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 49 lines · Rob Herring
Quoted 48 lines · Dave Airlie
Quoted 34 lines · Christian König via dri-devel

Am 14.02.19 um 03:52 schrieb Alex Deucher via dri-devel:

Quoted 28 lines

[SNIP]

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines

Review

Quoted 9 lines
(quoted diff) +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 8 lines
Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Quoted 6 lines

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

Quoted 3 lines

Having it in userspace provides a lot more flexibility and makes it easier to support things like unified address space between CPU and GPU. I guess it depends on the hw as to what is the right choice.

Quoted 7 lines

To summarize we actually have tried this approach with the radeon and it turned out to be a really bad mistake.

To implement features like partial residential textures and shared virtual address space you absolutely need userspace to be in charge of allocating virtual addresses.

I think for lima not having this is fine, but for panfrost it really should have it.

Quoted 7 lines
Quoted 3 lines

If you can implement vulkan you probably want this, nouveau hasn't a vulkan driver because of exactly this problem in their uapi, so maybe adjust panfrost to do user-space managed vma.

Wouldn't this just require an allocation flag to not map the BO up front and then new ioctl's like above to map and unmap at specified VAs? Seems like we could add that when we get there.

Sounds pretty reasonable to me.

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#20 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Am 14.03.19 um 23:28 schrieb Eric Anholt:

Quoted 47 lines
Quoted 45 lines · Rob Herring
Quoted 44 lines · Dave Airlie
Quoted 32 lines · Christian König via dri-devel

Am 14.02.19 um 03:52 schrieb Alex Deucher via dri-devel:

Quoted 28 lines

[SNIP]

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines

Review

Quoted 9 lines
(quoted diff) +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 8 lines
Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Quoted 6 lines

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

Quoted 3 lines

Having it in userspace provides a lot more flexibility and makes it easier to support things like unified address space between CPU and GPU. I guess it depends on the hw as to what is the right choice.

Quoted 7 lines

To summarize we actually have tried this approach with the radeon and it turned out to be a really bad mistake.

To implement features like partial residential textures and shared virtual address space you absolutely need userspace to be in charge of allocating virtual addresses.

I think for lima not having this is fine, but for panfrost it really should have it.

Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

If you can implement vulkan you probably want this, nouveau hasn't a vulkan driver because of exactly this problem in their uapi, so maybe adjust panfrost to do user-space managed vma.

Wouldn't this just require an allocation flag to not map the BO up front and then new ioctl's like above to map and unmap at specified VAs? Seems like we could add that when we get there.

Sounds pretty reasonable to me.

I can only advise to NOT do this.

A address space manager in userspace is rather easily doable, but fixing up UAPI without breaking existing applications isn't.

This is also one of the reasons why we don't support Vulkan with radeon on SI hardware generation.

So you got at least two drivers which have gone down this route and regret it now and I can perfectly understand if Dave says that any new DRM driver should not make the same mistake again.

Regards,
Christian.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#21 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 54 lines · Christian

Am 14.03.19 um 23:28 schrieb Eric Anholt:

Quoted 53 lines
Quoted 51 lines · Rob Herring
Quoted 41 lines · Dave Airlie
Quoted 32 lines · Christian König via dri-devel

Am 14.02.19 um 03:52 schrieb Alex Deucher via dri-devel:

Quoted 28 lines

[SNIP]

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines

Review

Quoted 9 lines
(quoted diff) +9 -0
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

Quoted 8 lines
Quoted 5 lines

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

I think it would be a good idea to look at the amdgpu driver. This driver is heavily modeled after it. Basically the GEM VA ioctl allows userspace to manage per process (per fd really) virtual addresses.

Quoted 6 lines

Why do you want userspace to manage assigning VAs versus the kernel to do so? Exposing that detail to userspace means the driver must support a per process address space. Letting the kernel assign addresses means it can either be a single address space or be a per process address space. It seems to me more flexible to allow the kernel driver to evolve without that ABI.

Quoted 3 lines

Having it in userspace provides a lot more flexibility and makes it easier to support things like unified address space between CPU and GPU. I guess it depends on the hw as to what is the right choice.

Quoted 7 lines

To summarize we actually have tried this approach with the radeon and it turned out to be a really bad mistake.

To implement features like partial residential textures and shared virtual address space you absolutely need userspace to be in charge of allocating virtual addresses.

Quoted 6 lines

I think for lima not having this is fine, but for panfrost it really should have it.

If you can implement vulkan you probably want this, nouveau hasn't a vulkan driver because of exactly this problem in their uapi, so maybe adjust panfrost to do user-space managed vma.

Wouldn't this just require an allocation flag to not map the BO up front and then new ioctl's like above to map and unmap at specified VAs? Seems like we could add that when we get there.

Quoted 6 lines
Quoted 1 line

Sounds pretty reasonable to me.

I can only advise to NOT do this.

A address space manager in userspace is rather easily doable, but fixing up UAPI without breaking existing applications isn't.

Can you expand on what goes wrong with Rob's idea? The only thing I can come up with is maybe dmabuf imports don't have a flag for don't-automatically-map?

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#22 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Am 15.03.19 um 17:05 schrieb Eric Anholt:

Quoted 10 lines

[SNIP]

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines

I think for lima not having this is fine, but for panfrost it really should have it.

If you can implement vulkan you probably want this, nouveau hasn't a vulkan driver because of exactly this problem in their uapi, so maybe adjust panfrost to do user-space managed vma.

Wouldn't this just require an allocation flag to not map the BO up front and then new ioctl's like above to map and unmap at specified VAs? Seems like we could add that when we get there.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Sounds pretty reasonable to me.

I can only advise to NOT do this.

A address space manager in userspace is rather easily doable, but fixing up UAPI without breaking existing applications isn't.

Can you expand on what goes wrong with Rob's idea? The only thing I can come up with is maybe dmabuf imports don't have a flag for don't-automatically-map?

Suppressing automatically mapping is not the problem. The problem is that you need exactly one instance to have the authority over the address space management.

With Vulkan and features like PRT this must be done by userspace. So can't just go ahead in the kernel and allocate some address space and map your buffers.

You can of course hope that all client respect this and you don't have combinations of multiple clients in the same process, but from experience I can say that this won't work at all.

Regards,
Christian.

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#23 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

In reply to Rob Herring via dri-devel
Quoted 76 lines · Rob Herring
Quoted 2 lines · Qiang Yu

From: Lima Project Developers <[email protected]>

This should be a person (you).

Quoted 6 lines

Being the submitter, your S-o-b should be last.

Quoted 22 lines

---

drivers/gpu/drm/Kconfig +2 -0
index 4385f00e1d05..dfefcb393858 100644
@@ -333,6 +333,8 @@ source "drivers/gpu/drm/tve200/Kconfig"
 
 source "drivers/gpu/drm/xen/Kconfig"
 
+source "drivers/gpu/drm/lima/Kconfig"
+
 # Keep legacy drivers last
 
 menuconfig DRM_LEGACY
drivers/gpu/drm/Makefile +0 -0
index ce8d1d384319..8d024b729902 100644

drivers/gpu/drm/Makefile obj-$(CONFIG_DRM_TINYDRM) += tinydrm/ · line 112

Quoted 4 lines
drivers/gpu/drm/Makefile +1 -0
@@ -109,3 +109,4 @@ obj-$(CONFIG_DRM_TINYDRM) += tinydrm/
 obj-$(CONFIG_DRM_PL111) += pl111/
 obj-$(CONFIG_DRM_TVE200) += tve200/
 obj-$(CONFIG_DRM_XEN) += xen/
+obj-$(CONFIG_DRM_LIMA)  += lima/

Not sure about this file, but normally these should be kept sorted.

Quoted 8 lines
drivers/gpu/drm/lima/lima_bcast.c new file +2 -0
new file mode 100644
index 000000000000..63754f6465ea
@@ -0,0 +1,46 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2018 Qiang Yu <[email protected]> */

drivers/gpu/drm/lima/lima_bcast.c line 12

Quoted 10 lines
drivers/gpu/drm/lima/lima_bcast.c new file +10 -0
+
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_bcast.h"
+#include "lima_regs.h"
+
+#define bcast_write(reg, data) writel(data, ip->iomem + LIMA_BCAST_##reg)
+#define bcast_read(reg) readl(ip->iomem + LIMA_BCAST_##reg)

There are 2 things about this I would change. Just pass in 'ip' to the macro so it is clear in calling functions that ip is actually used. Second, don't do token pasting. It is generally avoided in the kernel. It makes grepping the source code harder and is a pointless indirection.

If you do both of those, then these can be static inline functions instead which are preferred because you get type checking.

Same comment applies to all the other register accessors.

Quoted 16 lines

drivers/gpu/drm/lima/lima_bcast.c line 25

Quoted 13 lines
drivers/gpu/drm/lima/lima_bcast.c new file +13 -0
+struct lima_ip {
+       struct lima_device *dev;
+       enum lima_ip_id id;
+       bool present;
+
+       void __iomem *iomem;
+       int irq;
+
+       union {
+               /* gp/pp */
+               bool async_reset;
+               /* l2 cache */
+               spinlock_t lock;

What happens when you need 2 elements for a sub-block. I'd make this a struct pointer for each IP sub-block.

Let's use sub-block latter if there comes second element.

Quoted 360 lines
Quoted 214 lines
drivers/gpu/drm/lima/lima_bcast.c new file +54 -0
+       } data;
+};
+
+enum lima_pipe_id {
+       lima_pipe_gp,
+       lima_pipe_pp,
+       lima_pipe_num,
+};
+
+struct lima_device {
+       struct device *dev;
+       struct drm_device *ddev;
+       struct platform_device *pdev;
+
+       enum lima_gpu_id id;
+       int num_pp;
+
+       void __iomem *iomem;
+       struct clk *clk_bus;
+       struct clk *clk_gpu;
+       struct reset_control *reset;
+       struct regulator *regulator;
+
+       struct lima_ip ip[lima_ip_num];
+       struct lima_sched_pipe pipe[lima_pipe_num];
+
+       struct lima_mman mman;
+
+       struct lima_vm *empty_vm;
+       uint64_t va_start;
+       uint64_t va_end;
+
+       u32 *dlbu_cpu;
+       dma_addr_t dlbu_dma;
+};
+
+static inline struct lima_device *
+to_lima_dev(struct drm_device *dev)
+{
+       return dev->dev_private;
+}
+
+static inline struct lima_device *
+ttm_to_lima_dev(struct ttm_bo_device *dev)
+{
+       return container_of(dev, struct lima_device, mman.bdev);
+}
+
+int lima_device_init(struct lima_device *ldev);
+void lima_device_fini(struct lima_device *ldev);
+
+const char *lima_ip_name(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_dlbu.c new file +56 -0
new file mode 100644
index 000000000000..6697d4ddd887
@@ -0,0 +1,56 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_dlbu.h"
+#include "lima_vm.h"
+#include "lima_regs.h"
+
+#define dlbu_write(reg, data) writel(data, ip->iomem + LIMA_DLBU_##reg)
+#define dlbu_read(reg) readl(ip->iomem + LIMA_DLBU_##reg)
+
+void lima_dlbu_enable(struct lima_device *dev, int num_pp)
+{
+       struct lima_sched_pipe *pipe = dev->pipe + lima_pipe_pp;
+       struct lima_ip *ip = dev->ip + lima_ip_dlbu;
+       int i, mask = 0;
+
+       for (i = 0; i < num_pp; i++) {
+               struct lima_ip *pp = pipe->processor[i];
+               mask |= 1 << (pp->id - lima_ip_pp0);
+       }
+
+       dlbu_write(PP_ENABLE_MASK, mask);
+}
+
+void lima_dlbu_disable(struct lima_device *dev)
+{
+       struct lima_ip *ip = dev->ip + lima_ip_dlbu;
+       dlbu_write(PP_ENABLE_MASK, 0);
+}
+
+void lima_dlbu_set_reg(struct lima_ip *ip, u32 *reg)
+{
+       dlbu_write(TLLIST_VBASEADDR, reg[0]);
+       dlbu_write(FB_DIM, reg[1]);
+       dlbu_write(TLLIST_CONF, reg[2]);
+       dlbu_write(START_TILE_POS, reg[3]);
+}
+
+int lima_dlbu_init(struct lima_ip *ip)
+{
+       struct lima_device *dev = ip->dev;
+
+       dlbu_write(MASTER_TLLIST_PHYS_ADDR, dev->dlbu_dma | 1);
+       dlbu_write(MASTER_TLLIST_VADDR, LIMA_VA_RESERVE_DLBU);
+
+       return 0;
+}
+
+void lima_dlbu_fini(struct lima_ip *ip)
+{
+
+}
drivers/gpu/drm/lima/lima_dlbu.h new file +18 -0
new file mode 100644
index 000000000000..60cba387cf30
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_DLBU_H__
+#define __LIMA_DLBU_H__
+
+struct lima_ip;
+struct lima_device;
+
+void lima_dlbu_enable(struct lima_device *dev, int num_pp);
+void lima_dlbu_disable(struct lima_device *dev);
+
+void lima_dlbu_set_reg(struct lima_ip *ip, u32 *reg);
+
+int lima_dlbu_init(struct lima_ip *ip);
+void lima_dlbu_fini(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_drv.c new file +67 -0
new file mode 100644
index 000000000000..132071b9be9b
@@ -0,0 +1,459 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/log2.h>
+#include <drm/drm_prime.h>
+#include <drm/lima_drm.h>
+
+#include "lima_drv.h"
+#include "lima_gem.h"
+#include "lima_gem_prime.h"
+#include "lima_vm.h"
+
+int lima_sched_timeout_ms = 0;
+int lima_sched_max_tasks = 32;
+int lima_max_mem = -1;
+
+MODULE_PARM_DESC(sched_timeout_ms, "task run timeout in ms (0 = no timeout (default))");
+module_param_named(sched_timeout_ms, lima_sched_timeout_ms, int, 0444);
+
+MODULE_PARM_DESC(sched_max_tasks, "max queued task num in a context (default 32)");
+module_param_named(sched_max_tasks, lima_sched_max_tasks, int, 0444);
+
+MODULE_PARM_DESC(max_mem, "Max memory size in MB can be used (<0 = auto)");
+module_param_named(max_mem, lima_max_mem, int, 0444);
+
+static int lima_ioctl_info(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_info *info = data;
+       struct lima_device *ldev = to_lima_dev(dev);
+
+       switch (ldev->id) {
+       case lima_gpu_mali400:
+               info->gpu_id = LIMA_INFO_GPU_MALI400;
+               break;
+       case lima_gpu_mali450:
+               info->gpu_id = LIMA_INFO_GPU_MALI450;
+               break;
+       default:
+               return -ENODEV;
+       }
+       info->num_pp = ldev->pipe[lima_pipe_pp].num_processor;
+       info->va_start = ldev->va_start;
+       info->va_end = ldev->va_end;
+       return 0;
+}
+
+static int lima_ioctl_gem_create(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_create *args = data;
+
+       if (args->flags)
+               return -EINVAL;
+
+       if (args->size == 0)
+               return -EINVAL;
+
+       return lima_gem_create_handle(dev, file, args->size, args->flags, &args->handle);
+}
+
+static int lima_ioctl_gem_info(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_info *args = data;
+
+       return lima_gem_mmap_offset(file, args->handle, &args->offset);
+}

drivers/gpu/drm/lima/lima_drv.c line 77

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+
+static int lima_ioctl_gem_va(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_va *args = data;
+
+       switch (args->op) {
+       case LIMA_VA_OP_MAP:
+               return lima_gem_va_map(file, args->handle, args->flags, args->va);
+       case LIMA_VA_OP_UNMAP:
+               return lima_gem_va_unmap(file, args->handle, args->va);

These are mapping to GPU VA. Why not do that on GEM object creation or import or when the objects are submitted with cmd queue as other drivers do?

To put it another way, These ioctls look different than what other drivers do. Why do you need to do things differently? My understanding is best practice is to map and return the GPU offset when the GEM object is created. This is what v3d does. I think Intel is moving to that. And panfrost will do that.

Quoted 98 lines
drivers/gpu/drm/lima/lima_drv.c new file +98 -0
+       default:
+               return -EINVAL;
+       }
+}
+
+static int lima_ioctl_gem_submit(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_submit_in *args = data;
+       struct lima_device *ldev = to_lima_dev(dev);
+       struct lima_drm_priv *priv = file->driver_priv;
+       struct drm_lima_gem_submit_bo *bos;
+       struct ttm_validate_buffer *vbs;
+       union drm_lima_gem_submit_dep *deps = NULL;
+       struct lima_sched_pipe *pipe;
+       struct lima_sched_task *task;
+       struct lima_ctx *ctx;
+       struct lima_submit submit = {0};
+       int err = 0, size;
+
+       if (args->pipe >= lima_pipe_num || args->nr_bos == 0)
+               return -EINVAL;
+
+       if (args->flags & ~(LIMA_SUBMIT_FLAG_EXPLICIT_FENCE |
+                           LIMA_SUBMIT_FLAG_SYNC_FD_OUT))
+               return -EINVAL;
+
+       pipe = ldev->pipe + args->pipe;
+       if (args->frame_size != pipe->frame_size)
+               return -EINVAL;
+
+       size = args->nr_bos * (sizeof(*submit.bos) + sizeof(*submit.vbs)) +
+               args->nr_deps * sizeof(*submit.deps);
+       bos = kzalloc(size, GFP_KERNEL);
+       if (!bos)
+               return -ENOMEM;
+
+       size = args->nr_bos * sizeof(*submit.bos);
+       if (copy_from_user(bos, u64_to_user_ptr(args->bos), size)) {
+               err = -EFAULT;
+               goto out0;
+       }
+
+       vbs = (void *)bos + size;
+
+       if (args->nr_deps) {
+               deps = (void *)vbs + args->nr_bos * sizeof(*submit.vbs);
+               size = args->nr_deps * sizeof(*submit.deps);
+               if (copy_from_user(deps, u64_to_user_ptr(args->deps), size)) {
+                       err = -EFAULT;
+                       goto out0;
+               }
+       }
+
+       task = kmem_cache_zalloc(pipe->task_slab, GFP_KERNEL);
+       if (!task) {
+               err = -ENOMEM;
+               goto out0;
+       }
+
+       task->frame = task + 1;
+       if (copy_from_user(task->frame, u64_to_user_ptr(args->frame), args->frame_size)) {
+               err = -EFAULT;
+               goto out1;
+       }
+
+       err = pipe->task_validate(pipe, task);
+       if (err)
+               goto out1;
+
+       ctx = lima_ctx_get(&priv->ctx_mgr, args->ctx);
+       if (!ctx) {
+               err = -ENOENT;
+               goto out1;
+       }
+
+       submit.pipe = args->pipe;
+       submit.bos = bos;
+       submit.vbs = vbs;
+       submit.nr_bos = args->nr_bos;
+       submit.task = task;
+       submit.ctx = ctx;
+       submit.deps = deps;
+       submit.nr_deps = args->nr_deps;
+       submit.flags = args->flags;
+
+       err = lima_gem_submit(file, &submit);
+       if (!err) {
+               struct drm_lima_gem_submit_out *out = data;
+               out->fence = submit.fence;
+               out->done = submit.done;
+               out->sync_fd = submit.sync_fd;
+       }
+
+       lima_ctx_put(ctx);
+out1:
+       if (err)
+               kmem_cache_free(pipe->task_slab, task);
+out0:

drivers/gpu/drm/lima/lima_drv.c line 185

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+       kfree(bos);
+       return err;
+}
+
+static int lima_wait_fence(struct dma_fence *fence, u64 timeout_ns)
+{
+       signed long ret;
+
+       if (!timeout_ns)
+               ret = dma_fence_is_signaled(fence) ? 0 : -EBUSY;

I think you can just call dma_fence_wait_timeout with a 0 timeout below and remove this clause.

drivers/gpu/drm/lima/lima_drv.c line 197

Quoted 12 lines
drivers/gpu/drm/lima/lima_drv.c new file +12 -0
+       else {
+               unsigned long timeout = lima_timeout_to_jiffies(timeout_ns);
+
+               /* must use long for result check because in 64bit arch int
+                * will overflow if timeout is too large and get <0 result
+                */
+               ret = dma_fence_wait_timeout(fence, true, timeout);
+               if (ret == 0)
+                       ret = timeout ? -ETIMEDOUT : -EBUSY;
+               else if (ret > 0)
+                       ret = 0;
+       }
Quoted 3 lines

I suspect this could be common like reservation object waits. However, I'm curious why lima needs this ioctl in the first place when I don't see the same for other drivers.

I've prepared a v2 patch set which removed this ioctl by adopt Eric's suggestion to use drm_syncobj.

Quoted 410 lines
Quoted 125 lines
drivers/gpu/drm/lima/lima_drv.c new file +124 -0
+
+       return ret;
+}
+
+static int lima_ioctl_wait_fence(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_wait_fence *args = data;
+       struct lima_drm_priv *priv = file->driver_priv;
+       struct dma_fence *fence;
+       int err = 0;
+
+       fence = lima_ctx_get_native_fence(&priv->ctx_mgr, args->ctx,
+                                         args->pipe, args->seq);
+       if (IS_ERR(fence))
+               return PTR_ERR(fence);
+
+       if (fence) {
+               err = lima_wait_fence(fence, args->timeout_ns);
+               args->error = fence->error;
+               dma_fence_put(fence);
+       }
+       else
+               args->error = 0;
+
+       return err;
+}
+
+static int lima_ioctl_gem_wait(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_wait *args = data;
+
+       if (!(args->op & (LIMA_GEM_WAIT_READ|LIMA_GEM_WAIT_WRITE)))
+           return -EINVAL;
+
+       return lima_gem_wait(file, args->handle, args->op, args->timeout_ns);
+}
+
+static int lima_ioctl_ctx(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_ctx *args = data;
+       struct lima_drm_priv *priv = file->driver_priv;
+       struct lima_device *ldev = to_lima_dev(dev);
+
+       if (args->op == LIMA_CTX_OP_CREATE)
+               return lima_ctx_create(ldev, &priv->ctx_mgr, &args->id);
+       else if (args->op == LIMA_CTX_OP_FREE)
+               return lima_ctx_free(&priv->ctx_mgr, args->id);
+
+       return -EINVAL;
+}
+
+static int lima_ioctl_gem_mod(struct drm_device *dev, void *data, struct drm_file *file)
+{
+       struct drm_lima_gem_mod *args = data;
+
+       if (args->op == LIMA_GEM_MOD_OP_GET)
+               return lima_gem_get_modifier(file, args->handle, &args->modifier);
+       else if (args->op == LIMA_GEM_MOD_OP_SET)
+               return lima_gem_set_modifier(file, args->handle, args->modifier);
+
+       return -EINVAL;
+}
+
+static int lima_drm_driver_open(struct drm_device *dev, struct drm_file *file)
+{
+       int err;
+       struct lima_drm_priv *priv;
+       struct lima_device *ldev = to_lima_dev(dev);
+
+       priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+       if (!priv)
+               return -ENOMEM;
+
+       priv->vm = lima_vm_create(ldev);
+       if (!priv->vm) {
+               err = -ENOMEM;
+               goto err_out0;
+       }
+
+        lima_ctx_mgr_init(&priv->ctx_mgr);
+
+       file->driver_priv = priv;
+       return 0;
+
+err_out0:
+       kfree(priv);
+       return err;
+}
+
+static void lima_drm_driver_postclose(struct drm_device *dev, struct drm_file *file)
+{
+       struct lima_drm_priv *priv = file->driver_priv;
+
+       lima_ctx_mgr_fini(&priv->ctx_mgr);
+       lima_vm_put(priv->vm);
+       kfree(priv);
+}
+
+static const struct drm_ioctl_desc lima_drm_driver_ioctls[] = {
+       DRM_IOCTL_DEF_DRV(LIMA_INFO, lima_ioctl_info, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_CREATE, lima_ioctl_gem_create, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_INFO, lima_ioctl_gem_info, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_VA, lima_ioctl_gem_va, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_SUBMIT, lima_ioctl_gem_submit, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_WAIT_FENCE, lima_ioctl_wait_fence, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_WAIT, lima_ioctl_gem_wait, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_CTX, lima_ioctl_ctx, DRM_AUTH|DRM_RENDER_ALLOW),
+       DRM_IOCTL_DEF_DRV(LIMA_GEM_MOD, lima_ioctl_gem_mod, DRM_AUTH|DRM_RENDER_ALLOW),
+};
+
+static const struct file_operations lima_drm_driver_fops = {
+       .owner              = THIS_MODULE,
+       .open               = drm_open,
+       .release            = drm_release,
+       .unlocked_ioctl     = drm_ioctl,
+#ifdef CONFIG_COMPAT
+       .compat_ioctl       = drm_compat_ioctl,
+#endif
+       .mmap               = lima_gem_mmap,
+};
+
+static struct drm_driver lima_drm_driver = {
+       .driver_features    = DRIVER_RENDER | DRIVER_GEM | DRIVER_PRIME,
+       .open               = lima_drm_driver_open,

drivers/gpu/drm/lima/lima_drv.c line 331

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+       .postclose          = lima_drm_driver_postclose,
+       .ioctls             = lima_drm_driver_ioctls,
+       .num_ioctls         = ARRAY_SIZE(lima_drm_driver_ioctls),
+       .fops               = &lima_drm_driver_fops,
+       .gem_free_object_unlocked = lima_gem_free_object,
+       .gem_open_object    = lima_gem_object_open,
+       .gem_close_object   = lima_gem_object_close,
+       .name               = "lima",
+       .desc               = "lima DRM",
+       .date               = "20170325",

Perhaps this should be updated? TBH, I don't know why this is even useful.

drivers/gpu/drm/lima/lima_drv.c line 340

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +9 -0
+       .major              = 1,
+       .minor              = 0,
+       .patchlevel         = 0,
+
+       .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
+       .gem_prime_import   = drm_gem_prime_import,
+       .gem_prime_import_sg_table = lima_gem_prime_import_sg_table,
+       .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
+       .gem_prime_export   = drm_gem_prime_export,

import and export don't have to be set if you use the defaults.

Quoted 26 lines
drivers/gpu/drm/lima/lima_drv.c new file +25 -0
+       .gem_prime_res_obj  = lima_gem_prime_res_obj,
+       .gem_prime_get_sg_table = lima_gem_prime_get_sg_table,
+       .gem_prime_vmap = lima_gem_prime_vmap,
+       .gem_prime_vunmap = lima_gem_prime_vunmap,
+       .gem_prime_mmap = lima_gem_prime_mmap,
+};
+
 
+int lima_gem_wait(struct drm_file *file, u32 handle, u32 op, u64 timeout_ns)
+{
+       bool write = op & LIMA_GEM_WAIT_WRITE;
+       struct drm_gem_object *obj;
+       struct lima_bo *bo;
+       signed long ret;
+       unsigned long timeout;
+
+       obj = drm_gem_object_lookup(file, handle);
+       if (!obj)
+               return -ENOENT;
+
+       bo = to_lima_bo(obj);
+
+       timeout = timeout_ns ? lima_timeout_to_jiffies(timeout_ns) : 0;
+
+       ret = lima_bo_reserve(bo, true);
+       if (ret)

drivers/gpu/drm/lima/lima_drv.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+               goto out;
+
+       /* must use long for result check because in 64bit arch int
+        * will overflow if timeout is too large and get <0 result
+        */
+       ret = reservation_object_wait_timeout_rcu(bo->tbo.resv, write, true, timeout);
+       if (ret == 0)
+               ret = timeout ? -ETIMEDOUT : -EBUSY;
+       else if (ret > 0)
+               ret = 0;

There's a helper I added for all this that should land in 5.1.

Quoted 9 lines
drivers/gpu/drm/lima/lima_drv.c new file +8 -0
+
+       lima_bo_unreserve(bo);
+out:
+       drm_gem_object_put_unlocked(obj);
+       return ret;
+}
+
 
+static int lima_gp_soft_reset_async_wait(struct lima_ip *ip)

drivers/gpu/drm/lima/lima_drv.c

Quoted 10 lines
drivers/gpu/drm/lima/lima_drv.c new file +10 -0
+{
+       struct lima_device *dev = ip->dev;
+       int timeout;
+
+       if (!ip->data.async_reset)
+               return 0;
+
+       for (timeout = 1000; timeout > 0; timeout--) {
+               if (gp_read(INT_RAWSTAT) & LIMA_GP_IRQ_RESET_COMPLETED)
+                       break;

Use readl_poll_timeout instead of writing your own. At least add a udelay to the loop so the timing is fixed and not dependent on how fast the code can run.

Quoted 30 lines
drivers/gpu/drm/lima/lima_drv.c new file +12 -0
+       }
+       if (!timeout) {
+               dev_err(dev->dev, "gp soft reset time out\n");
+               return -ETIMEDOUT;
+       }
+
+       gp_write(INT_CLEAR, LIMA_GP_IRQ_MASK_ALL);
+       gp_write(INT_MASK, LIMA_GP_IRQ_MASK_USED);
+
+       ip->data.async_reset = false;
+       return 0;
+}
drivers/gpu/drm/lima/lima_l2_cache.c new file +11 -0
new file mode 100644
index 000000000000..e7cdec720e5d
@@ -0,0 +1,79 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_l2_cache.h"
+#include "lima_regs.h"
+
+#define l2_cache_write(reg, data) writel(data, ip->iomem + LIMA_L2_CACHE_##reg)

drivers/gpu/drm/lima/lima_l2_cache.c line 21

Quoted 10 lines
drivers/gpu/drm/lima/lima_l2_cache.c new file +10 -0
+#define l2_cache_read(reg) readl(ip->iomem + LIMA_L2_CACHE_##reg)
+
+static int lima_l2_cache_wait_idle(struct lima_ip *ip)
+{
+       int timeout;
+       struct lima_device *dev = ip->dev;
+
+       for (timeout = 100000; timeout > 0; timeout--) {
+           if (!(l2_cache_read(STATUS) & LIMA_L2_CACHE_STATUS_COMMAND_BUSY))
+               break;

Use readl_poll_timeout or variant.

Quoted 115 lines
drivers/gpu/drm/lima/lima_l2_cache.c new file +58 -0
+       }
+       if (!timeout) {
+           dev_err(dev->dev, "l2 cache wait command timeout\n");
+           return -ETIMEDOUT;
+       }
+       return 0;
+}
+
+int lima_l2_cache_flush(struct lima_ip *ip)
+{
+       int ret;
+
+       spin_lock(&ip->data.lock);
+       l2_cache_write(COMMAND, LIMA_L2_CACHE_COMMAND_CLEAR_ALL);
+       ret = lima_l2_cache_wait_idle(ip);
+       spin_unlock(&ip->data.lock);
+       return ret;
+}
+
+int lima_l2_cache_init(struct lima_ip *ip)
+{
+       int i, err;
+       u32 size;
+       struct lima_device *dev = ip->dev;
+
+       /* l2_cache2 only exists when one of PP4-7 present */
+       if (ip->id == lima_ip_l2_cache2) {
+               for (i = lima_ip_pp4; i <= lima_ip_pp7; i++) {
+                       if (dev->ip[i].present)
+                               break;
+               }
+               if (i > lima_ip_pp7)
+                       return -ENODEV;
+       }
+
+       spin_lock_init(&ip->data.lock);
+
+       size = l2_cache_read(SIZE);
+       dev_info(dev->dev, "l2 cache %uK, %u-way, %ubyte cache line, %ubit external bus\n",
+                1 << (((size >> 16) & 0xff) - 10),
+                1 << ((size >> 8) & 0xff),
+                1 << (size & 0xff),
+                1 << ((size >> 24) & 0xff));
+
+       err = lima_l2_cache_flush(ip);
+       if (err)
+               return err;
+
+       l2_cache_write(ENABLE, LIMA_L2_CACHE_ENABLE_ACCESS | LIMA_L2_CACHE_ENABLE_READ_ALLOCATE);
+       l2_cache_write(MAX_READS, 0x1c);
+
+       return 0;
+}
+
+void lima_l2_cache_fini(struct lima_ip *ip)
+{
+
+}
drivers/gpu/drm/lima/lima_l2_cache.h new file +14 -0
new file mode 100644
index 000000000000..2ff91eafefbe
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_L2_CACHE_H__
+#define __LIMA_L2_CACHE_H__
+
+struct lima_ip;
+
+int lima_l2_cache_init(struct lima_ip *ip);
+void lima_l2_cache_fini(struct lima_ip *ip);
+
+int lima_l2_cache_flush(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_mmu.c new file +31 -0
new file mode 100644
index 000000000000..234fb90a4285
@@ -0,0 +1,135 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/device.h>
+
+#include "lima_device.h"
+#include "lima_mmu.h"
+#include "lima_vm.h"
+#include "lima_object.h"
+#include "lima_regs.h"
+
+#define mmu_write(reg, data) writel(data, ip->iomem + LIMA_MMU_##reg)
+#define mmu_read(reg) readl(ip->iomem + LIMA_MMU_##reg)
+
+#define lima_mmu_send_command(command, condition)           \
+({                                                          \
+       int __timeout, __ret = 0;                            \
+                                                            \
+       mmu_write(COMMAND, command);                         \
+       for (__timeout = 1000; __timeout > 0; __timeout--) { \
+               if (condition)                               \
+                       break;                               \
+       }                                                    \
+       if (!__timeout) {                                    \
+               dev_err(dev->dev, "mmu command %x timeout\n", command); \
+               __ret = -ETIMEDOUT;                          \
+       }                                                    \
+       __ret;                                               \
+})

drivers/gpu/drm/lima/lima_mmu.c line 41

Quoted 10 lines
drivers/gpu/drm/lima/lima_mmu.c new file +10 -0
+
+static irqreturn_t lima_mmu_irq_handler(int irq, void *data)
+{
+       struct lima_ip *ip = data;
+       struct lima_device *dev = ip->dev;
+       u32 status = mmu_read(INT_STATUS);
+       struct lima_sched_pipe *pipe;
+
+       /* for shared irq case */
+       if (!status)

Can status have masked irq's? If so, you should be masking out the disabled irq bits.

Quoted 23 lines
drivers/gpu/drm/lima/lima_mmu.c new file +23 -0
+               return IRQ_NONE;
+
+       if (status & LIMA_MMU_INT_PAGE_FAULT) {
+               u32 fault = mmu_read(PAGE_FAULT_ADDR);
+               dev_err(dev->dev, "mmu page fault at 0x%x from bus id %d of type %s on %s\n",
+                       fault, LIMA_MMU_STATUS_BUS_ID(status),
+                       status & LIMA_MMU_STATUS_PAGE_FAULT_IS_WRITE ? "write" : "read",
+                       lima_ip_name(ip));
+       }
+
+       if (status & LIMA_MMU_INT_READ_BUS_ERROR) {
+               dev_err(dev->dev, "mmu %s irq bus error\n", lima_ip_name(ip));
+       }
+
+       /* mask all interrupts before resume */
+       mmu_write(INT_MASK, 0);
+       mmu_write(INT_CLEAR, status);
+
+       pipe = dev->pipe + (ip->id == lima_ip_gpmmu ? lima_pipe_gp : lima_pipe_pp);
+       lima_sched_pipe_mmu_error(pipe);
+
+       return IRQ_HANDLED;
+}
Quoted 5 lines

drivers/gpu/drm/lima/lima_mmu.c line 66

Quoted 2 lines
drivers/gpu/drm/lima/lima_mmu.c new file +2 -0
+
+unsigned long lima_timeout_to_jiffies(u64 timeout_ns)

Create a common helper instead of copy-n-pasting this from other drivers (etnaviv).

There's one drm_timeout_abs_to_jiffies but not exported.

Quoted 124 lines
Quoted 124 lines
drivers/gpu/drm/lima/lima_mmu.c new file +31 -0
+{
+       unsigned long timeout_jiffies;
+       ktime_t timeout;
+
+       /* clamp timeout if it's to large */
+       if (((s64)timeout_ns) < 0)
+               return MAX_SCHEDULE_TIMEOUT;
+
+       timeout = ktime_sub(ns_to_ktime(timeout_ns), ktime_get());
+       if (ktime_to_ns(timeout) < 0)
+               return 0;
+
+       timeout_jiffies = nsecs_to_jiffies(ktime_to_ns(timeout));
+       /*  clamp timeout to avoid unsigned-> signed overflow */
+       if (timeout_jiffies > MAX_SCHEDULE_TIMEOUT )
+               return MAX_SCHEDULE_TIMEOUT;
+
+       return timeout_jiffies;
+}
+
+void lima_sched_pipe_task_done(struct lima_sched_pipe *pipe)
+{
+       if (pipe->error)
+               schedule_work(&pipe->error_work);
+       else {
+               struct lima_sched_task *task = pipe->current_task;
+
+               pipe->task_fini(pipe);
+               dma_fence_signal(task->fence);
+       }
+}
drivers/gpu/drm/lima/lima_vm.c new file +85 -0
new file mode 100644
index 000000000000..a264f3ae83fe
@@ -0,0 +1,354 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#include <linux/slab.h>
+#include <linux/dma-mapping.h>
+#include <linux/interval_tree_generic.h>
+
+#include "lima_device.h"
+#include "lima_vm.h"
+#include "lima_object.h"
+#include "lima_regs.h"
+
+struct lima_bo_va_mapping {
+       struct list_head list;
+       struct rb_node rb;
+       uint32_t start;
+       uint32_t last;
+       uint32_t __subtree_last;
+};
+
+struct lima_bo_va {
+       struct list_head list;
+       unsigned ref_count;
+
+       struct list_head mapping;
+
+       struct lima_vm *vm;
+};
+
+#define LIMA_VM_PD_SHIFT 22
+#define LIMA_VM_PT_SHIFT 12
+#define LIMA_VM_PB_SHIFT (LIMA_VM_PD_SHIFT + LIMA_VM_NUM_PT_PER_BT_SHIFT)
+#define LIMA_VM_BT_SHIFT LIMA_VM_PT_SHIFT
+
+#define LIMA_VM_PT_MASK ((1 << LIMA_VM_PD_SHIFT) - 1)
+#define LIMA_VM_BT_MASK ((1 << LIMA_VM_PB_SHIFT) - 1)
+
+#define LIMA_PDE(va) (va >> LIMA_VM_PD_SHIFT)
+#define LIMA_PTE(va) ((va & LIMA_VM_PT_MASK) >> LIMA_VM_PT_SHIFT)
+#define LIMA_PBE(va) (va >> LIMA_VM_PB_SHIFT)
+#define LIMA_BTE(va) ((va & LIMA_VM_BT_MASK) >> LIMA_VM_BT_SHIFT)
+
+#define START(node) ((node)->start)
+#define LAST(node) ((node)->last)
+
+INTERVAL_TREE_DEFINE(struct lima_bo_va_mapping, rb, uint32_t, __subtree_last,
+                    START, LAST, static, lima_vm_it)
+
+#undef START
+#undef LAST
+
+static void lima_vm_unmap_page_table(struct lima_vm *vm, u32 start, u32 end)
+{
+       u32 addr;
+
+       for (addr = start; addr <= end; addr += LIMA_PAGE_SIZE) {
+               u32 pbe = LIMA_PBE(addr);
+               u32 bte = LIMA_BTE(addr);
+               u32 *bt;
+
+               bt = lima_bo_kmap(vm->bts[pbe]);
+               bt[bte] = 0;
+       }
+}
+
+static int lima_vm_map_page_table(struct lima_vm *vm, dma_addr_t *dma,
+                                 u32 start, u32 end)
+{
+       u64 addr;
+       int err, i = 0;
+
+       for (addr = start; addr <= end; addr += LIMA_PAGE_SIZE) {
+               u32 pbe = LIMA_PBE(addr);
+               u32 bte = LIMA_BTE(addr);
+               u32 *bt;
+
+               if (vm->bts[pbe])
+                       bt = lima_bo_kmap(vm->bts[pbe]);
+               else {
+                       struct lima_bo *bt_bo;
+                       dma_addr_t *pts;
+                       u32 *pd;
+                       int j;
+
+                       bt_bo = lima_bo_create(
Quoted 8 lines

drivers/gpu/drm/lima/lima_vm.c line 88

Quoted 3 lines
drivers/gpu/drm/lima/lima_vm.c new file +3 -0
+                               vm->dev, LIMA_PAGE_SIZE << LIMA_VM_NUM_PT_PER_BT_SHIFT,
+                               0, ttm_bo_type_kernel,
+                               NULL, vm->pd->tbo.resv);

I don't think using BOs for page tables buys you anything. You could just use the kernel DMA API directly. See io-pgtable-arm-v7s.c for inspiration. For panfrost, it's standard ARM format page tables so we can just use the io-pgtable library.

Right, v2 will use DMA API directly.

Quoted 62 lines
Quoted 62 lines
drivers/gpu/drm/lima/lima_vm.c new file +61 -0
+                       if (IS_ERR(bt_bo)) {
+                               err = PTR_ERR(bt_bo);
+                               goto err_out;
+                       }
+
+                       bt = lima_bo_kmap(bt_bo);
+                       if (IS_ERR(bt)) {
+                               lima_bo_unref(bt_bo);
+                               err = PTR_ERR(bt);
+                               goto err_out;
+                       }
+                       memset(bt, 0, LIMA_PAGE_SIZE << LIMA_VM_NUM_PT_PER_BT_SHIFT);
+
+                       vm->bts[pbe] = bt_bo;
+                       pd = lima_bo_kmap(vm->pd);
+                       pd += pbe << LIMA_VM_NUM_PT_PER_BT_SHIFT;
+                       pts = lima_bo_get_pages(bt_bo);
+                       for (j = 0; j < LIMA_VM_NUM_PT_PER_BT; j++)
+                               *pd++ = *pts++ | LIMA_VM_FLAG_PRESENT;
+               }
+
+               bt[bte] = dma[i++] | LIMA_VM_FLAGS_CACHE;
+       }
+
+       return 0;
+
+err_out:
+       if (addr != start)
+               lima_vm_unmap_page_table(vm, start, addr - 1);
+       return err;
+}
+
+static struct lima_bo_va *
+lima_vm_bo_find(struct lima_vm *vm, struct lima_bo *bo)
+{
+       struct lima_bo_va *bo_va, *ret = NULL;
+
+       list_for_each_entry(bo_va, &bo->va, list) {
+               if (bo_va->vm == vm) {
+                       ret = bo_va;
+                       break;
+               }
+       }
+
+       return ret;
+}
+
+int lima_vm_bo_map(struct lima_vm *vm, struct lima_bo *bo, u32 start)
+{
+       int err;
+       struct lima_bo_va_mapping *it, *mapping;
+       u32 end = start + bo->gem.size - 1;
+       dma_addr_t *pages_dma = lima_bo_get_pages(bo);
+       struct lima_bo_va *bo_va;
+
+       it = lima_vm_it_iter_first(&vm->va, start, end);
+       if (it) {
+               dev_dbg(bo->gem.dev->dev, "lima vm map va overlap %x-%x %x-%x\n",
+                       start, end, it->start, it->last);
+               return -EINVAL;
+       }
Quoted 8 lines

drivers/gpu/drm/lima/lima_vm.c line 155

Quoted 6 lines
drivers/gpu/drm/lima/lima_vm.c new file +6 -0
+
+       mapping = kmalloc(sizeof(*mapping), GFP_KERNEL);
+       if (!mapping)
+               return -ENOMEM;
+       mapping->start = start;
+       mapping->last = end;

Why don't you use the drm_mm_XX APIs instead of writing your own?

Use it in v2.

Quoted 15 lines
Quoted 5 lines
drivers/gpu/drm/lima/lima_vm.c new file +4 -0
+
+       err = lima_vm_map_page_table(vm, pages_dma, start, end);
+       if (err) {
+               kfree(mapping);

drivers/gpu/drm/lima/lima_vm.c line 169

Quoted 10 lines
drivers/gpu/drm/lima/lima_vm.c new file +10 -0
+               return err;
+       }
+
+       lima_vm_it_insert(mapping, &vm->va);
+
+       bo_va = lima_vm_bo_find(vm, bo);
+       list_add_tail(&mapping->list, &bo_va->mapping);
+
+       return 0;
+}

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#24 lore raw

Re: [PATCH 2/2] drm/lima: driver for ARM Mali4xx GPUs

Quoted 22 lines · Qiang Yu
Quoted 19 lines · Rob Herring
Quoted 2 lines · Qiang Yu

From: Lima Project Developers <[email protected]>

This should be a person (you).

Quoted 6 lines

Being the submitter, your S-o-b should be last.

Quoted 3 lines

---

Quoted 6 lines
Quoted 5 lines

drivers/gpu/drm/lima/lima_sched.c line 455

Quoted 2 lines
drivers/gpu/drm/lima/lima_sched.c new file +2 -0
+
+unsigned long lima_timeout_to_jiffies(u64 timeout_ns)

Create a common helper instead of copy-n-pasting this from other drivers (etnaviv).

There's one drm_timeout_abs_to_jiffies but not exported.

Then let's export it. We don't work around kernel APIs or lack of. We fix them.

Rob
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#25 lore raw
In reply to Qiang Yu
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

What's the design choice behind this?

From an upstream pov having all soc gpu drivers use similar approaches should help with shared infrastructure and stuff like that.

Another one: What's the plan with extending this to panfrost? Or are the architectures for command submission totally different, and we'll need separate kernel drivers for utgard/midgard/bifrost?

Thanks, Daniel

Quoted 104 lines

- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

This patch serial is based on 5.0-rc5 and squash all the commits. For whole history of this driver's development, see:
https://gitlab.freedesktop.org/lima/linux/commits/lima-5.0-rc5
https://gitlab.freedesktop.org/lima/linux/commits/lima-4.17-rc4

Mesa driver is still in development and not ready for daily usage, but can run some simple tests like kmscube and glamrk2, and some single full screen application like kodi-gbm, see:
https://gitlab.freedesktop.org/lima/mesa

[rfc]
https://lists.freedesktop.org/archives/dri-devel/2018-May/177314.html

Lima Project Developers (1):
drm/lima: driver for ARM Mali4xx GPUs

Qiang Yu (1):
drm/fourcc: add ARM tiled format modifier

 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  22 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  46 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       | 124 +++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  33 ++
 drivers/gpu/drm/lima/lima_device.c    | 384 ++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 116 ++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  56 +++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 +
 drivers/gpu/drm/lima/lima_drv.c       | 459 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  59 ++++
 drivers/gpu/drm/lima/lima_gem.c       | 485 +++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c | 144 ++++++++
 drivers/gpu/drm/lima/lima_gem_prime.h |  18 +
 drivers/gpu/drm/lima/lima_gp.c        | 280 +++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  79 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 135 +++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 103 ++++++
 drivers/gpu/drm/lima/lima_object.h    |  72 ++++
 drivers/gpu/drm/lima/lima_pmu.c       |  61 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 419 ++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_pp.h        |  19 +
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 486 ++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 108 ++++++
 drivers/gpu/drm/lima/lima_ttm.c       | 319 +++++++++++++++++
 drivers/gpu/drm/lima/lima_ttm.h       |  24 ++
 drivers/gpu/drm/lima/lima_vm.c        | 354 +++++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  59 ++++
 include/uapi/drm/drm_fourcc.h         |   9 +
 include/uapi/drm/lima_drm.h           | 193 ++++++++++
 39 files changed, 5092 insertions(+)
 create mode 100644 drivers/gpu/drm/lima/Kconfig
 create mode 100644 drivers/gpu/drm/lima/Makefile
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.c
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.h
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.c
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.h
 create mode 100644 drivers/gpu/drm/lima/lima_device.c
 create mode 100644 drivers/gpu/drm/lima/lima_device.h
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h
 create mode 100644 drivers/gpu/drm/lima/lima_drv.c
 create mode 100644 drivers/gpu/drm/lima/lima_drv.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h
 create mode 100644 drivers/gpu/drm/lima/lima_gp.c
 create mode 100644 drivers/gpu/drm/lima/lima_gp.h
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_object.c
 create mode 100644 drivers/gpu/drm/lima/lima_object.h
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_pp.c
 create mode 100644 drivers/gpu/drm/lima/lima_pp.h
 create mode 100644 drivers/gpu/drm/lima/lima_regs.h
 create mode 100644 drivers/gpu/drm/lima/lima_sched.c
 create mode 100644 drivers/gpu/drm/lima/lima_sched.h
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.c
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.h
 create mode 100644 drivers/gpu/drm/lima/lima_vm.c
 create mode 100644 drivers/gpu/drm/lima/lima_vm.h
 create mode 100644 include/uapi/drm/lima_drm.h
Signature · 1 line
2.17.1
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#26 lore raw

Am 07.02.19 um 10:09 schrieb Daniel Vetter:

Quoted 35 lines · Qiang Yu
Quoted 26 lines

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

Quoted 6 lines

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

What's the design choice behind this?

Agree that this seems unnecessary complicated.

Additional to that why do you use TTM_PL_FLAG_NO_EVICT? That is a serious show stopper and as far as I can of hand see completely unnecessary.

Christian.

Quoted 114 lines

From an upstream pov having all soc gpu drivers use similar approaches
should help with shared infrastructure and stuff like that.

Another one: What's the plan with extending this to panfrost? Or are the architectures for command submission totally different, and we'll need separate kernel drivers for utgard/midgard/bifrost?

Thanks, Daniel

Quoted 104 lines

- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

This patch serial is based on 5.0-rc5 and squash all the commits. For whole history of this driver's development, see:
https://gitlab.freedesktop.org/lima/linux/commits/lima-5.0-rc5
https://gitlab.freedesktop.org/lima/linux/commits/lima-4.17-rc4

Mesa driver is still in development and not ready for daily usage, but can run some simple tests like kmscube and glamrk2, and some single full screen application like kodi-gbm, see:
https://gitlab.freedesktop.org/lima/mesa

[rfc]
https://lists.freedesktop.org/archives/dri-devel/2018-May/177314.html

Lima Project Developers (1):
drm/lima: driver for ARM Mali4xx GPUs

Qiang Yu (1):
drm/fourcc: add ARM tiled format modifier

  drivers/gpu/drm/Kconfig               |   2 +
  drivers/gpu/drm/Makefile              |   1 +
  drivers/gpu/drm/lima/Kconfig          |  10 +
  drivers/gpu/drm/lima/Makefile         |  22 ++
  drivers/gpu/drm/lima/lima_bcast.c     |  46 +++
  drivers/gpu/drm/lima/lima_bcast.h     |  14 +
  drivers/gpu/drm/lima/lima_ctx.c       | 124 +++++++
  drivers/gpu/drm/lima/lima_ctx.h       |  33 ++
  drivers/gpu/drm/lima/lima_device.c    | 384 ++++++++++++++++++++
  drivers/gpu/drm/lima/lima_device.h    | 116 ++++++
  drivers/gpu/drm/lima/lima_dlbu.c      |  56 +++
  drivers/gpu/drm/lima/lima_dlbu.h      |  18 +
  drivers/gpu/drm/lima/lima_drv.c       | 459 ++++++++++++++++++++++++
  drivers/gpu/drm/lima/lima_drv.h       |  59 ++++
  drivers/gpu/drm/lima/lima_gem.c       | 485 +++++++++++++++++++++++++
  drivers/gpu/drm/lima/lima_gem.h       |  25 ++
  drivers/gpu/drm/lima/lima_gem_prime.c | 144 ++++++++
  drivers/gpu/drm/lima/lima_gem_prime.h |  18 +
  drivers/gpu/drm/lima/lima_gp.c        | 280 +++++++++++++++
  drivers/gpu/drm/lima/lima_gp.h        |  16 +
  drivers/gpu/drm/lima/lima_l2_cache.c  |  79 +++++
  drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
  drivers/gpu/drm/lima/lima_mmu.c       | 135 +++++++
  drivers/gpu/drm/lima/lima_mmu.h       |  16 +
  drivers/gpu/drm/lima/lima_object.c    | 103 ++++++
  drivers/gpu/drm/lima/lima_object.h    |  72 ++++
  drivers/gpu/drm/lima/lima_pmu.c       |  61 ++++
  drivers/gpu/drm/lima/lima_pmu.h       |  12 +
  drivers/gpu/drm/lima/lima_pp.c        | 419 ++++++++++++++++++++++
  drivers/gpu/drm/lima/lima_pp.h        |  19 +
  drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++
  drivers/gpu/drm/lima/lima_sched.c     | 486 ++++++++++++++++++++++++++
  drivers/gpu/drm/lima/lima_sched.h     | 108 ++++++
  drivers/gpu/drm/lima/lima_ttm.c       | 319 +++++++++++++++++
  drivers/gpu/drm/lima/lima_ttm.h       |  24 ++
  drivers/gpu/drm/lima/lima_vm.c        | 354 +++++++++++++++++++
  drivers/gpu/drm/lima/lima_vm.h        |  59 ++++
  include/uapi/drm/drm_fourcc.h         |   9 +
  include/uapi/drm/lima_drm.h           | 193 ++++++++++
  39 files changed, 5092 insertions(+)

create mode 100644 drivers/gpu/drm/lima/Kconfig create mode 100644 drivers/gpu/drm/lima/Makefile create mode 100644 drivers/gpu/drm/lima/lima_bcast.c create mode 100644 drivers/gpu/drm/lima/lima_bcast.h create mode 100644 drivers/gpu/drm/lima/lima_ctx.c create mode 100644 drivers/gpu/drm/lima/lima_ctx.h create mode 100644 drivers/gpu/drm/lima/lima_device.c create mode 100644 drivers/gpu/drm/lima/lima_device.h create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h create mode 100644 drivers/gpu/drm/lima/lima_drv.c create mode 100644 drivers/gpu/drm/lima/lima_drv.h create mode 100644 drivers/gpu/drm/lima/lima_gem.c create mode 100644 drivers/gpu/drm/lima/lima_gem.h create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h create mode 100644 drivers/gpu/drm/lima/lima_gp.c create mode 100644 drivers/gpu/drm/lima/lima_gp.h create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h create mode 100644 drivers/gpu/drm/lima/lima_mmu.c create mode 100644 drivers/gpu/drm/lima/lima_mmu.h create mode 100644 drivers/gpu/drm/lima/lima_object.c create mode 100644 drivers/gpu/drm/lima/lima_object.h create mode 100644 drivers/gpu/drm/lima/lima_pmu.c create mode 100644 drivers/gpu/drm/lima/lima_pmu.h create mode 100644 drivers/gpu/drm/lima/lima_pp.c create mode 100644 drivers/gpu/drm/lima/lima_pp.h create mode 100644 drivers/gpu/drm/lima/lima_regs.h create mode 100644 drivers/gpu/drm/lima/lima_sched.c create mode 100644 drivers/gpu/drm/lima/lima_sched.h create mode 100644 drivers/gpu/drm/lima/lima_ttm.c create mode 100644 drivers/gpu/drm/lima/lima_ttm.h create mode 100644 drivers/gpu/drm/lima/lima_vm.c create mode 100644 drivers/gpu/drm/lima/lima_vm.h create mode 100644 include/uapi/drm/lima_drm.h

Signature · 1 line
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#27 lore raw
Quoted 42 lines · Christian König

Am 07.02.19 um 10:09 schrieb Daniel Vetter:

Quoted 40 lines · Qiang Yu
Quoted 26 lines

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 6 lines
Quoted 1 line

What's the design choice behind this?

Agree that this seems unnecessary complicated.

Additional to that why do you use TTM_PL_FLAG_NO_EVICT? That is a serious show stopper and as far as I can of hand see completely unnecessary.

Just for simplification. There's no eviction for unified mem, but swap will be introduced when this flag is not set. So I have to do vm table clear/restore and call bo validation which I plan to implement in the future.

Thanks,
Qiang

Quoted 118 lines

Christian.

Quoted 114 lines

From an upstream pov having all soc gpu drivers use similar approaches
should help with shared infrastructure and stuff like that.

Another one: What's the plan with extending this to panfrost? Or are the architectures for command submission totally different, and we'll need separate kernel drivers for utgard/midgard/bifrost?

Thanks, Daniel

Quoted 104 lines

- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

This patch serial is based on 5.0-rc5 and squash all the commits. For whole history of this driver's development, see:
https://gitlab.freedesktop.org/lima/linux/commits/lima-5.0-rc5
https://gitlab.freedesktop.org/lima/linux/commits/lima-4.17-rc4

Mesa driver is still in development and not ready for daily usage, but can run some simple tests like kmscube and glamrk2, and some single full screen application like kodi-gbm, see:
https://gitlab.freedesktop.org/lima/mesa

[rfc]
https://lists.freedesktop.org/archives/dri-devel/2018-May/177314.html

Lima Project Developers (1):
drm/lima: driver for ARM Mali4xx GPUs

Qiang Yu (1):
drm/fourcc: add ARM tiled format modifier

  drivers/gpu/drm/Kconfig               |   2 +
  drivers/gpu/drm/Makefile              |   1 +
  drivers/gpu/drm/lima/Kconfig          |  10 +
  drivers/gpu/drm/lima/Makefile         |  22 ++
  drivers/gpu/drm/lima/lima_bcast.c     |  46 +++
  drivers/gpu/drm/lima/lima_bcast.h     |  14 +
  drivers/gpu/drm/lima/lima_ctx.c       | 124 +++++++
  drivers/gpu/drm/lima/lima_ctx.h       |  33 ++
  drivers/gpu/drm/lima/lima_device.c    | 384 ++++++++++++++++++++
  drivers/gpu/drm/lima/lima_device.h    | 116 ++++++
  drivers/gpu/drm/lima/lima_dlbu.c      |  56 +++
  drivers/gpu/drm/lima/lima_dlbu.h      |  18 +
  drivers/gpu/drm/lima/lima_drv.c       | 459 ++++++++++++++++++++++++
  drivers/gpu/drm/lima/lima_drv.h       |  59 ++++
  drivers/gpu/drm/lima/lima_gem.c       | 485 +++++++++++++++++++++++++
  drivers/gpu/drm/lima/lima_gem.h       |  25 ++
  drivers/gpu/drm/lima/lima_gem_prime.c | 144 ++++++++
  drivers/gpu/drm/lima/lima_gem_prime.h |  18 +
  drivers/gpu/drm/lima/lima_gp.c        | 280 +++++++++++++++
  drivers/gpu/drm/lima/lima_gp.h        |  16 +
  drivers/gpu/drm/lima/lima_l2_cache.c  |  79 +++++
  drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
  drivers/gpu/drm/lima/lima_mmu.c       | 135 +++++++
  drivers/gpu/drm/lima/lima_mmu.h       |  16 +
  drivers/gpu/drm/lima/lima_object.c    | 103 ++++++
  drivers/gpu/drm/lima/lima_object.h    |  72 ++++
  drivers/gpu/drm/lima/lima_pmu.c       |  61 ++++
  drivers/gpu/drm/lima/lima_pmu.h       |  12 +
  drivers/gpu/drm/lima/lima_pp.c        | 419 ++++++++++++++++++++++
  drivers/gpu/drm/lima/lima_pp.h        |  19 +
  drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++
  drivers/gpu/drm/lima/lima_sched.c     | 486 ++++++++++++++++++++++++++
  drivers/gpu/drm/lima/lima_sched.h     | 108 ++++++
  drivers/gpu/drm/lima/lima_ttm.c       | 319 +++++++++++++++++
  drivers/gpu/drm/lima/lima_ttm.h       |  24 ++
  drivers/gpu/drm/lima/lima_vm.c        | 354 +++++++++++++++++++
  drivers/gpu/drm/lima/lima_vm.h        |  59 ++++
  include/uapi/drm/drm_fourcc.h         |   9 +
  include/uapi/drm/lima_drm.h           | 193 ++++++++++
  39 files changed, 5092 insertions(+)

create mode 100644 drivers/gpu/drm/lima/Kconfig create mode 100644 drivers/gpu/drm/lima/Makefile create mode 100644 drivers/gpu/drm/lima/lima_bcast.c create mode 100644 drivers/gpu/drm/lima/lima_bcast.h create mode 100644 drivers/gpu/drm/lima/lima_ctx.c create mode 100644 drivers/gpu/drm/lima/lima_ctx.h create mode 100644 drivers/gpu/drm/lima/lima_device.c create mode 100644 drivers/gpu/drm/lima/lima_device.h create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h create mode 100644 drivers/gpu/drm/lima/lima_drv.c create mode 100644 drivers/gpu/drm/lima/lima_drv.h create mode 100644 drivers/gpu/drm/lima/lima_gem.c create mode 100644 drivers/gpu/drm/lima/lima_gem.h create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h create mode 100644 drivers/gpu/drm/lima/lima_gp.c create mode 100644 drivers/gpu/drm/lima/lima_gp.h create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h create mode 100644 drivers/gpu/drm/lima/lima_mmu.c create mode 100644 drivers/gpu/drm/lima/lima_mmu.h create mode 100644 drivers/gpu/drm/lima/lima_object.c create mode 100644 drivers/gpu/drm/lima/lima_object.h create mode 100644 drivers/gpu/drm/lima/lima_pmu.c create mode 100644 drivers/gpu/drm/lima/lima_pmu.h create mode 100644 drivers/gpu/drm/lima/lima_pp.c create mode 100644 drivers/gpu/drm/lima/lima_pp.h create mode 100644 drivers/gpu/drm/lima/lima_regs.h create mode 100644 drivers/gpu/drm/lima/lima_sched.c create mode 100644 drivers/gpu/drm/lima/lima_sched.h create mode 100644 drivers/gpu/drm/lima/lima_ttm.c create mode 100644 drivers/gpu/drm/lima/lima_ttm.h create mode 100644 drivers/gpu/drm/lima/lima_vm.c create mode 100644 drivers/gpu/drm/lima/lima_vm.h create mode 100644 include/uapi/drm/lima_drm.h

--
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#28 lore raw

Am 07.02.19 um 16:33 schrieb Qiang Yu:

Quoted 43 lines
Quoted 41 lines · Christian König

Am 07.02.19 um 10:09 schrieb Daniel Vetter:

Quoted 40 lines · Qiang Yu
Quoted 26 lines

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

What's the design choice behind this?

Agree that this seems unnecessary complicated.

Additional to that why do you use TTM_PL_FLAG_NO_EVICT? That is a serious show stopper and as far as I can of hand see completely unnecessary.

Just for simplification. There's no eviction for unified mem, but swap will be introduced when this flag is not set. So I have to do vm table clear/restore and call bo validation which I plan to implement in the future.

Ok, well if you are using GEM or GEM+TTM is up to you. I just think using GEM directly like Daniel suggest would be simpler in the long term.

But support for eviction is a serious prerequisite to allowing this upstream. So that is really something you need to clean up first.

Regards,
Christian.

Quoted 119 lines

Thanks,
Qiang

Quoted 115 lines

Christian.

Quoted 113 lines

From an upstream pov having all soc gpu drivers use similar approaches
should help with shared infrastructure and stuff like that.

Another one: What's the plan with extending this to panfrost? Or are the architectures for command submission totally different, and we'll need separate kernel drivers for utgard/midgard/bifrost?

Thanks, Daniel

Quoted 104 lines

- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

This patch serial is based on 5.0-rc5 and squash all the commits. For whole history of this driver's development, see:
https://gitlab.freedesktop.org/lima/linux/commits/lima-5.0-rc5
https://gitlab.freedesktop.org/lima/linux/commits/lima-4.17-rc4

Mesa driver is still in development and not ready for daily usage, but can run some simple tests like kmscube and glamrk2, and some single full screen application like kodi-gbm, see:
https://gitlab.freedesktop.org/lima/mesa

[rfc]
https://lists.freedesktop.org/archives/dri-devel/2018-May/177314.html

Lima Project Developers (1):
drm/lima: driver for ARM Mali4xx GPUs

Qiang Yu (1):
drm/fourcc: add ARM tiled format modifier

   drivers/gpu/drm/Kconfig               |   2 +
   drivers/gpu/drm/Makefile              |   1 +
   drivers/gpu/drm/lima/Kconfig          |  10 +
   drivers/gpu/drm/lima/Makefile         |  22 ++
   drivers/gpu/drm/lima/lima_bcast.c     |  46 +++
   drivers/gpu/drm/lima/lima_bcast.h     |  14 +
   drivers/gpu/drm/lima/lima_ctx.c       | 124 +++++++
   drivers/gpu/drm/lima/lima_ctx.h       |  33 ++
   drivers/gpu/drm/lima/lima_device.c    | 384 ++++++++++++++++++++
   drivers/gpu/drm/lima/lima_device.h    | 116 ++++++
   drivers/gpu/drm/lima/lima_dlbu.c      |  56 +++
   drivers/gpu/drm/lima/lima_dlbu.h      |  18 +
   drivers/gpu/drm/lima/lima_drv.c       | 459 ++++++++++++++++++++++++
   drivers/gpu/drm/lima/lima_drv.h       |  59 ++++
   drivers/gpu/drm/lima/lima_gem.c       | 485 +++++++++++++++++++++++++
   drivers/gpu/drm/lima/lima_gem.h       |  25 ++
   drivers/gpu/drm/lima/lima_gem_prime.c | 144 ++++++++
   drivers/gpu/drm/lima/lima_gem_prime.h |  18 +
   drivers/gpu/drm/lima/lima_gp.c        | 280 +++++++++++++++
   drivers/gpu/drm/lima/lima_gp.h        |  16 +
   drivers/gpu/drm/lima/lima_l2_cache.c  |  79 +++++
   drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
   drivers/gpu/drm/lima/lima_mmu.c       | 135 +++++++
   drivers/gpu/drm/lima/lima_mmu.h       |  16 +
   drivers/gpu/drm/lima/lima_object.c    | 103 ++++++
   drivers/gpu/drm/lima/lima_object.h    |  72 ++++
   drivers/gpu/drm/lima/lima_pmu.c       |  61 ++++
   drivers/gpu/drm/lima/lima_pmu.h       |  12 +
   drivers/gpu/drm/lima/lima_pp.c        | 419 ++++++++++++++++++++++
   drivers/gpu/drm/lima/lima_pp.h        |  19 +
   drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++
   drivers/gpu/drm/lima/lima_sched.c     | 486 ++++++++++++++++++++++++++
   drivers/gpu/drm/lima/lima_sched.h     | 108 ++++++
   drivers/gpu/drm/lima/lima_ttm.c       | 319 +++++++++++++++++
   drivers/gpu/drm/lima/lima_ttm.h       |  24 ++
   drivers/gpu/drm/lima/lima_vm.c        | 354 +++++++++++++++++++
   drivers/gpu/drm/lima/lima_vm.h        |  59 ++++
   include/uapi/drm/drm_fourcc.h         |   9 +
   include/uapi/drm/lima_drm.h           | 193 ++++++++++
   39 files changed, 5092 insertions(+)

create mode 100644 drivers/gpu/drm/lima/Kconfig create mode 100644 drivers/gpu/drm/lima/Makefile create mode 100644 drivers/gpu/drm/lima/lima_bcast.c create mode 100644 drivers/gpu/drm/lima/lima_bcast.h create mode 100644 drivers/gpu/drm/lima/lima_ctx.c create mode 100644 drivers/gpu/drm/lima/lima_ctx.h create mode 100644 drivers/gpu/drm/lima/lima_device.c create mode 100644 drivers/gpu/drm/lima/lima_device.h create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h create mode 100644 drivers/gpu/drm/lima/lima_drv.c create mode 100644 drivers/gpu/drm/lima/lima_drv.h create mode 100644 drivers/gpu/drm/lima/lima_gem.c create mode 100644 drivers/gpu/drm/lima/lima_gem.h create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h create mode 100644 drivers/gpu/drm/lima/lima_gp.c create mode 100644 drivers/gpu/drm/lima/lima_gp.h create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h create mode 100644 drivers/gpu/drm/lima/lima_mmu.c create mode 100644 drivers/gpu/drm/lima/lima_mmu.h create mode 100644 drivers/gpu/drm/lima/lima_object.c create mode 100644 drivers/gpu/drm/lima/lima_object.h create mode 100644 drivers/gpu/drm/lima/lima_pmu.c create mode 100644 drivers/gpu/drm/lima/lima_pmu.h create mode 100644 drivers/gpu/drm/lima/lima_pp.c create mode 100644 drivers/gpu/drm/lima/lima_pp.h create mode 100644 drivers/gpu/drm/lima/lima_regs.h create mode 100644 drivers/gpu/drm/lima/lima_sched.c create mode 100644 drivers/gpu/drm/lima/lima_sched.h create mode 100644 drivers/gpu/drm/lima/lima_ttm.c create mode 100644 drivers/gpu/drm/lima/lima_ttm.h create mode 100644 drivers/gpu/drm/lima/lima_vm.c create mode 100644 drivers/gpu/drm/lima/lima_vm.h create mode 100644 include/uapi/drm/lima_drm.h

--
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#29 lore raw
In reply to Daniel Vetter
Quoted 37 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

Quoted 6 lines

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

Quoted 3 lines

From an upstream pov having all soc gpu drivers use similar approaches should help with shared infrastructure and stuff like that.

Hope GEM gets more helpers now if Iima have to use it.

Quoted 4 lines

Another one: What's the plan with extending this to panfrost? Or are the architectures for command submission totally different, and we'll need separate kernel drivers for utgard/midgard/bifrost?

+ Alyssa & Rob
There is a gitlab issue about sharing kernel driver:
https://gitlab.freedesktop.org/panfrost/linux/issues/1

But seems utgard (discrete shader arch) really differs with midgard/bifrost (unified shader arch), so that ARM also have different official kernel drivers.

Thanks,
Qiang

Quoted 104 lines

Thanks, Daniel

Quoted 101 lines

- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

This patch serial is based on 5.0-rc5 and squash all the commits. For whole history of this driver's development, see:
https://gitlab.freedesktop.org/lima/linux/commits/lima-5.0-rc5
https://gitlab.freedesktop.org/lima/linux/commits/lima-4.17-rc4

Mesa driver is still in development and not ready for daily usage, but can run some simple tests like kmscube and glamrk2, and some single full screen application like kodi-gbm, see:
https://gitlab.freedesktop.org/lima/mesa

[rfc]
https://lists.freedesktop.org/archives/dri-devel/2018-May/177314.html

Lima Project Developers (1):
drm/lima: driver for ARM Mali4xx GPUs

Qiang Yu (1):
drm/fourcc: add ARM tiled format modifier

 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  22 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  46 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       | 124 +++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  33 ++
 drivers/gpu/drm/lima/lima_device.c    | 384 ++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 116 ++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  56 +++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 +
 drivers/gpu/drm/lima/lima_drv.c       | 459 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  59 ++++
 drivers/gpu/drm/lima/lima_gem.c       | 485 +++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c | 144 ++++++++
 drivers/gpu/drm/lima/lima_gem_prime.h |  18 +
 drivers/gpu/drm/lima/lima_gp.c        | 280 +++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  79 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 135 +++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 103 ++++++
 drivers/gpu/drm/lima/lima_object.h    |  72 ++++
 drivers/gpu/drm/lima/lima_pmu.c       |  61 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 419 ++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_pp.h        |  19 +
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 486 ++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 108 ++++++
 drivers/gpu/drm/lima/lima_ttm.c       | 319 +++++++++++++++++
 drivers/gpu/drm/lima/lima_ttm.h       |  24 ++
 drivers/gpu/drm/lima/lima_vm.c        | 354 +++++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  59 ++++
 include/uapi/drm/drm_fourcc.h         |   9 +
 include/uapi/drm/lima_drm.h           | 193 ++++++++++
 39 files changed, 5092 insertions(+)
 create mode 100644 drivers/gpu/drm/lima/Kconfig
 create mode 100644 drivers/gpu/drm/lima/Makefile
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.c
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.h
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.c
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.h
 create mode 100644 drivers/gpu/drm/lima/lima_device.c
 create mode 100644 drivers/gpu/drm/lima/lima_device.h
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h
 create mode 100644 drivers/gpu/drm/lima/lima_drv.c
 create mode 100644 drivers/gpu/drm/lima/lima_drv.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h
 create mode 100644 drivers/gpu/drm/lima/lima_gp.c
 create mode 100644 drivers/gpu/drm/lima/lima_gp.h
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_object.c
 create mode 100644 drivers/gpu/drm/lima/lima_object.h
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_pp.c
 create mode 100644 drivers/gpu/drm/lima/lima_pp.h
 create mode 100644 drivers/gpu/drm/lima/lima_regs.h
 create mode 100644 drivers/gpu/drm/lima/lima_sched.c
 create mode 100644 drivers/gpu/drm/lima/lima_sched.h
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.c
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.h
 create mode 100644 drivers/gpu/drm/lima/lima_vm.c
 create mode 100644 drivers/gpu/drm/lima/lima_vm.h
 create mode 100644 include/uapi/drm/lima_drm.h
Quoted 8 lines
Quoted 3 lines

--
2.17.1

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#30 lore raw
Quoted 9 lines
Quoted 3 lines

Another one: What's the plan with extending this to panfrost? Or are the architectures for command submission totally different, and we'll need separate kernel drivers for utgard/midgard/bifrost?

+ Alyssa & Rob
There is a gitlab issue about sharing kernel driver:
https://gitlab.freedesktop.org/panfrost/linux/issues/1

But seems utgard (discrete shader arch) really differs with midgard/bifrost (unified shader arch), so that ARM also have different official kernel drivers.

We're operating off that Utgard and Midgard/Bifrost are two completely separate architectures. They share a name, but not much else in practice. Separate shaders vs unified shaders is one distinction; command streams vs state descriptors is another huge one (as far as userspace is concerned); etc. As Qiang pointed out, Arm has two (kernel) drivers, one for Utgard, one for Midgard.

Beyond a few isolated routines (e.g. texture tiling), IMO sharing between lima and panfrost makes about as much sense as sharing between lima and etnaviv/vc4/freedreno. Yeah, there are similarities, but that's what common code is for, not for forcing code cohabitation.

-Alyssa
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#31 lore raw
In reply to Qiang Yu
Quoted 43 lines · Qiang Yu
Quoted 42 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

Quoted 8 lines
Quoted 2 lines

From an upstream pov having all soc gpu drivers use similar approaches should help with shared infrastructure and stuff like that.

Hope GEM gets more helpers now if Iima have to use it.

Quoted 4 lines

Another one: What's the plan with extending this to panfrost? Or are the architectures for command submission totally different, and we'll need separate kernel drivers for utgard/midgard/bifrost?

Quoted 6 lines

+ Alyssa & Rob
There is a gitlab issue about sharing kernel driver:
https://gitlab.freedesktop.org/panfrost/linux/issues/1

But seems utgard (discrete shader arch) really differs with midgard/bifrost (unified shader arch), so that ARM also have different official kernel drivers.

Ok, makes sense, just wanted to check.
-Daniel

Quoted 118 lines

Thanks,
Qiang

Quoted 112 lines

Thanks, Daniel

Quoted 101 lines

- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

This patch serial is based on 5.0-rc5 and squash all the commits. For whole history of this driver's development, see:
https://gitlab.freedesktop.org/lima/linux/commits/lima-5.0-rc5
https://gitlab.freedesktop.org/lima/linux/commits/lima-4.17-rc4

Mesa driver is still in development and not ready for daily usage, but can run some simple tests like kmscube and glamrk2, and some single full screen application like kodi-gbm, see:
https://gitlab.freedesktop.org/lima/mesa

[rfc]
https://lists.freedesktop.org/archives/dri-devel/2018-May/177314.html

Lima Project Developers (1):
drm/lima: driver for ARM Mali4xx GPUs

Qiang Yu (1):
drm/fourcc: add ARM tiled format modifier

 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  22 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  46 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       | 124 +++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  33 ++
 drivers/gpu/drm/lima/lima_device.c    | 384 ++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 116 ++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  56 +++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 +
 drivers/gpu/drm/lima/lima_drv.c       | 459 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  59 ++++
 drivers/gpu/drm/lima/lima_gem.c       | 485 +++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c | 144 ++++++++
 drivers/gpu/drm/lima/lima_gem_prime.h |  18 +
 drivers/gpu/drm/lima/lima_gp.c        | 280 +++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  79 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 135 +++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 103 ++++++
 drivers/gpu/drm/lima/lima_object.h    |  72 ++++
 drivers/gpu/drm/lima/lima_pmu.c       |  61 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 419 ++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_pp.h        |  19 +
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 486 ++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 108 ++++++
 drivers/gpu/drm/lima/lima_ttm.c       | 319 +++++++++++++++++
 drivers/gpu/drm/lima/lima_ttm.h       |  24 ++
 drivers/gpu/drm/lima/lima_vm.c        | 354 +++++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  59 ++++
 include/uapi/drm/drm_fourcc.h         |   9 +
 include/uapi/drm/lima_drm.h           | 193 ++++++++++
 39 files changed, 5092 insertions(+)
 create mode 100644 drivers/gpu/drm/lima/Kconfig
 create mode 100644 drivers/gpu/drm/lima/Makefile
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.c
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.h
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.c
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.h
 create mode 100644 drivers/gpu/drm/lima/lima_device.c
 create mode 100644 drivers/gpu/drm/lima/lima_device.h
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h
 create mode 100644 drivers/gpu/drm/lima/lima_drv.c
 create mode 100644 drivers/gpu/drm/lima/lima_drv.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h
 create mode 100644 drivers/gpu/drm/lima/lima_gp.c
 create mode 100644 drivers/gpu/drm/lima/lima_gp.h
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_object.c
 create mode 100644 drivers/gpu/drm/lima/lima_object.h
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_pp.c
 create mode 100644 drivers/gpu/drm/lima/lima_pp.h
 create mode 100644 drivers/gpu/drm/lima/lima_regs.h
 create mode 100644 drivers/gpu/drm/lima/lima_sched.c
 create mode 100644 drivers/gpu/drm/lima/lima_sched.h
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.c
 create mode 100644 drivers/gpu/drm/lima/lima_ttm.h
 create mode 100644 drivers/gpu/drm/lima/lima_vm.c
 create mode 100644 drivers/gpu/drm/lima/lima_vm.h
 create mode 100644 include/uapi/drm/lima_drm.h
Quoted 3 lines

--
2.17.1

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#32 lore raw
Quoted 58 lines · Daniel Vetter
Quoted 43 lines · Qiang Yu
Quoted 42 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Quoted 6 lines

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Rob
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#33 lore raw
Quoted 66 lines · Rob Herring
Quoted 58 lines · Daniel Vetter
Quoted 43 lines · Qiang Yu
Quoted 42 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Quoted 6 lines

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

Quoted 4 lines

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Did you just convert the shmem helpers over to doing alloc_coherent? If so, I'd be interested in picking them up for v3d, and that might help get another patch out of your stack.

I'm particularly interested in the shmem helpers because I should start doing dynamic binding in and out of the GPU's page table, to avoid pinning so much memory all the time.

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#34 lore raw
Quoted 69 lines · Eric Anholt
Quoted 66 lines · Rob Herring
Quoted 64 lines · Daniel Vetter
Quoted 43 lines · Qiang Yu
Quoted 42 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

Quoted 8 lines
Quoted 4 lines

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Did you just convert the shmem helpers over to doing alloc_coherent? If so, I'd be interested in picking them up for v3d, and that might help get another patch out of your stack.

I haven't really fully addressed that yet, but yeah, my plan is just to switch to WC alloc and mappings. I was going to try to make it configurable, but there is a comment in the ARM dma mapping code which makes me wonder if tinydrm using streaming DMA for SPI is fundamentally broken (and maybe CMA is less broken?). If not broken, not guaranteed to work.

/*
* The whole dma_get_sgtable() idea is fundamentally unsafe - it seems
* that the intention is to allow exporting memory allocated via the
* coherent DMA APIs through the dma_buf API, which only accepts a
* scattertable. This presents a couple of problems:
* 1. Not all memory allocated via the coherent DMA APIs is backed by
* a struct page
* 2. Passing coherent DMA memory into the streaming APIs is not allowed
* as we will try to flush the memory through a different alias to that
* actually being used (and the flushes are redundant.)
*/

Quoted 3 lines

I'm particularly interested in the shmem helpers because I should start doing dynamic binding in and out of the GPU's page table, to avoid pinning so much memory all the time.

I'll try to post something in the next couple of days.

Rob
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#35 lore raw
Quoted 87 lines · Rob Herring
Quoted 69 lines · Eric Anholt
Quoted 66 lines · Rob Herring
Quoted 64 lines · Daniel Vetter
Quoted 43 lines · Qiang Yu
Quoted 42 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

Quoted 8 lines
Quoted 4 lines

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Did you just convert the shmem helpers over to doing alloc_coherent? If so, I'd be interested in picking them up for v3d, and that might help get another patch out of your stack.

I haven't really fully addressed that yet, but yeah, my plan is just to switch to WC alloc and mappings. I was going to try to make it configurable, but there is a comment in the ARM dma mapping code which makes me wonder if tinydrm using streaming DMA for SPI is fundamentally broken (and maybe CMA is less broken?). If not broken, not guaranteed to work.

Quoted 11 lines

/*
* The whole dma_get_sgtable() idea is fundamentally unsafe - it seems
* that the intention is to allow exporting memory allocated via the
* coherent DMA APIs through the dma_buf API, which only accepts a
* scattertable. This presents a couple of problems:
* 1. Not all memory allocated via the coherent DMA APIs is backed by
* a struct page
* 2. Passing coherent DMA memory into the streaming APIs is not allowed
* as we will try to flush the memory through a different alias to that
* actually being used (and the flushes are redundant.)
*/

The sg table is only for device access, which avoids both of these issues. That's the idea at least, except all ttm-based drivers don't care, instead they expect a struct page and then use that to build a ttm_bo. And then use all the ttm cpu side access functions, instead of using the dma-buf interfaces (which need to exist to avoid the above issues).

So except if you want to fix ttm dma-buf import (which is going to be a pile of work), add this to the list of why ttm is probably not the best choice for something mostly running on arm soc. x86 gets away because dma is easy on x86.
-Daniel

Quoted 7 lines
Quoted 3 lines

I'm particularly interested in the shmem helpers because I should start doing dynamic binding in and out of the GPU's page table, to avoid pinning so much memory all the time.

I'll try to post something in the next couple of days.

Rob

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#36 lore raw

Am 13.02.19 um 08:59 schrieb Daniel Vetter:

Quoted 90 lines · Rob Herring
Quoted 78 lines · Eric Anholt
Quoted 63 lines
Quoted 61 lines · Rob Herring
Quoted 60 lines · Daniel Vetter
Quoted 41 lines · Qiang Yu
Quoted 40 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

Quoted 7 lines
Quoted 4 lines

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Did you just convert the shmem helpers over to doing alloc_coherent? If so, I'd be interested in picking them up for v3d, and that might help get another patch out of your stack.

I haven't really fully addressed that yet, but yeah, my plan is just to switch to WC alloc and mappings. I was going to try to make it configurable, but there is a comment in the ARM dma mapping code which makes me wonder if tinydrm using streaming DMA for SPI is fundamentally broken (and maybe CMA is less broken?). If not broken, not guaranteed to work.

Quoted 11 lines

/*
* The whole dma_get_sgtable() idea is fundamentally unsafe - it seems
* that the intention is to allow exporting memory allocated via the
* coherent DMA APIs through the dma_buf API, which only accepts a
* scattertable. This presents a couple of problems:
* 1. Not all memory allocated via the coherent DMA APIs is backed by
* a struct page
* 2. Passing coherent DMA memory into the streaming APIs is not allowed
* as we will try to flush the memory through a different alias to that
* actually being used (and the flushes are redundant.)
*/

Quoted 6 lines

The sg table is only for device access, which avoids both of these issues. That's the idea at least, except all ttm-based drivers don't care, instead they expect a struct page and then use that to build a ttm_bo. And then use all the ttm cpu side access functions, instead of using the dma-buf interfaces (which need to exist to avoid the above issues).

Actually that is not correct any more. I've fixed this while working on directly sharing BOs between amdgpu devices.

TTM now uses the DMA addresses from the sg table and I actually wanted to remove the pages for imported DMA-buf BOs for a while now.

Regards,
Christian.

Quoted 15 lines

So except if you want to fix ttm dma-buf import (which is going to be a pile of work), add this to the list of why ttm is probably not the best choice for something mostly running on arm soc. x86 gets away because dma is easy on x86.
-Daniel

Quoted 6 lines
Quoted 3 lines

I'm particularly interested in the shmem helpers because I should start doing dynamic binding in and out of the GPU's page table, to avoid pinning so much memory all the time.

I'll try to post something in the next couple of days.

Rob

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#37 lore raw
Quoted 98 lines · Christian König

Am 13.02.19 um 08:59 schrieb Daniel Vetter:

Quoted 90 lines · Rob Herring
Quoted 78 lines · Eric Anholt
Quoted 63 lines
Quoted 61 lines · Rob Herring
Quoted 60 lines · Daniel Vetter
Quoted 41 lines · Qiang Yu
Quoted 40 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

Quoted 7 lines
Quoted 4 lines

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Did you just convert the shmem helpers over to doing alloc_coherent? If so, I'd be interested in picking them up for v3d, and that might help get another patch out of your stack.

I haven't really fully addressed that yet, but yeah, my plan is just to switch to WC alloc and mappings. I was going to try to make it configurable, but there is a comment in the ARM dma mapping code which makes me wonder if tinydrm using streaming DMA for SPI is fundamentally broken (and maybe CMA is less broken?). If not broken, not guaranteed to work.

Quoted 11 lines

/*
* The whole dma_get_sgtable() idea is fundamentally unsafe - it seems
* that the intention is to allow exporting memory allocated via the
* coherent DMA APIs through the dma_buf API, which only accepts a
* scattertable. This presents a couple of problems:
* 1. Not all memory allocated via the coherent DMA APIs is backed by
* a struct page
* 2. Passing coherent DMA memory into the streaming APIs is not allowed
* as we will try to flush the memory through a different alias to that
* actually being used (and the flushes are redundant.)
*/

Quoted 6 lines

The sg table is only for device access, which avoids both of these issues. That's the idea at least, except all ttm-based drivers don't care, instead they expect a struct page and then use that to build a ttm_bo. And then use all the ttm cpu side access functions, instead of using the dma-buf interfaces (which need to exist to avoid the above issues).

Quoted 5 lines

Actually that is not correct any more. I've fixed this while working on directly sharing BOs between amdgpu devices.

TTM now uses the DMA addresses from the sg table and I actually wanted to remove the pages for imported DMA-buf BOs for a while now.

Nice! And yeah it's been a while since I looked at this ... So just a bit of cleanup work left to do, fundamentals are in place. Shouldn't be too hard to get rid of the pages, since the dma-buf cpu accessor functions have been modelled after the ttm_bo interfaces.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#38 lore raw

Am 13.02.19 um 10:38 schrieb Daniel Vetter:

Quoted 101 lines · Christian König
Quoted 100 lines

Am 13.02.19 um 08:59 schrieb Daniel Vetter:

Quoted 90 lines · Rob Herring
Quoted 78 lines · Eric Anholt
Quoted 63 lines
Quoted 61 lines · Rob Herring
Quoted 60 lines · Daniel Vetter
Quoted 41 lines · Qiang Yu
Quoted 40 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

Quoted 7 lines
Quoted 4 lines

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Did you just convert the shmem helpers over to doing alloc_coherent? If so, I'd be interested in picking them up for v3d, and that might help get another patch out of your stack.

I haven't really fully addressed that yet, but yeah, my plan is just to switch to WC alloc and mappings. I was going to try to make it configurable, but there is a comment in the ARM dma mapping code which makes me wonder if tinydrm using streaming DMA for SPI is fundamentally broken (and maybe CMA is less broken?). If not broken, not guaranteed to work.

Quoted 11 lines

/*
* The whole dma_get_sgtable() idea is fundamentally unsafe - it seems
* that the intention is to allow exporting memory allocated via the
* coherent DMA APIs through the dma_buf API, which only accepts a
* scattertable. This presents a couple of problems:
* 1. Not all memory allocated via the coherent DMA APIs is backed by
* a struct page
* 2. Passing coherent DMA memory into the streaming APIs is not allowed
* as we will try to flush the memory through a different alias to that
* actually being used (and the flushes are redundant.)
*/

Quoted 6 lines

The sg table is only for device access, which avoids both of these issues. That's the idea at least, except all ttm-based drivers don't care, instead they expect a struct page and then use that to build a ttm_bo. And then use all the ttm cpu side access functions, instead of using the dma-buf interfaces (which need to exist to avoid the above issues).

Actually that is not correct any more. I've fixed this while working on directly sharing BOs between amdgpu devices.

Quoted 6 lines
Quoted 2 lines

TTM now uses the DMA addresses from the sg table and I actually wanted to remove the pages for imported DMA-buf BOs for a while now.

Nice! And yeah it's been a while since I looked at this ... So just a bit of cleanup work left to do, fundamentals are in place. Shouldn't be too hard to get rid of the pages, since the dma-buf cpu accessor functions have been modelled after the ttm_bo interfaces.

Well at least in radeon and amdgpu CPU mapping an imported BO is forbidden (userspace directly maps the DMA-buf fd).

The only case left is mapping a BO in the kernel, and that in turn is only used in very few places in radeon/amdgpu:
1. Command stream patching.
2. CPU based page table updates.
3. Debugging

And I think all of them doesn't make sense on a DMA-buf imported BO.

Regards,
Christian.

Quoted 1 line

-Daniel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#39 lore raw
In reply to Christian König via dri-devel
Quoted 99 lines · Christian König

Am 13.02.19 um 08:59 schrieb Daniel Vetter:

Quoted 90 lines · Rob Herring
Quoted 78 lines · Eric Anholt
Quoted 63 lines
Quoted 61 lines · Rob Herring
Quoted 60 lines · Daniel Vetter
Quoted 41 lines · Qiang Yu
Quoted 40 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

Quoted 7 lines
Quoted 4 lines

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Did you just convert the shmem helpers over to doing alloc_coherent? If so, I'd be interested in picking them up for v3d, and that might help get another patch out of your stack.

I haven't really fully addressed that yet, but yeah, my plan is just to switch to WC alloc and mappings. I was going to try to make it configurable, but there is a comment in the ARM dma mapping code which makes me wonder if tinydrm using streaming DMA for SPI is fundamentally broken (and maybe CMA is less broken?). If not broken, not guaranteed to work.

Quoted 11 lines

/*
* The whole dma_get_sgtable() idea is fundamentally unsafe - it seems
* that the intention is to allow exporting memory allocated via the
* coherent DMA APIs through the dma_buf API, which only accepts a
* scattertable. This presents a couple of problems:
* 1. Not all memory allocated via the coherent DMA APIs is backed by
* a struct page
* 2. Passing coherent DMA memory into the streaming APIs is not allowed
* as we will try to flush the memory through a different alias to that
* actually being used (and the flushes are redundant.)
*/

Quoted 6 lines

The sg table is only for device access, which avoids both of these issues. That's the idea at least, except all ttm-based drivers don't care, instead they expect a struct page and then use that to build a ttm_bo. And then use all the ttm cpu side access functions, instead of using the dma-buf interfaces (which need to exist to avoid the above issues).

Quoted 5 lines

Actually that is not correct any more. I've fixed this while working on directly sharing BOs between amdgpu devices.

TTM now uses the DMA addresses from the sg table and I actually wanted to remove the pages for imported DMA-buf BOs for a while now.

Finally gotten around to reading ttm code to update my understanding, and I think I realized why I never realized this changed:
TTM_PAGE_FLAG_SG and related code seems to be the fancy new code you added to go sg table native in ttm, and from a quick look rolled out everywhere. But drm_prime_sg_to_page_addr_arrays is still called. Is that the missing cleanup you're referring to? Would be nice if we could nuke it to stop the copypasta spread (and spreading it seems to do :-/). Maybe as a todo.rst entry?

Cheers, Daniel

Quoted 20 lines

Regards,
Christian.

Quoted 15 lines

So except if you want to fix ttm dma-buf import (which is going to be a pile of work), add this to the list of why ttm is probably not the best choice for something mostly running on arm soc. x86 gets away because dma is easy on x86.
-Daniel

Quoted 6 lines
Quoted 3 lines

I'm particularly interested in the shmem helpers because I should start doing dynamic binding in and out of the GPU's page table, to avoid pinning so much memory all the time.

I'll try to post something in the next couple of days.

Rob

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#40 lore raw

Am 26.02.19 um 16:58 schrieb Daniel Vetter:

Quoted 102 lines
Quoted 100 lines · Christian König

Am 13.02.19 um 08:59 schrieb Daniel Vetter:

Quoted 90 lines · Rob Herring
Quoted 78 lines · Eric Anholt
Quoted 63 lines
Quoted 61 lines · Rob Herring
Quoted 60 lines · Daniel Vetter
Quoted 41 lines · Qiang Yu
Quoted 40 lines · Daniel Vetter
Quoted 26 lines · Qiang Yu

Kernel DRM driver for ARM Mali 400/450 GPUs.

Since last RFC, all feedback has been addressed. Most Mali DTS changes are already upstreamed by SoC maintainers. The kernel driver and user-kernel interface are quite stable for several months, so I think it's ready to be upstreamed.

This implementation mainly take amdgpu DRM driver as reference.

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user

Quoted 8 lines

process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use TTM as MM. TTM_PL_TT type memory is used as the content of lima buffer object which is allocated from TTM page pool. all lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when allocation, so there's no buffer eviction and swap for now.

All other render gpu drivers that have unified memory (aka is on the SoC) use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4 (and i915 is kinda the same too really). TTM makes sense if you have some discrete memory to manage, but imo not in any other place really.

Quoted 7 lines
Quoted 1 line

What's the design choice behind this?

To be honest, it's just because TTM offers more helpers. I did implement a GEM way with cma alloc at the beginning. But when implement paged mem, I found TTM has mem pool alloc, sync and mmap related helpers which covers much of my existing code. It's totally possible with GEM, but not as easy as TTM to me. And virtio-gpu seems an example to use TTM without discrete mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM without the vram migration, it's just that most of the complexity of TTM is due to buffer placement and migration and all that stuff. If you never need to move buffers, then you don't need that ever.

Wrt lack of helpers, what exactly are you looking for? A big part of these for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing the lima kernel driver on vc4, freedreno or etnaviv (last one is probably closest, since it doesn't have a display block either) would be better I think.

Quoted 7 lines
Quoted 4 lines

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Did you just convert the shmem helpers over to doing alloc_coherent? If so, I'd be interested in picking them up for v3d, and that might help get another patch out of your stack.

I haven't really fully addressed that yet, but yeah, my plan is just to switch to WC alloc and mappings. I was going to try to make it configurable, but there is a comment in the ARM dma mapping code which makes me wonder if tinydrm using streaming DMA for SPI is fundamentally broken (and maybe CMA is less broken?). If not broken, not guaranteed to work.

Quoted 11 lines

/*
* The whole dma_get_sgtable() idea is fundamentally unsafe - it seems
* that the intention is to allow exporting memory allocated via the
* coherent DMA APIs through the dma_buf API, which only accepts a
* scattertable. This presents a couple of problems:
* 1. Not all memory allocated via the coherent DMA APIs is backed by
* a struct page
* 2. Passing coherent DMA memory into the streaming APIs is not allowed
* as we will try to flush the memory through a different alias to that
* actually being used (and the flushes are redundant.)
*/

Quoted 6 lines

The sg table is only for device access, which avoids both of these issues. That's the idea at least, except all ttm-based drivers don't care, instead they expect a struct page and then use that to build a ttm_bo. And then use all the ttm cpu side access functions, instead of using the dma-buf interfaces (which need to exist to avoid the above issues).

Actually that is not correct any more. I've fixed this while working on directly sharing BOs between amdgpu devices.

Quoted 8 lines
Quoted 2 lines

TTM now uses the DMA addresses from the sg table and I actually wanted to remove the pages for imported DMA-buf BOs for a while now.

Finally gotten around to reading ttm code to update my understanding, and I think I realized why I never realized this changed:
TTM_PAGE_FLAG_SG and related code seems to be the fancy new code you added to go sg table native in ttm, and from a quick look rolled out everywhere. But drm_prime_sg_to_page_addr_arrays is still called. Is that the missing cleanup you're referring to?

Yes, exactly. The last thing I pushed upstream was making the pages optional in drm_prime_sg_to_page_addr_arrays.

I just never got around to really not fill ttm->pages any more, but in theory it should be possible to just comment that out and be happy about it.

Christian.

Quoted 4 lines

Would be nice if we
could nuke it to stop the copypasta spread (and spreading it seems to do :-/). Maybe as a todo.rst entry?

Quoted 19 lines

Cheers, Daniel

Quoted 16 lines

Regards,
Christian.

Quoted 13 lines

So except if you want to fix ttm dma-buf import (which is going to be a pile of work), add this to the list of why ttm is probably not the best choice for something mostly running on arm soc. x86 gets away because dma is easy on x86.
-Daniel

Quoted 6 lines
Quoted 3 lines

I'm particularly interested in the shmem helpers because I should start doing dynamic binding in and out of the GPU's page table, to avoid pinning so much memory all the time.

I'll try to post something in the next couple of days.

Rob

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#41 lore raw
In reply to Rob Herring via dri-devel

Den 13.02.2019 02.44, skrev Rob Herring:

Quoted 4 lines · Eric Anholt
Quoted 3 lines

Rob Herring <[email protected]> writes:

[snip]

Quoted 16 lines
Quoted 8 lines
Quoted 4 lines

FWIW, I'm working on the panfrost driver and am using the shmem helpers from Noralf. It's the early stages though. I started a patch for etnaviv to use it too, but found I need to rework it to sub-class the shmem GEM object.

Did you just convert the shmem helpers over to doing alloc_coherent? If so, I'd be interested in picking them up for v3d, and that might help get another patch out of your stack.

I haven't really fully addressed that yet, but yeah, my plan is just to switch to WC alloc and mappings. I was going to try to make it configurable, but there is a comment in the ARM dma mapping code which makes me wonder if tinydrm using streaming DMA for SPI is fundamentally broken (and maybe CMA is less broken?). If not broken, not guaranteed to work.

Quoted 12 lines

/*
* The whole dma_get_sgtable() idea is fundamentally unsafe - it seems
* that the intention is to allow exporting memory allocated via the
* coherent DMA APIs through the dma_buf API, which only accepts a
* scattertable. This presents a couple of problems:
* 1. Not all memory allocated via the coherent DMA APIs is backed by
* a struct page
* 2. Passing coherent DMA memory into the streaming APIs is not allowed
* as we will try to flush the memory through a different alias to that
* actually being used (and the flushes are redundant.)
*/

Thanks for drawing my attention to this, I wasn't aware of it. Sadly the SPI subsystem doesn't have a way to pass in dma buffers, everything has to go through the streaming API. Long term I guess I have to add support for that.

Noralf.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel