Skip to contentSkip to conversation

drm/lima: driver for ARM Mali4xx GPUs

PATCH Superseded Qiang Yu 4 participants 9 messages Mar 6, 2019 – Mar 7, 2019

Subsystems (3)

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore

Qiang Yu

author maintainer
#0 lore raw

PATCH v7 2/2 drm/lima: driver for ARM Mali4xx GPUs

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use GEM+shmem for MM. Currently just alloc and pin memory when gem object creation. GPU vm map of the buffer is also done in the alloc stage in kernel space. We may delay the memory allocation and real GPU vm map to command submission stage in the furture as improvement.
- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

mesa driver can be found here before upstreamed:
https://gitlab.freedesktop.org/lima/mesa

v7:
- remove lima_fence_ops with default value
- move fence slab create to device probe
- check pad ioctl args to be zero
- add comments for user/kernel interface

v6:
- fix comments by checkpatch.pl

v5:
- export gp/pp version to userspace
- rebase on drm-misc-next

v4:
- use get param interface to get info
- separate context create/free ioctl
- remove unused max sched task param
- update copyright time
- use xarray instead of idr
- stop using drmP.h

v3:
- fix comments from kbuild robot
- restrict supported arch to tested ones

v2:
- fix syscall argument check
- fix job finish fence leak since kernel 5.0
- use drm syncobj to replace native fence
- move buffer object GPU va map into kernel
- reserve syscall argument space for future info
- remove kernel gem modifier
- switch TTM back to GEM+shmem MM
- use time based io poll
- use whole register name
- adopt gem reservation obj integration
- use drm_timeout_abs_to_jiffies

---
 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  21 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  47 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       |  97 ++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  30 ++
 drivers/gpu/drm/lima/lima_device.c    | 385 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 131 ++++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  58 ++++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 ++
 drivers/gpu/drm/lima/lima_drv.c       | 376 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  45 +++
 drivers/gpu/drm/lima/lima_gem.c       | 381 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c |  47 +++
 drivers/gpu/drm/lima/lima_gem_prime.h |  13 +
 drivers/gpu/drm/lima/lima_gp.c        | 283 +++++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  80 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 142 +++++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 122 ++++++++
 drivers/gpu/drm/lima/lima_object.h    |  36 +++
 drivers/gpu/drm/lima/lima_pmu.c       |  60 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 427 ++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_pp.h        |  19 ++
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 404 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 104 +++++++
 drivers/gpu/drm/lima/lima_vm.c        | 282 +++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  62 ++++
 include/uapi/drm/lima_drm.h           | 164 ++++++++++
 36 files changed, 4242 insertions(+)
 create mode 100644 drivers/gpu/drm/lima/Kconfig
 create mode 100644 drivers/gpu/drm/lima/Makefile
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.c
 create mode 100644 drivers/gpu/drm/lima/lima_bcast.h
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.c
 create mode 100644 drivers/gpu/drm/lima/lima_ctx.h
 create mode 100644 drivers/gpu/drm/lima/lima_device.c
 create mode 100644 drivers/gpu/drm/lima/lima_device.h
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c
 create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h
 create mode 100644 drivers/gpu/drm/lima/lima_drv.c
 create mode 100644 drivers/gpu/drm/lima/lima_drv.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem.h
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c
 create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h
 create mode 100644 drivers/gpu/drm/lima/lima_gp.c
 create mode 100644 drivers/gpu/drm/lima/lima_gp.h
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c
 create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_mmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_object.c
 create mode 100644 drivers/gpu/drm/lima/lima_object.h
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.c
 create mode 100644 drivers/gpu/drm/lima/lima_pmu.h
 create mode 100644 drivers/gpu/drm/lima/lima_pp.c
 create mode 100644 drivers/gpu/drm/lima/lima_pp.h
 create mode 100644 drivers/gpu/drm/lima/lima_regs.h
 create mode 100644 drivers/gpu/drm/lima/lima_sched.c
 create mode 100644 drivers/gpu/drm/lima/lima_sched.h
 create mode 100644 drivers/gpu/drm/lima/lima_vm.c
 create mode 100644 drivers/gpu/drm/lima/lima_vm.h
 create mode 100644 include/uapi/drm/lima_drm.h
drivers/gpu/drm/Kconfig +2 -0
index bd943a71756c..1716e420baf9 100644
@@ -329,6 +329,8 @@ source "drivers/gpu/drm/tve200/Kconfig"
 
 source "drivers/gpu/drm/xen/Kconfig"
 
+source "drivers/gpu/drm/lima/Kconfig"
+
 # Keep legacy drivers last
 
 menuconfig DRM_LEGACY
drivers/gpu/drm/Makefile +1 -0
index 1ac55c65eac0..1f9d0076f99a 100644
@@ -109,3 +109,4 @@ obj-$(CONFIG_DRM_TINYDRM) += tinydrm/
 obj-$(CONFIG_DRM_PL111) += pl111/
 obj-$(CONFIG_DRM_TVE200) += tve200/
 obj-$(CONFIG_DRM_XEN) += xen/
+obj-$(CONFIG_DRM_LIMA)  += lima/
drivers/gpu/drm/lima/Kconfig new file +10 -0
new file mode 100644
index 000000000000..f11314448093
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0 OR MIT
+# Copyright 2017-2019 Qiang Yu <[email protected]>
+
+config DRM_LIMA
+       tristate "LIMA (DRM support for ARM Mali 400/450 GPU)"
+       depends on DRM
+       depends on ARM || ARM64 || COMPILE_TEST
+       select DRM_SCHED
+       help
+         DRM driver for ARM Mali 400/450 GPUs.
drivers/gpu/drm/lima/Makefile new file +21 -0
new file mode 100644
index 000000000000..38cc70281ba5
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0 OR MIT
+# Copyright 2017-2019 Qiang Yu <[email protected]>
+
+lima-y := \
+	lima_drv.o \
+	lima_device.o \
+	lima_pmu.o \
+	lima_l2_cache.o \
+	lima_mmu.o \
+	lima_gp.o \
+	lima_pp.o \
+	lima_gem.o \
+	lima_vm.o \
+	lima_sched.o \
+	lima_ctx.o \
+	lima_gem_prime.o \
+	lima_dlbu.o \
+	lima_bcast.o \
+	lima_object.o
+
+obj-$(CONFIG_DRM_LIMA) += lima.o
drivers/gpu/drm/lima/lima_bcast.c new file +47 -0

47 lines not on this page · raw message

drivers/gpu/drm/lima/lima_bcast.h new file +14 -0
new file mode 100644
index 000000000000..c47e58563d0a
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_BCAST_H__
+#define __LIMA_BCAST_H__
+
+struct lima_ip;
+
+int lima_bcast_init(struct lima_ip *ip);
+void lima_bcast_fini(struct lima_ip *ip);
+
+void lima_bcast_enable(struct lima_device *dev, int num_pp);
+
+#endif
drivers/gpu/drm/lima/lima_ctx.c new file +97 -0

97 lines not on this page · raw message

drivers/gpu/drm/lima/lima_ctx.h new file +30 -0
new file mode 100644
index 000000000000..6154e5c9bfe4
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_CTX_H__
+#define __LIMA_CTX_H__
+
+#include <linux/xarray.h>
+
+#include "lima_device.h"
+
+struct lima_ctx {
+	struct kref refcnt;
+	struct lima_device *dev;
+	struct lima_sched_context context[lima_pipe_num];
+	atomic_t guilty;
+};
+
+struct lima_ctx_mgr {
+	struct mutex lock;
+	struct xarray handles;
+};
+
+int lima_ctx_create(struct lima_device *dev, struct lima_ctx_mgr *mgr, u32 *id);
+int lima_ctx_free(struct lima_ctx_mgr *mgr, u32 id);
+struct lima_ctx *lima_ctx_get(struct lima_ctx_mgr *mgr, u32 id);
+void lima_ctx_put(struct lima_ctx *ctx);
+void lima_ctx_mgr_init(struct lima_ctx_mgr *mgr);
+void lima_ctx_mgr_fini(struct lima_ctx_mgr *mgr);
+
+#endif
drivers/gpu/drm/lima/lima_device.c new file +385 -0

385 lines not on this page · raw message

drivers/gpu/drm/lima/lima_device.h new file +131 -0

131 lines not on this page · raw message

drivers/gpu/drm/lima/lima_dlbu.c new file +58 -0

58 lines not on this page · raw message

drivers/gpu/drm/lima/lima_dlbu.h new file +18 -0
new file mode 100644
index 000000000000..16f877984466
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_DLBU_H__
+#define __LIMA_DLBU_H__
+
+struct lima_ip;
+struct lima_device;
+
+void lima_dlbu_enable(struct lima_device *dev, int num_pp);
+void lima_dlbu_disable(struct lima_device *dev);
+
+void lima_dlbu_set_reg(struct lima_ip *ip, u32 *reg);
+
+int lima_dlbu_init(struct lima_ip *ip);
+void lima_dlbu_fini(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_drv.c new file +376 -0

376 lines not on this page · raw message

drivers/gpu/drm/lima/lima_drv.h new file +45 -0

45 lines not on this page · raw message

drivers/gpu/drm/lima/lima_gem.c new file +381 -0

381 lines not on this page · raw message

drivers/gpu/drm/lima/lima_gem.h new file +25 -0
new file mode 100644
index 000000000000..556111a01135
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_GEM_H__
+#define __LIMA_GEM_H__
+
+struct lima_bo;
+struct lima_submit;
+
+extern const struct vm_operations_struct lima_gem_vm_ops;
+
+struct lima_bo *lima_gem_create_bo(struct drm_device *dev, u32 size, u32 flags);
+int lima_gem_create_handle(struct drm_device *dev, struct drm_file *file,
+			   u32 size, u32 flags, u32 *handle);
+void lima_gem_free_object(struct drm_gem_object *obj);
+int lima_gem_object_open(struct drm_gem_object *obj, struct drm_file *file);
+void lima_gem_object_close(struct drm_gem_object *obj, struct drm_file *file);
+int lima_gem_get_info(struct drm_file *file, u32 handle, u32 *va, u64 *offset);
+int lima_gem_mmap(struct file *filp, struct vm_area_struct *vma);
+int lima_gem_submit(struct drm_file *file, struct lima_submit *submit);
+int lima_gem_wait(struct drm_file *file, u32 handle, u32 op, s64 timeout_ns);
+
+void lima_set_vma_flags(struct vm_area_struct *vma);
+
+#endif
drivers/gpu/drm/lima/lima_gem_prime.c new file +47 -0

47 lines not on this page · raw message

drivers/gpu/drm/lima/lima_gem_prime.h new file +13 -0
new file mode 100644
index 000000000000..34b4d35c21e3
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_GEM_PRIME_H__
+#define __LIMA_GEM_PRIME_H__
+
+struct drm_gem_object *lima_gem_prime_import_sg_table(
+	struct drm_device *dev, struct dma_buf_attachment *attach,
+	struct sg_table *sgt);
+struct sg_table *lima_gem_prime_get_sg_table(struct drm_gem_object *obj);
+int lima_gem_prime_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma);
+
+#endif
drivers/gpu/drm/lima/lima_gp.c new file +283 -0

283 lines not on this page · raw message

drivers/gpu/drm/lima/lima_gp.h new file +16 -0
new file mode 100644
index 000000000000..516e5c1babbb
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_GP_H__
+#define __LIMA_GP_H__
+
+struct lima_ip;
+struct lima_device;
+
+int lima_gp_init(struct lima_ip *ip);
+void lima_gp_fini(struct lima_ip *ip);
+
+int lima_gp_pipe_init(struct lima_device *dev);
+void lima_gp_pipe_fini(struct lima_device *dev);
+
+#endif
drivers/gpu/drm/lima/lima_l2_cache.c new file +80 -0

80 lines not on this page · raw message

drivers/gpu/drm/lima/lima_l2_cache.h new file +14 -0
new file mode 100644
index 000000000000..c63fb676ff14
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_L2_CACHE_H__
+#define __LIMA_L2_CACHE_H__
+
+struct lima_ip;
+
+int lima_l2_cache_init(struct lima_ip *ip);
+void lima_l2_cache_fini(struct lima_ip *ip);
+
+int lima_l2_cache_flush(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_mmu.c new file +142 -0

142 lines not on this page · raw message

drivers/gpu/drm/lima/lima_mmu.h new file +16 -0
new file mode 100644
index 000000000000..8c78319bcc8e
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_MMU_H__
+#define __LIMA_MMU_H__
+
+struct lima_ip;
+struct lima_vm;
+
+int lima_mmu_init(struct lima_ip *ip);
+void lima_mmu_fini(struct lima_ip *ip);
+
+void lima_mmu_switch_vm(struct lima_ip *ip, struct lima_vm *vm);
+void lima_mmu_page_fault_resume(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_object.c new file +122 -0

122 lines not on this page · raw message

drivers/gpu/drm/lima/lima_object.h new file +36 -0
new file mode 100644
index 000000000000..6738724afb7b
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2018-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_OBJECT_H__
+#define __LIMA_OBJECT_H__
+
+#include <drm/drm_gem.h>
+
+#include "lima_device.h"
+
+struct lima_bo {
+	struct drm_gem_object gem;
+
+	struct page **pages;
+	dma_addr_t *pages_dma_addr;
+	struct sg_table *sgt;
+	void *vaddr;
+
+	struct mutex lock;
+	struct list_head va;
+};
+
+static inline struct lima_bo *
+to_lima_bo(struct drm_gem_object *obj)
+{
+	return container_of(obj, struct lima_bo, gem);
+}
+
+struct lima_bo *lima_bo_create(struct lima_device *dev, u32 size,
+			       u32 flags, struct sg_table *sgt,
+			       struct reservation_object *resv);
+void lima_bo_destroy(struct lima_bo *bo);
+void *lima_bo_vmap(struct lima_bo *bo);
+void lima_bo_vunmap(struct lima_bo *bo);
+
+#endif
drivers/gpu/drm/lima/lima_pmu.c new file +60 -0

60 lines not on this page · raw message

drivers/gpu/drm/lima/lima_pmu.h new file +12 -0
new file mode 100644
index 000000000000..a2a18775eb07
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_PMU_H__
+#define __LIMA_PMU_H__
+
+struct lima_ip;
+
+int lima_pmu_init(struct lima_ip *ip);
+void lima_pmu_fini(struct lima_ip *ip);
+
+#endif
drivers/gpu/drm/lima/lima_pp.c new file +427 -0

427 lines not on this page · raw message

drivers/gpu/drm/lima/lima_pp.h new file +19 -0
new file mode 100644
index 000000000000..bf60c77b2633
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2017-2019 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_PP_H__
+#define __LIMA_PP_H__
+
+struct lima_ip;
+struct lima_device;
+
+int lima_pp_init(struct lima_ip *ip);
+void lima_pp_fini(struct lima_ip *ip);
+
+int lima_pp_bcast_init(struct lima_ip *ip);
+void lima_pp_bcast_fini(struct lima_ip *ip);
+
+int lima_pp_pipe_init(struct lima_device *dev);
+void lima_pp_pipe_fini(struct lima_device *dev);
+
+#endif
drivers/gpu/drm/lima/lima_regs.h new file +298 -0

63 lines not on this page · raw message

drivers/gpu/drm/lima/lima_sched.c new file +404 -0

404 lines not on this page · raw message

drivers/gpu/drm/lima/lima_sched.h new file +104 -0

104 lines not on this page · raw message

drivers/gpu/drm/lima/lima_vm.c new file +282 -0

282 lines not on this page · raw message

drivers/gpu/drm/lima/lima_vm.h new file +62 -0

62 lines not on this page · raw message

include/uapi/drm/lima_drm.h new file +164 -0

164 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Part of the diff is not shown here. View the raw message.

Reviewed-by on v7 2/2
#1 lore raw
Quoted 31 lines · Qiang Yu

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use GEM+shmem for MM. Currently just alloc and pin memory when gem object creation. GPU vm map of the buffer is also done in the alloc stage in kernel space. We may delay the memory allocation and real GPU vm map to command submission stage in the furture as improvement.
- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

Quoted 8 lines

mesa driver can be found here before upstreamed:
https://gitlab.freedesktop.org/lima/mesa

v7:
- remove lima_fence_ops with default value
- move fence slab create to device probe
- check pad ioctl args to be zero
- add comments for user/kernel interface

Thanks for adding the comments! That helps a lot. I feel pretty good about the ABI at this point.

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Reviewed-by on v7 2/2
#2 lore raw
In reply to Qiang Yu
Quoted 118 lines · Qiang Yu

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use GEM+shmem for MM. Currently just alloc and pin memory when gem object creation. GPU vm map of the buffer is also done in the alloc stage in kernel space. We may delay the memory allocation and real GPU vm map to command submission stage in the furture as improvement.
- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

mesa driver can be found here before upstreamed:
https://gitlab.freedesktop.org/lima/mesa

v7:
- remove lima_fence_ops with default value
- move fence slab create to device probe
- check pad ioctl args to be zero
- add comments for user/kernel interface

v6:
- fix comments by checkpatch.pl

v5:
- export gp/pp version to userspace
- rebase on drm-misc-next

v4:
- use get param interface to get info
- separate context create/free ioctl
- remove unused max sched task param
- update copyright time
- use xarray instead of idr
- stop using drmP.h

v3:
- fix comments from kbuild robot
- restrict supported arch to tested ones

v2:
- fix syscall argument check
- fix job finish fence leak since kernel 5.0
- use drm syncobj to replace native fence
- move buffer object GPU va map into kernel
- reserve syscall argument space for future info
- remove kernel gem modifier
- switch TTM back to GEM+shmem MM
- use time based io poll
- use whole register name
- adopt gem reservation obj integration
- use drm_timeout_abs_to_jiffies

---
 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  21 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  47 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       |  97 ++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  30 ++
 drivers/gpu/drm/lima/lima_device.c    | 385 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 131 ++++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  58 ++++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 ++
 drivers/gpu/drm/lima/lima_drv.c       | 376 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  45 +++
 drivers/gpu/drm/lima/lima_gem.c       | 381 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c |  47 +++
 drivers/gpu/drm/lima/lima_gem_prime.h |  13 +
 drivers/gpu/drm/lima/lima_gp.c        | 283 +++++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  80 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 142 +++++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 122 ++++++++
 drivers/gpu/drm/lima/lima_object.h    |  36 +++
 drivers/gpu/drm/lima/lima_pmu.c       |  60 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 427 ++++++++++++++++++++++++++
Quoted 8 lines
 drivers/gpu/drm/lima/lima_pp.h        |  19 ++
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 404 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 104 +++++++
 drivers/gpu/drm/lima/lima_vm.c        | 282 +++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  62 ++++
 include/uapi/drm/lima_drm.h           | 164 ++++++++++
 36 files changed, 4242 insertions(+)

I can apply this if you want, though I'm not completely sure whether folks want the mesa bits to go upstream first.

Rob
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

maintainer
#3 lore raw
Quoted 129 lines · Rob Herring
Quoted 118 lines · Qiang Yu

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use GEM+shmem for MM. Currently just alloc and pin memory when gem object creation. GPU vm map of the buffer is also done in the alloc stage in kernel space. We may delay the memory allocation and real GPU vm map to command submission stage in the furture as improvement.
- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

mesa driver can be found here before upstreamed:
https://gitlab.freedesktop.org/lima/mesa

v7:
- remove lima_fence_ops with default value
- move fence slab create to device probe
- check pad ioctl args to be zero
- add comments for user/kernel interface

v6:
- fix comments by checkpatch.pl

v5:
- export gp/pp version to userspace
- rebase on drm-misc-next

v4:
- use get param interface to get info
- separate context create/free ioctl
- remove unused max sched task param
- update copyright time
- use xarray instead of idr
- stop using drmP.h

v3:
- fix comments from kbuild robot
- restrict supported arch to tested ones

v2:
- fix syscall argument check
- fix job finish fence leak since kernel 5.0
- use drm syncobj to replace native fence
- move buffer object GPU va map into kernel
- reserve syscall argument space for future info
- remove kernel gem modifier
- switch TTM back to GEM+shmem MM
- use time based io poll
- use whole register name
- adopt gem reservation obj integration
- use drm_timeout_abs_to_jiffies

---
 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  21 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  47 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       |  97 ++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  30 ++
 drivers/gpu/drm/lima/lima_device.c    | 385 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 131 ++++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  58 ++++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 ++
 drivers/gpu/drm/lima/lima_drv.c       | 376 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  45 +++
 drivers/gpu/drm/lima/lima_gem.c       | 381 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c |  47 +++
 drivers/gpu/drm/lima/lima_gem_prime.h |  13 +
 drivers/gpu/drm/lima/lima_gp.c        | 283 +++++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  80 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 142 +++++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 122 ++++++++
 drivers/gpu/drm/lima/lima_object.h    |  36 +++
 drivers/gpu/drm/lima/lima_pmu.c       |  60 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 427 ++++++++++++++++++++++++++
Quoted 8 lines
 drivers/gpu/drm/lima/lima_pp.h        |  19 ++
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 404 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 104 +++++++
 drivers/gpu/drm/lima/lima_vm.c        | 282 +++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  62 ++++
 include/uapi/drm/lima_drm.h           | 164 ++++++++++
 36 files changed, 4242 insertions(+)
Quoted 5 lines

I can apply this if you want, though I'm not completely sure whether folks want the mesa bits to go upstream first.

We generally want the kernel bits upstream first.

We'd want the upstream mesa bits reviewed though and having a mesa MR with those bits reviewed should be a prereq of this landing.

Then we land this, and then land that.

Dave.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#4 lore raw
Quoted 137 lines · Dave Airlie
Quoted 129 lines · Rob Herring
Quoted 118 lines · Qiang Yu

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use GEM+shmem for MM. Currently just alloc and pin memory when gem object creation. GPU vm map of the buffer is also done in the alloc stage in kernel space. We may delay the memory allocation and real GPU vm map to command submission stage in the furture as improvement.
- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

mesa driver can be found here before upstreamed:
https://gitlab.freedesktop.org/lima/mesa

v7:
- remove lima_fence_ops with default value
- move fence slab create to device probe
- check pad ioctl args to be zero
- add comments for user/kernel interface

v6:
- fix comments by checkpatch.pl

v5:
- export gp/pp version to userspace
- rebase on drm-misc-next

v4:
- use get param interface to get info
- separate context create/free ioctl
- remove unused max sched task param
- update copyright time
- use xarray instead of idr
- stop using drmP.h

v3:
- fix comments from kbuild robot
- restrict supported arch to tested ones

v2:
- fix syscall argument check
- fix job finish fence leak since kernel 5.0
- use drm syncobj to replace native fence
- move buffer object GPU va map into kernel
- reserve syscall argument space for future info
- remove kernel gem modifier
- switch TTM back to GEM+shmem MM
- use time based io poll
- use whole register name
- adopt gem reservation obj integration
- use drm_timeout_abs_to_jiffies

---
 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  21 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  47 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       |  97 ++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  30 ++
 drivers/gpu/drm/lima/lima_device.c    | 385 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 131 ++++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  58 ++++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 ++
 drivers/gpu/drm/lima/lima_drv.c       | 376 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  45 +++
 drivers/gpu/drm/lima/lima_gem.c       | 381 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c |  47 +++
 drivers/gpu/drm/lima/lima_gem_prime.h |  13 +
 drivers/gpu/drm/lima/lima_gp.c        | 283 +++++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  80 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 142 +++++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 122 ++++++++
 drivers/gpu/drm/lima/lima_object.h    |  36 +++
 drivers/gpu/drm/lima/lima_pmu.c       |  60 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 427 ++++++++++++++++++++++++++
Quoted 8 lines
 drivers/gpu/drm/lima/lima_pp.h        |  19 ++
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 404 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 104 +++++++
 drivers/gpu/drm/lima/lima_vm.c        | 282 +++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  62 ++++
 include/uapi/drm/lima_drm.h           | 164 ++++++++++
 36 files changed, 4242 insertions(+)
Quoted 5 lines

I can apply this if you want, though I'm not completely sure whether folks want the mesa bits to go upstream first.

Quoted 7 lines

We generally want the kernel bits upstream first.

We'd want the upstream mesa bits reviewed though and having a mesa MR with those bits reviewed should be a prereq of this landing.

Then we land this, and then land that.

Mesa part is definitely my next upstream step.

Regards,
Qiang
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#5 lore raw
In reply to Rob Herring
Quoted 128 lines · Rob Herring
Quoted 118 lines · Qiang Yu

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use GEM+shmem for MM. Currently just alloc and pin memory when gem object creation. GPU vm map of the buffer is also done in the alloc stage in kernel space. We may delay the memory allocation and real GPU vm map to command submission stage in the furture as improvement.
- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

mesa driver can be found here before upstreamed:
https://gitlab.freedesktop.org/lima/mesa

v7:
- remove lima_fence_ops with default value
- move fence slab create to device probe
- check pad ioctl args to be zero
- add comments for user/kernel interface

v6:
- fix comments by checkpatch.pl

v5:
- export gp/pp version to userspace
- rebase on drm-misc-next

v4:
- use get param interface to get info
- separate context create/free ioctl
- remove unused max sched task param
- update copyright time
- use xarray instead of idr
- stop using drmP.h

v3:
- fix comments from kbuild robot
- restrict supported arch to tested ones

v2:
- fix syscall argument check
- fix job finish fence leak since kernel 5.0
- use drm syncobj to replace native fence
- move buffer object GPU va map into kernel
- reserve syscall argument space for future info
- remove kernel gem modifier
- switch TTM back to GEM+shmem MM
- use time based io poll
- use whole register name
- adopt gem reservation obj integration
- use drm_timeout_abs_to_jiffies

---
 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  21 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  47 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       |  97 ++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  30 ++
 drivers/gpu/drm/lima/lima_device.c    | 385 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 131 ++++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  58 ++++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 ++
 drivers/gpu/drm/lima/lima_drv.c       | 376 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  45 +++
 drivers/gpu/drm/lima/lima_gem.c       | 381 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c |  47 +++
 drivers/gpu/drm/lima/lima_gem_prime.h |  13 +
 drivers/gpu/drm/lima/lima_gp.c        | 283 +++++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  80 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 142 +++++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 122 ++++++++
 drivers/gpu/drm/lima/lima_object.h    |  36 +++
 drivers/gpu/drm/lima/lima_pmu.c       |  60 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 427 ++++++++++++++++++++++++++
Quoted 8 lines
 drivers/gpu/drm/lima/lima_pp.h        |  19 ++
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 404 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 104 +++++++
 drivers/gpu/drm/lima/lima_vm.c        | 282 +++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  62 ++++
 include/uapi/drm/lima_drm.h           | 164 ++++++++++
 36 files changed, 4242 insertions(+)
Quoted 4 lines

I can apply this if you want, though I'm not completely sure whether folks want the mesa bits to go upstream first.

They'd need to hide the Mesa usage of the ABI until the kernel merges anyway, so it's enough that it's public in a way that targets Mesa.

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#6 lore raw
Quoted 131 lines · Eric Anholt
Quoted 128 lines · Rob Herring
Quoted 126 lines · Qiang Yu

- Mali 4xx GPUs have two kinds of processors GP and PP. GP is for OpenGL vertex shader processing and PP is for fragment shader processing. Each processor has its own MMU so prcessors work in virtual address space.
- There's only one GP but multiple PP (max 4 for mali 400 and 8 for mali 450) in the same mali 4xx GPU. All PPs are grouped togather to handle a single fragment shader task divided by FB output tiled pixels. Mali 400 user space driver is responsible for assign target tiled pixels to each PP, but mali 450 has a HW module called DLBU to dynamically balance each PP's load.
- User space driver allocate buffer object and map into GPU virtual address space, upload command stream and draw data with CPU mmap of the buffer object, then submit task to GP/PP with a register frame indicating where is the command stream and misc settings.
- There's no command stream validation/relocation due to each user process has its own GPU virtual address space. GP/PP's MMU switch virtual address space before running two tasks from different user process. Error or evil user space code just get MMU fault or GP/PP error IRQ, then the HW/SW will be recovered.
- Use GEM+shmem for MM. Currently just alloc and pin memory when gem object creation. GPU vm map of the buffer is also done in the alloc stage in kernel space. We may delay the memory allocation and real GPU vm map to command submission stage in the furture as improvement.
- Use drm_sched for GPU task schedule. Each OpenGL context should have a lima context object in the kernel to distinguish tasks from different user. drm_sched gets task from each lima context in a fair way.

mesa driver can be found here before upstreamed:
https://gitlab.freedesktop.org/lima/mesa

v7:
- remove lima_fence_ops with default value
- move fence slab create to device probe
- check pad ioctl args to be zero
- add comments for user/kernel interface

v6:
- fix comments by checkpatch.pl

v5:
- export gp/pp version to userspace
- rebase on drm-misc-next

v4:
- use get param interface to get info
- separate context create/free ioctl
- remove unused max sched task param
- update copyright time
- use xarray instead of idr
- stop using drmP.h

v3:
- fix comments from kbuild robot
- restrict supported arch to tested ones

v2:
- fix syscall argument check
- fix job finish fence leak since kernel 5.0
- use drm syncobj to replace native fence
- move buffer object GPU va map into kernel
- reserve syscall argument space for future info
- remove kernel gem modifier
- switch TTM back to GEM+shmem MM
- use time based io poll
- use whole register name
- adopt gem reservation obj integration
- use drm_timeout_abs_to_jiffies

---
 drivers/gpu/drm/Kconfig               |   2 +
 drivers/gpu/drm/Makefile              |   1 +
 drivers/gpu/drm/lima/Kconfig          |  10 +
 drivers/gpu/drm/lima/Makefile         |  21 ++
 drivers/gpu/drm/lima/lima_bcast.c     |  47 +++
 drivers/gpu/drm/lima/lima_bcast.h     |  14 +
 drivers/gpu/drm/lima/lima_ctx.c       |  97 ++++++
 drivers/gpu/drm/lima/lima_ctx.h       |  30 ++
 drivers/gpu/drm/lima/lima_device.c    | 385 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_device.h    | 131 ++++++++
 drivers/gpu/drm/lima/lima_dlbu.c      |  58 ++++
 drivers/gpu/drm/lima/lima_dlbu.h      |  18 ++
 drivers/gpu/drm/lima/lima_drv.c       | 376 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_drv.h       |  45 +++
 drivers/gpu/drm/lima/lima_gem.c       | 381 +++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.h       |  25 ++
 drivers/gpu/drm/lima/lima_gem_prime.c |  47 +++
 drivers/gpu/drm/lima/lima_gem_prime.h |  13 +
 drivers/gpu/drm/lima/lima_gp.c        | 283 +++++++++++++++++
 drivers/gpu/drm/lima/lima_gp.h        |  16 +
 drivers/gpu/drm/lima/lima_l2_cache.c  |  80 +++++
 drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
 drivers/gpu/drm/lima/lima_mmu.c       | 142 +++++++++
 drivers/gpu/drm/lima/lima_mmu.h       |  16 +
 drivers/gpu/drm/lima/lima_object.c    | 122 ++++++++
 drivers/gpu/drm/lima/lima_object.h    |  36 +++
 drivers/gpu/drm/lima/lima_pmu.c       |  60 ++++
 drivers/gpu/drm/lima/lima_pmu.h       |  12 +
 drivers/gpu/drm/lima/lima_pp.c        | 427 ++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_pp.h        |  19 ++
 drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c     | 404 ++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h     | 104 +++++++
 drivers/gpu/drm/lima/lima_vm.c        | 282 +++++++++++++++++
 drivers/gpu/drm/lima/lima_vm.h        |  62 ++++
 include/uapi/drm/lima_drm.h           | 164 ++++++++++
 36 files changed, 4242 insertions(+)
Quoted 7 lines
Quoted 4 lines

I can apply this if you want, though I'm not completely sure whether folks want the mesa bits to go upstream first.

They'd need to hide the Mesa usage of the ABI until the kernel merges anyway, so it's enough that it's public in a way that targets Mesa.

I'd like to get this merged first if accepted instead of maintain a separate branch when mesa upstream process. mesa part could have a kernel driver when the upstream gets reviewed too.

Regards,
Qiang
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

maintainer
#7 lore raw
In reply to Qiang Yu
Quoted 111 lines
(quoted diff) +1 -0
+#endif
include/uapi/drm/lima_drm.h new file +104 -0
new file mode 100644
index 000000000000..05f8c910d7fb
@@ -0,0 +1,164 @@
+/* SPDX-License-Identifier: (GPL-2.0 WITH Linux-syscall-note) OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_DRM_H__
+#define __LIMA_DRM_H__
+
+#include "drm.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+enum drm_lima_param_gpu_id {
+       DRM_LIMA_PARAM_GPU_ID_UNKNOWN,
+       DRM_LIMA_PARAM_GPU_ID_MALI400,
+       DRM_LIMA_PARAM_GPU_ID_MALI450,
+};
+
+enum drm_lima_param {
+       DRM_LIMA_PARAM_GPU_ID,
+       DRM_LIMA_PARAM_NUM_PP,
+       DRM_LIMA_PARAM_GP_VERSION,
+       DRM_LIMA_PARAM_PP_VERSION,
+};
+
+/**
+ * get various information of the GPU
+ */
+struct drm_lima_get_param {
+       __u32 param; /* in, value in enum drm_lima_param */
+       __u32 pad;   /* pad, must be zero */
+       __u64 value; /* out, parameter value */
+};
+
+/**
+ * create a buffer for used by GPU
+ */
+struct drm_lima_gem_create {
+       __u32 size;    /* in, buffer size */
+       __u32 flags;   /* in, currently no flags, must be zero */
+       __u32 handle;  /* out, GEM buffer handle */
+       __u32 pad;     /* pad, must be zero */
+};
+
+/**
+ * get information of a buffer
+ */
+struct drm_lima_gem_info {
+       __u32 handle;  /* in, GEM buffer handle */
+       __u32 va;      /* out, virtual address mapped into GPU MMU */
+       __u64 offset;  /* out, used to mmap this buffer to CPU */
+};
+
+#define LIMA_SUBMIT_BO_READ   0x01
+#define LIMA_SUBMIT_BO_WRITE  0x02
+
+/* buffer information used by one task */
+struct drm_lima_gem_submit_bo {
+       __u32 handle;  /* in, GEM buffer handle */
+       __u32 flags;   /* in, buffer read/write by GPU */
+};
+
+#define LIMA_GP_FRAME_REG_NUM 6
+
+/* frame used to setup GP for each task */
+struct drm_lima_gp_frame {
+       __u32 frame[LIMA_GP_FRAME_REG_NUM];
+};
+
+#define LIMA_PP_FRAME_REG_NUM 23
+#define LIMA_PP_WB_REG_NUM 12
+
+/* frame used to setup mali400 GPU PP for each task */
+struct drm_lima_m400_pp_frame {
+       __u32 frame[LIMA_PP_FRAME_REG_NUM];
+       __u32 num_pp;
+       __u32 wb[3 * LIMA_PP_WB_REG_NUM];
+       __u32 plbu_array_address[4];
+       __u32 fragment_stack_address[4];
+};
+
+/* frame used to setup mali450 GPU PP for each task */
+struct drm_lima_m450_pp_frame {
+       __u32 frame[LIMA_PP_FRAME_REG_NUM];
+       __u32 num_pp;
+       __u32 wb[3 * LIMA_PP_WB_REG_NUM];
+       __u32 use_dlbu;
+       __u32 _pad;
+       union {
+               __u32 plbu_array_address[8];
+               __u32 dlbu_regs[4];
+       };
+       __u32 fragment_stack_address[8];
+};
+
+#define LIMA_PIPE_GP  0x00
+#define LIMA_PIPE_PP  0x01
+
+#define LIMA_SUBMIT_FLAG_EXPLICIT_FENCE (1 << 0)
+
+/**
+ * submit a task to GPU
+ */
+struct drm_lima_gem_submit {

include/uapi/drm/lima_drm.h line 114

Quoted 10 lines
include/uapi/drm/lima_drm.h new file +10 -0
+       __u32 ctx;         /* in, context handle task is submitted to */
+       __u32 pipe;        /* in, which pipe to use, GP/PP */
+       __u32 nr_bos;      /* in, array length of bos field */
+       __u32 frame_size;  /* in, size of frame field */
+       __u64 bos;         /* in, array of drm_lima_gem_submit_bo */
+       __u64 frame;       /* in, GP/PP frame */
+       __u32 flags;       /* in, submit flags */
+       __u32 out_sync;    /* in, drm_syncobj handle used to wait task finish after submission */
+       __u32 in_sync[2];  /* in, drm_syncobj handle used to wait before start this task */
+};

This seems a bit limited, is there a reason it's two, at least in Vulkan drivers we'd want more than two I suspect (Vulkan may not work on this hw anyways), but it might be required in the future to make this extensible.

At least a comment stating why 2 was picked is sufficient for current use cases.

Thanks,
Dave
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#8 lore raw
Quoted 119 lines · Dave Airlie
Quoted 119 lines
(quoted diff) +1 -0
+#endif
include/uapi/drm/lima_drm.h new file +111 -0
new file mode 100644
index 000000000000..05f8c910d7fb
@@ -0,0 +1,164 @@
+/* SPDX-License-Identifier: (GPL-2.0 WITH Linux-syscall-note) OR MIT */
+/* Copyright 2017-2018 Qiang Yu <[email protected]> */
+
+#ifndef __LIMA_DRM_H__
+#define __LIMA_DRM_H__
+
+#include "drm.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+enum drm_lima_param_gpu_id {
+       DRM_LIMA_PARAM_GPU_ID_UNKNOWN,
+       DRM_LIMA_PARAM_GPU_ID_MALI400,
+       DRM_LIMA_PARAM_GPU_ID_MALI450,
+};
+
+enum drm_lima_param {
+       DRM_LIMA_PARAM_GPU_ID,
+       DRM_LIMA_PARAM_NUM_PP,
+       DRM_LIMA_PARAM_GP_VERSION,
+       DRM_LIMA_PARAM_PP_VERSION,
+};
+
+/**
+ * get various information of the GPU
+ */
+struct drm_lima_get_param {
+       __u32 param; /* in, value in enum drm_lima_param */
+       __u32 pad;   /* pad, must be zero */
+       __u64 value; /* out, parameter value */
+};
+
+/**
+ * create a buffer for used by GPU
+ */
+struct drm_lima_gem_create {
+       __u32 size;    /* in, buffer size */
+       __u32 flags;   /* in, currently no flags, must be zero */
+       __u32 handle;  /* out, GEM buffer handle */
+       __u32 pad;     /* pad, must be zero */
+};
+
+/**
+ * get information of a buffer
+ */
+struct drm_lima_gem_info {
+       __u32 handle;  /* in, GEM buffer handle */
+       __u32 va;      /* out, virtual address mapped into GPU MMU */
+       __u64 offset;  /* out, used to mmap this buffer to CPU */
+};
+
+#define LIMA_SUBMIT_BO_READ   0x01
+#define LIMA_SUBMIT_BO_WRITE  0x02
+
+/* buffer information used by one task */
+struct drm_lima_gem_submit_bo {
+       __u32 handle;  /* in, GEM buffer handle */
+       __u32 flags;   /* in, buffer read/write by GPU */
+};
+
+#define LIMA_GP_FRAME_REG_NUM 6
+
+/* frame used to setup GP for each task */
+struct drm_lima_gp_frame {
+       __u32 frame[LIMA_GP_FRAME_REG_NUM];
+};
+
+#define LIMA_PP_FRAME_REG_NUM 23
+#define LIMA_PP_WB_REG_NUM 12
+
+/* frame used to setup mali400 GPU PP for each task */
+struct drm_lima_m400_pp_frame {
+       __u32 frame[LIMA_PP_FRAME_REG_NUM];
+       __u32 num_pp;
+       __u32 wb[3 * LIMA_PP_WB_REG_NUM];
+       __u32 plbu_array_address[4];
+       __u32 fragment_stack_address[4];
+};
+
+/* frame used to setup mali450 GPU PP for each task */
+struct drm_lima_m450_pp_frame {
+       __u32 frame[LIMA_PP_FRAME_REG_NUM];
+       __u32 num_pp;
+       __u32 wb[3 * LIMA_PP_WB_REG_NUM];
+       __u32 use_dlbu;
+       __u32 _pad;
+       union {
+               __u32 plbu_array_address[8];
+               __u32 dlbu_regs[4];
+       };
+       __u32 fragment_stack_address[8];
+};
+
+#define LIMA_PIPE_GP  0x00
+#define LIMA_PIPE_PP  0x01
+
+#define LIMA_SUBMIT_FLAG_EXPLICIT_FENCE (1 << 0)
+
+/**
+ * submit a task to GPU
+ */
+struct drm_lima_gem_submit {
+       __u32 ctx;         /* in, context handle task is submitted to */
+       __u32 pipe;        /* in, which pipe to use, GP/PP */
+       __u32 nr_bos;      /* in, array length of bos field */
+       __u32 frame_size;  /* in, size of frame field */
+       __u64 bos;         /* in, array of drm_lima_gem_submit_bo */
+       __u64 frame;       /* in, GP/PP frame */
+       __u32 flags;       /* in, submit flags */
Quoted 8 lines

include/uapi/drm/lima_drm.h line 114

Quoted 3 lines
include/uapi/drm/lima_drm.h new file +3 -0
+       __u32 out_sync;    /* in, drm_syncobj handle used to wait task finish after submission */
+       __u32 in_sync[2];  /* in, drm_syncobj handle used to wait before start this task */
+};

This seems a bit limited, is there a reason it's two, at least in Vulkan drivers we'd want more than two I suspect (Vulkan may not work on this hw anyways), but it might be required in the future to make this extensible.

Mali4xx GPU does not support Vulkan, the reason I pick two is, one for sync_file fd imported drm_syncobj, one for GP out_sync be able to pass to PP in_sync directly when explicit fence without drm_syncobj -> sync_file -> merge sync_file -> drm_syncobj pass.

Quoted 3 lines

At least a comment stating why 2 was picked is sufficient for current use cases.

OK, will add it.

Regards,
Qiang
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel