Skip to contentSkip to conversation

AMDGPU Doorbell manager

PATCH In review Shashank Sharma 3 participants 24 messages Jun 20, 2023 – Jul 13, 2023

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v3 cover AMDGPU Doorbell manager

Doorbells in AMDGPU drivers are currently managed by different clients in a scattered way, across may places. The existing clients are:
- AMDGPU graphics driver for kernel level doorbell writes.
- AMDGPU MES module for kernel level doorbell write (MES ring test and aggregated doorbells).
- AMDGPU MES module for MES process doorbells.
- AMDKFD module for KFD/KIQ kernel doorbells.
- AMDKFD module for KFD process doorbells.
- AMDGPU usermode queues for usermode doorbell writes (upcoming).

This patch series introduces doorbell-manager to keep the doorbell handling at a central place. The fundamental changes are:
- Introduce and accommodate a new GEM domain for doorbells.
- Prepare the AMDGPU ttm backend for handling doorbell allocation.
- Create doorbell BOs for kernel-level and process level doorbell opertations, and place it in existing structures.
- Modify the existing graphics, KFD and MES code to use the doorbell-manager functions.
- Remove the existing doorbell management code in KFD/MES.

The idea is that:
- a kernel client can call doorbell manager functions to allocate/free doorbell pages.
- a usermode app can directly allocate a page from the doorbell bar just like a GEM object and use it for different usermode queues.
- There is no direct UAPI change in this series, just an additional flag for GEM_OBJECT_DOORBELL, here:
https://gitlab.freedesktop.org/mesa/drm/-/merge_requests/287

PS: This series has been sanity tested with kfd_test_suit to ensure
it is not introducing any regressions due to kfd doorbell changes.

Alex Deucher (2):
drm/amdgpu: add UAPI for allocating doorbell memory drm/amdgpu: accommodate DOMAIN/PL_DOORBELL

Shashank Sharma (10):
drm/amdgpu: create a new file for doorbell manager drm/amdgpu: don't modify num_doorbells for mes drm/amdgpu: initialize ttm for doorbells drm/amdgpu: create kernel doorbell pages
get absolute offset from doorbell index drm/amdgpu: use doorbell manager for kfd kernel doorbells drm/amdgpu: use doorbell manager for kfd process doorbells drm/amdgpu: remove unused functions and variables drm/amdgpu: use doorbell mgr for MES kernel doorbells drm/amdgpu: cleanup MES process level doorbells

 drivers/gpu/drm/amd/amdgpu/Makefile           |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    | 174 +------------
 drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |  17 +-
 .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 230 +++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c       | 148 +++--------
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |  16 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c    |  11 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.h    |   2 +
 .../gpu/drm/amd/amdgpu/amdgpu_res_cursor.h    |   2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  30 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h       |   1 +
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c      |  20 +-
 drivers/gpu/drm/amd/amdkfd/kfd_device.c       |   2 -
 .../drm/amd/amdkfd/kfd_device_queue_manager.c |   8 +-
 drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c     | 232 ++++++++----------
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h         |  27 +-
 drivers/gpu/drm/amd/amdkfd/kfd_process.c      |  40 +--
 .../amd/amdkfd/kfd_process_queue_manager.c    |  23 +-
 include/uapi/drm/amdgpu_drm.h                 |   7 +-
 19 files changed, 487 insertions(+), 505 deletions(-)
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c
Signature · 1 line
2.40.1
#1 lore raw

PATCH v3 1/12 drm/amdgpu: create a new file for doorbell manager

This patch:
- creates a new file for doorbell management.
- moves doorbell code from amdgpu_device.c to this file.

V2:
- remove doc from function declaration (Christian)
- remove 'device' from function names to make it consistent (Alex)
- add SPDX license identifier (Luben)

V3:
- change license to MIT license(Christian)

---
 drivers/gpu/drm/amd/amdgpu/Makefile           |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    | 174 +----------------
 drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |   6 +
 .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 183 ++++++++++++++++++
 4 files changed, 195 insertions(+), 170 deletions(-)
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c
drivers/gpu/drm/amd/amdgpu/Makefile +1 -1
index 415a7fa395c4..b0b2cd21160a 100644
@@ -42,7 +42,7 @@ ccflags-y := -I$(FULL_AMD_PATH)/include/asic_reg \
 amdgpu-y := amdgpu_drv.o
 
 # add KMS driver
-amdgpu-y += amdgpu_device.o amdgpu_kms.o \
+amdgpu-y += amdgpu_device.o amdgpu_doorbell_mgr.o amdgpu_kms.o \
 	amdgpu_atombios.o atombios_crtc.o amdgpu_connectors.o \
 	atom.o amdgpu_fence.o amdgpu_ttm.o amdgpu_object.o amdgpu_gart.o \
 	amdgpu_encoders.o amdgpu_display.o amdgpu_i2c.o \
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +5 -169

216 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h +6 -0
index 8fd11497faba..783e2b8b086c 100644
@@ -306,6 +306,12 @@ void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v);
 u64 amdgpu_mm_rdoorbell64(struct amdgpu_device *adev, u32 index);
 void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v);
 
+/*
+ * GPU doorbell aperture helpers function.
+ */
+int amdgpu_doorbell_init(struct amdgpu_device *adev);
+void amdgpu_doorbell_fini(struct amdgpu_device *adev);
+
 #define RDOORBELL32(index) amdgpu_mm_rdoorbell(adev, (index))
 #define WDOORBELL32(index, v) amdgpu_mm_wdoorbell(adev, (index), (v))
 #define RDOORBELL64(index) amdgpu_mm_rdoorbell64(adev, (index))
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c new file +183 -0

183 lines not on this page · raw message

Signature · 1 line
2.40.1
#2 lore raw

PATCH v3 2/12 drm/amdgpu: don't modify num_doorbells for mes

In reply to Shashank Sharma

This patch removes the check and change in num_kernel_doorbells for MES, which is not being used anywhere by MES code.

---
 .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 34 ++++++++-----------
 1 file changed, 15 insertions(+), 19 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +15 -19
index 31db526d4921..eb113e38f5e9 100644
@@ -140,25 +140,21 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev)
 	adev->doorbell.base = pci_resource_start(adev->pdev, 2);
 	adev->doorbell.size = pci_resource_len(adev->pdev, 2);
 
-	if (adev->enable_mes) {
-		adev->doorbell.num_kernel_doorbells =
-			adev->doorbell.size / sizeof(u32);
-	} else {
-		adev->doorbell.num_kernel_doorbells =
-			min_t(u32, adev->doorbell.size / sizeof(u32),
-			      adev->doorbell_index.max_assignment+1);
-		if (adev->doorbell.num_kernel_doorbells == 0)
-			return -EINVAL;
-
-		/* For Vega, reserve and map two pages on doorbell BAR since SDMA
-		 * paging queue doorbell use the second page. The
-		 * AMDGPU_DOORBELL64_MAX_ASSIGNMENT definition assumes all the
-		 * doorbells are in the first page. So with paging queue enabled,
-		 * the max num_kernel_doorbells should + 1 page (0x400 in dword)
-		 */
-		if (adev->asic_type >= CHIP_VEGA10)
-			adev->doorbell.num_kernel_doorbells += 0x400;
-	}
+	adev->doorbell.num_kernel_doorbells =
+		min_t(u32, adev->doorbell.size / sizeof(u32),
+				adev->doorbell_index.max_assignment+1);
+	if (adev->doorbell.num_kernel_doorbells == 0)
+		return -EINVAL;
+
+	/*
+	 * For Vega, reserve and map two pages on doorbell BAR since SDMA
+	 * paging queue doorbell use the second page. The
+	 * AMDGPU_DOORBELL64_MAX_ASSIGNMENT definition assumes all the
+	 * doorbells are in the first page. So with paging queue enabled,
+	 * the max num_kernel_doorbells should + 1 page (0x400 in dword)
+	 */
+	if (adev->asic_type >= CHIP_VEGA10)
+		adev->doorbell.num_kernel_doorbells += 0x400;
 
 	adev->doorbell.ptr = ioremap(adev->doorbell.base,
 				     adev->doorbell.num_kernel_doorbells *
Signature · 1 line
2.40.1
#3 lore raw

PATCH v3 3/12 drm/amdgpu: add UAPI for allocating doorbell memory

In reply to Shashank Sharma

From: Alex Deucher <[email protected]>

This patch adds flags for a new gem domain AMDGPU_GEM_DOMAIN_DOORBELL in the UAPI layer.

V2: Drop 'memory' from description (Christian)

---
 include/uapi/drm/amdgpu_drm.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
include/uapi/drm/amdgpu_drm.h +6 -1
index b6eb90df5d05..36f41bfa987f 100644
@@ -94,6 +94,9 @@ extern "C" {
  *
  * %AMDGPU_GEM_DOMAIN_OA	Ordered append, used by 3D or Compute engines
  * for appending data.
+ *
+ * %AMDGPU_GEM_DOMAIN_DOORBELL	Doorbell. It is an MMIO region for
+ * signalling user mode queues.
  */
 #define AMDGPU_GEM_DOMAIN_CPU		0x1
 #define AMDGPU_GEM_DOMAIN_GTT		0x2
@@ -101,12 +104,14 @@ extern "C" {
 #define AMDGPU_GEM_DOMAIN_GDS		0x8
 #define AMDGPU_GEM_DOMAIN_GWS		0x10
 #define AMDGPU_GEM_DOMAIN_OA		0x20
+#define AMDGPU_GEM_DOMAIN_DOORBELL	0x40
 #define AMDGPU_GEM_DOMAIN_MASK		(AMDGPU_GEM_DOMAIN_CPU | \
 					 AMDGPU_GEM_DOMAIN_GTT | \
 					 AMDGPU_GEM_DOMAIN_VRAM | \
 					 AMDGPU_GEM_DOMAIN_GDS | \
 					 AMDGPU_GEM_DOMAIN_GWS | \
-					 AMDGPU_GEM_DOMAIN_OA)
+					 AMDGPU_GEM_DOMAIN_OA | \
+					 AMDGPU_GEM_DOMAIN_DOORBELL)
 
 /* Flag that CPU access will be required for the case of VRAM domain */
 #define AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED	(1 << 0)
Signature · 1 line
2.40.1
#4 lore raw

PATCH v3 4/12 drm/amdgpu: accommodate DOMAIN/PL_DOORBELL

In reply to Shashank Sharma

From: Alex Deucher <[email protected]>

This patch adds changes:
- to accommodate the new GEM domain DOORBELL
- to accommodate the new TTM PL DOORBELL

in order to manage doorbell pages as GEM object.

V2: Addressed reviwe comments from Christian
- drop the doorbell changes for pinning/unpinning
- drop the doorbell changes for dma-buf map
- drop the doorbell changes for sgt
- no need to handle TTM_PL_FLAG_CONTIGUOUS for doorbell
- add caching type for doorbell

V3: - Removed unrelated empty line (Christian)
- Add PL_DOORBELL in mem_type_to_domain() as well (Alex)

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c     | 11 ++++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.h     |  2 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h |  2 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c        | 16 +++++++++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h        |  1 +
 5 files changed, 30 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10 -1
index 3b225be89cb7..6c743b03d182 100644
@@ -148,6 +148,14 @@ void amdgpu_bo_placement_from_domain(struct amdgpu_bo *abo, u32 domain)
 		c++;
 	}
 
+	if (domain & AMDGPU_GEM_DOMAIN_DOORBELL) {
+		places[c].fpfn = 0;
+		places[c].lpfn = 0;
+		places[c].mem_type = AMDGPU_PL_DOORBELL;
+		places[c].flags = 0;
+		c++;
+	}
+
 	if (domain & AMDGPU_GEM_DOMAIN_GTT) {
 		places[c].fpfn = 0;
 		places[c].lpfn = 0;
@@ -467,7 +475,7 @@ static bool amdgpu_bo_validate_size(struct amdgpu_device *adev,
 		goto fail;
 	}
 
-	/* TODO add more domains checks, such as AMDGPU_GEM_DOMAIN_CPU */
+	/* TODO add more domains checks, such as AMDGPU_GEM_DOMAIN_CPU, _DOMAIN_DOORBELL */
 	return true;
 
 fail:
@@ -1012,6 +1020,7 @@ void amdgpu_bo_unpin(struct amdgpu_bo *bo)
 	} else if (bo->tbo.resource->mem_type == TTM_PL_TT) {
 		atomic64_sub(amdgpu_bo_size(bo), &adev->gart_pin_size);
 	}
+
 }
 
 static const char *amdgpu_vram_names[] = {
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +2 -0
index 35b8106816a1..710bbcbaf98b 100644
@@ -173,6 +173,8 @@ static inline unsigned amdgpu_mem_type_to_domain(u32 mem_type)
 		return AMDGPU_GEM_DOMAIN_GWS;
 	case AMDGPU_PL_OA:
 		return AMDGPU_GEM_DOMAIN_OA;
+	case AMDGPU_PL_DOORBELL:
+		return AMDGPU_GEM_DOMAIN_DOORBELL;
 	default:
 		break;
 	}
drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h +2 -0
index 5c4f93ee0c57..3c988cc406e4 100644
@@ -90,6 +90,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
 		cur->node = block;
 		break;
 	case TTM_PL_TT:
+	case AMDGPU_PL_DOORBELL:
 		node = to_ttm_range_mgr_node(res)->mm_nodes;
 		while (start >= node->size << PAGE_SHIFT)
 			start -= node++->size << PAGE_SHIFT;
@@ -152,6 +153,7 @@ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size)
 		cur->size = min(amdgpu_vram_mgr_block_size(block), cur->remaining);
 		break;
 	case TTM_PL_TT:
+	case AMDGPU_PL_DOORBELL:
 		node = cur->node;
 
 		cur->node = ++node;
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +15 -1
index 2cd081cbf706..29066e9a3439 100644
@@ -128,6 +128,7 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo,
 	case AMDGPU_PL_GDS:
 	case AMDGPU_PL_GWS:
 	case AMDGPU_PL_OA:
+	case AMDGPU_PL_DOORBELL:
 		placement->num_placement = 0;
 		placement->num_busy_placement = 0;
 		return;
@@ -496,9 +497,11 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo, bool evict,
 	if (old_mem->mem_type == AMDGPU_PL_GDS ||
 	    old_mem->mem_type == AMDGPU_PL_GWS ||
 	    old_mem->mem_type == AMDGPU_PL_OA ||
+	    old_mem->mem_type == AMDGPU_PL_DOORBELL ||
 	    new_mem->mem_type == AMDGPU_PL_GDS ||
 	    new_mem->mem_type == AMDGPU_PL_GWS ||
-	    new_mem->mem_type == AMDGPU_PL_OA) {
+	    new_mem->mem_type == AMDGPU_PL_OA ||
+	    new_mem->mem_type == AMDGPU_PL_DOORBELL) {
 		/* Nothing to save here */
 		ttm_bo_move_null(bo, new_mem);
 		goto out;
@@ -582,6 +585,12 @@ static int amdgpu_ttm_io_mem_reserve(struct ttm_device *bdev,
 		mem->bus.offset += adev->gmc.aper_base;
 		mem->bus.is_iomem = true;
 		break;
+	case AMDGPU_PL_DOORBELL:
+		mem->bus.offset = mem->start << PAGE_SHIFT;
+		mem->bus.offset += adev->doorbell.base;
+		mem->bus.is_iomem = true;
+		mem->bus.caching = ttm_uncached;
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -596,6 +605,10 @@ static unsigned long amdgpu_ttm_io_mem_pfn(struct ttm_buffer_object *bo,
 
 	amdgpu_res_first(bo->resource, (u64)page_offset << PAGE_SHIFT, 0,
 			 &cursor);
+
+	if (bo->resource->mem_type == AMDGPU_PL_DOORBELL)
+		return ((uint64_t)(adev->doorbell.base + cursor.start)) >> PAGE_SHIFT;
+
 	return (adev->gmc.aper_base + cursor.start) >> PAGE_SHIFT;
 }
 
@@ -1263,6 +1276,7 @@ uint64_t amdgpu_ttm_tt_pde_flags(struct ttm_tt *ttm, struct ttm_resource *mem)
 		flags |= AMDGPU_PTE_VALID;
 
 	if (mem && (mem->mem_type == TTM_PL_TT ||
+		    mem->mem_type == AMDGPU_PL_DOORBELL ||
 		    mem->mem_type == AMDGPU_PL_PREEMPT)) {
 		flags |= AMDGPU_PTE_SYSTEM;
 
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h +1 -0
index e2cd5894afc9..761cd6b2b942 100644
@@ -33,6 +33,7 @@
 #define AMDGPU_PL_GWS		(TTM_PL_PRIV + 1)
 #define AMDGPU_PL_OA		(TTM_PL_PRIV + 2)
 #define AMDGPU_PL_PREEMPT	(TTM_PL_PRIV + 3)
+#define AMDGPU_PL_DOORBELL	(TTM_PL_PRIV + 4)
 
 #define AMDGPU_GTT_MAX_TRANSFER_SIZE	512
 #define AMDGPU_GTT_NUM_TRANSFER_WINDOWS	2
Signature · 1 line
2.40.1
#5 lore raw

PATCH v3 5/12 drm/amdgpu: initialize ttm for doorbells

In reply to Shashank Sharma

This patch initialzes the ttm resource manager for doorbells.

V2: Do not round up doorbell size (Alex)

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +7 -0
index 29066e9a3439..6d1587bbda52 100644
@@ -1847,6 +1847,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
 	DRM_INFO("amdgpu: %uM of GTT memory ready.\n",
 		 (unsigned)(gtt_size / (1024 * 1024)));
 
+	/* Initiailize doorbell pool on PCI BAR */
+	r = amdgpu_ttm_init_on_chip(adev, AMDGPU_PL_DOORBELL, adev->doorbell.size / PAGE_SIZE);
+	if (r) {
+		DRM_ERROR("Failed initializing doorbell heap.\n");
+		return r;
+	}
+
 	/* Initialize preemptible memory pool */
 	r = amdgpu_preempt_mgr_init(adev);
 	if (r) {
Signature · 1 line
2.40.1
#6 lore raw

PATCH v3 6/12 drm/amdgpu: create kernel doorbell pages

In reply to Shashank Sharma

This patch:
- creates a doorbell page for graphics driver usages.
- adds a few new varlables in adev->doorbell structure to keep track of kernel's doorbell-bo.
- removes the adev->doorbell.ptr variable, replaces it with kernel-doorbell-bo's cpu address.

V2: - Create doorbell BO directly, no wrappe functions (Alex)
- no additional doorbell structure (Alex, Christian)
- Use doorbell_cpu_ptr, remove ioremap (Christian, Alex)
- Allocate one extra page of doorbells for MES (Alex)

Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |  8 ++-
 .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 56 ++++++++++++++-----
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |  3 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  7 +++
 4 files changed, 60 insertions(+), 14 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h +7 -1
index 783e2b8b086c..002899edb9d7 100644
@@ -31,10 +31,15 @@ struct amdgpu_doorbell {
 	/* doorbell mmio */
 	resource_size_t		base;
 	resource_size_t		size;
-	u32 __iomem		*ptr;
 
 	/* Number of doorbells reserved for amdgpu kernel driver */
 	u32 num_kernel_doorbells;
+
+	/* Kernel doorbells */
+	struct amdgpu_bo *kernel_doorbells;
+
+	/* For CPU access of doorbells */
+	uint32_t *cpu_addr;
 };
 
 /* Reserved doorbells for amdgpu (including multimedia).
@@ -311,6 +316,7 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v);
  */
 int amdgpu_doorbell_init(struct amdgpu_device *adev);
 void amdgpu_doorbell_fini(struct amdgpu_device *adev);
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev);
 
 #define RDOORBELL32(index) amdgpu_mm_rdoorbell(adev, (index))
 #define WDOORBELL32(index, v) amdgpu_mm_wdoorbell(adev, (index), (v))
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +43 -13

100 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +3 -0
index 547ec35691fa..a403418d5eac 100644
@@ -128,6 +128,9 @@ struct amdgpu_mes {
 	int                             (*kiq_hw_init)(struct amdgpu_device *adev);
 	int                             (*kiq_hw_fini)(struct amdgpu_device *adev);
 
+	/* MES doorbells */
+	uint32_t			db_start_dw_offset;
+
 	/* ip specific functions */
 	const struct amdgpu_mes_funcs   *funcs;
 };
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +7 -0
index 6d1587bbda52..c48e97646541 100644
@@ -1854,6 +1854,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
 		return r;
 	}
 
+	/* Create a boorbell page for kernel usages */
+	r = amdgpu_doorbell_create_kernel_doorbells(adev);
+	if (r) {
+		DRM_ERROR("Failed to initialize kernel doorbells.\n");
+		return r;
+	}
+
 	/* Initialize preemptible memory pool */
 	r = amdgpu_preempt_mgr_init(adev);
 	if (r) {
Signature · 1 line
2.40.1
maintainer
#7 lore raw

Re: [PATCH v3 06/12] drm/amdgpu: create kernel doorbell pages

Am 20.06.23 um 19:16 schrieb Shashank Sharma:

Quoted 101 lines

This patch:
- creates a doorbell page for graphics driver usages.
- adds a few new varlables in adev->doorbell structure to keep track of kernel's doorbell-bo.
- removes the adev->doorbell.ptr variable, replaces it with kernel-doorbell-bo's cpu address.

V2: - Create doorbell BO directly, no wrappe functions (Alex)
- no additional doorbell structure (Alex, Christian)
- Use doorbell_cpu_ptr, remove ioremap (Christian, Alex)
- Allocate one extra page of doorbells for MES (Alex)

Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |  8 ++-
  .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 56 ++++++++++++++-----
  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |  3 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  7 +++
  4 files changed, 60 insertions(+), 14 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h +7 -1
index 783e2b8b086c..002899edb9d7 100644
@@ -31,10 +31,15 @@ struct amdgpu_doorbell {
  	/* doorbell mmio */
  	resource_size_t		base;
  	resource_size_t		size;
-	u32 __iomem		*ptr;
 
  	/* Number of doorbells reserved for amdgpu kernel driver */
  	u32 num_kernel_doorbells;
+
+	/* Kernel doorbells */
+	struct amdgpu_bo *kernel_doorbells;
+
+	/* For CPU access of doorbells */
+	uint32_t *cpu_addr;
  };
 
  /* Reserved doorbells for amdgpu (including multimedia).
@@ -311,6 +316,7 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v);
   */
  int amdgpu_doorbell_init(struct amdgpu_device *adev);
  void amdgpu_doorbell_fini(struct amdgpu_device *adev);
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev);
 
  #define RDOORBELL32(index) amdgpu_mm_rdoorbell(adev, (index))
  #define WDOORBELL32(index, v) amdgpu_mm_wdoorbell(adev, (index), (v))
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +13 -4
index eb113e38f5e9..118f4bed32fd 100644
@@ -39,7 +39,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index)
  		return 0;
 
  	if (index < adev->doorbell.num_kernel_doorbells)
-		return readl(adev->doorbell.ptr + index);
+		return readl(adev->doorbell.cpu_addr + index);
 
  	DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);
  	return 0;
@@ -61,7 +61,7 @@ void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v)
  		return;
 
  	if (index < adev->doorbell.num_kernel_doorbells)
-		writel(v, adev->doorbell.ptr + index);
+		writel(v, adev->doorbell.cpu_addr + index);
  	else
  		DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index);
  }
@@ -81,7 +81,7 @@ u64 amdgpu_mm_rdoorbell64(struct amdgpu_device *adev, u32 index)
  		return 0;
 
  	if (index < adev->doorbell.num_kernel_doorbells)
-		return atomic64_read((atomic64_t *)(adev->doorbell.ptr + index));
+		return atomic64_read((atomic64_t *)(adev->doorbell.cpu_addr + index));
 
  	DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);
  	return 0;
@@ -103,11 +103,47 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v)
  		return;
 
  	if (index < adev->doorbell.num_kernel_doorbells)
-		atomic64_set((atomic64_t *)(adev->doorbell.ptr + index), v);
+		atomic64_set((atomic64_t *)(adev->doorbell.cpu_addr + index), v);
  	else
  		DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index);
  }
 
+/**
+ * amdgpu_doorbell_create_kernel_doorbells - Create kernel doorbells for graphics
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Creates doorbells for graphics driver usages.
+ * returns 0 on success, error otherwise.
+ */
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v) · line 129

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +10 -0
+{
+	int r;
+	int size;
+
+	/* Reserve first num_kernel_doorbells (page-aligned) for kernel ops */
+	size = ALIGN(adev->doorbell.num_kernel_doorbells * sizeof(u32), PAGE_SIZE);
+
+	/* Allocate an extra page for MES kernel usages (ring test) */
+	adev->mes.db_start_dw_offset = size / sizeof(u32);
+	size += PAGE_SIZE;

This is initialized but not used, so I suspect it will temporary break the MES test? Or is the MES just implicitly using the correct offset somehow?

Apart from that it looks good to me,
Christian.

Quoted 83 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +20 -9
+
+	r = amdgpu_bo_create_kernel(adev,
+				    size,
+				    PAGE_SIZE,
+				    AMDGPU_GEM_DOMAIN_DOORBELL,
+				    &adev->doorbell.kernel_doorbells,
+				    NULL,
+				    (void **)&adev->doorbell.cpu_addr);
+	if (r) {
+		DRM_ERROR("Failed to allocate kernel doorbells, err=%d\n", r);
+		return r;
+	}
+
+	adev->doorbell.num_kernel_doorbells = size / sizeof(u32);
+	return 0;
+}
+
  /*
   * GPU doorbell aperture helpers function.
   */
@@ -127,7 +163,6 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev)
  		adev->doorbell.base = 0;
  		adev->doorbell.size = 0;
  		adev->doorbell.num_kernel_doorbells = 0;
-		adev->doorbell.ptr = NULL;
  		return 0;
  	}
 
@@ -156,12 +191,6 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev)
  	if (adev->asic_type >= CHIP_VEGA10)
  		adev->doorbell.num_kernel_doorbells += 0x400;
 
-	adev->doorbell.ptr = ioremap(adev->doorbell.base,
-				     adev->doorbell.num_kernel_doorbells *
-				     sizeof(u32));
-	if (adev->doorbell.ptr == NULL)
-		return -ENOMEM;
-
  	return 0;
  }
 
@@ -174,6 +203,7 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev)
   */
  void amdgpu_doorbell_fini(struct amdgpu_device *adev)
  {
-	iounmap(adev->doorbell.ptr);
-	adev->doorbell.ptr = NULL;
+	amdgpu_bo_free_kernel(&adev->doorbell.kernel_doorbells,
+			      NULL,
+			      (void **)&adev->doorbell.cpu_addr);
  }
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +3 -0
index 547ec35691fa..a403418d5eac 100644
@@ -128,6 +128,9 @@ struct amdgpu_mes {
  	int                             (*kiq_hw_init)(struct amdgpu_device *adev);
  	int                             (*kiq_hw_fini)(struct amdgpu_device *adev);
 
+	/* MES doorbells */
+	uint32_t			db_start_dw_offset;
+
  	/* ip specific functions */
  	const struct amdgpu_mes_funcs   *funcs;
  };
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +7 -0
index 6d1587bbda52..c48e97646541 100644
@@ -1854,6 +1854,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
  		return r;
  	}
 
+	/* Create a boorbell page for kernel usages */
+	r = amdgpu_doorbell_create_kernel_doorbells(adev);
+	if (r) {
+		DRM_ERROR("Failed to initialize kernel doorbells.\n");
+		return r;
+	}
+
  	/* Initialize preemptible memory pool */
  	r = amdgpu_preempt_mgr_init(adev);
  	if (r) {
#8 lore raw

Re: [PATCH v3 06/12] drm/amdgpu: create kernel doorbell pages

Hey Christian,

Quoted 108 lines · Christian König

Am 20.06.23 um 19:16 schrieb Shashank Sharma:

Quoted 107 lines

This patch:
- creates a doorbell page for graphics driver usages.
- adds a few new varlables in adev->doorbell structure to keep track of kernel's doorbell-bo.
- removes the adev->doorbell.ptr variable, replaces it with kernel-doorbell-bo's cpu address.

V2: - Create doorbell BO directly, no wrappe functions (Alex)
     - no additional doorbell structure (Alex, Christian)
     - Use doorbell_cpu_ptr, remove ioremap (Christian, Alex)
     - Allocate one extra page of doorbells for MES (Alex)

Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |  8 ++- .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 56 ++++++++++++++----- drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |  3 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  7 +++

  4 files changed, 60 insertions(+), 14 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h +7 -1
index 783e2b8b086c..002899edb9d7 100644
@@ -31,10 +31,15 @@ struct amdgpu_doorbell {
       /* doorbell mmio */
       resource_size_t        base;
       resource_size_t        size;
-    u32 __iomem        *ptr;
         /* Number of doorbells reserved for amdgpu kernel driver */
       u32 num_kernel_doorbells;
+
+    /* Kernel doorbells */
+    struct amdgpu_bo *kernel_doorbells;
+
+    /* For CPU access of doorbells */
+    uint32_t *cpu_addr;
   };
     /* Reserved doorbells for amdgpu (including multimedia).
@@ -311,6 +316,7 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device 
 *adev, u32 index, u64 v);
    */
   int amdgpu_doorbell_init(struct amdgpu_device *adev);
   void amdgpu_doorbell_fini(struct amdgpu_device *adev);
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device 
 *adev);
     #define RDOORBELL32(index) amdgpu_mm_rdoorbell(adev, (index))

  #define WDOORBELL32(index, v) amdgpu_mm_wdoorbell(adev, (index), (v))

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +2 -2
index eb113e38f5e9..118f4bed32fd 100644
@@ -39,7 +39,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, 
 u32 index)
           return 0;
         if (index < adev->doorbell.num_kernel_doorbells)
-        return readl(adev->doorbell.ptr + index);
+        return readl(adev->doorbell.cpu_addr + index);
         DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);
       return 0;
@@ -61,7 +61,7 @@ void amdgpu_mm_wdoorbell(struct amdgpu_device 
 *adev, u32 index, u32 v)
           return;
         if (index < adev->doorbell.num_kernel_doorbells)
-        writel(v, adev->doorbell.ptr + index);
+        writel(v, adev->doorbell.cpu_addr + index);

      else
          DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index); }

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +2 -1
@@ -81,7 +81,7 @@ u64 amdgpu_mm_rdoorbell64(struct amdgpu_device 
 *adev, u32 index)
           return 0;
         if (index < adev->doorbell.num_kernel_doorbells)
-        return atomic64_read((atomic64_t *)(adev->doorbell.ptr + 
 index));
+        return atomic64_read((atomic64_t *)(adev->doorbell.cpu_addr 
+ index));
         DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);

      return 0;

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +1 -1
@@ -103,11 +103,47 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device 
 *adev, u32 index, u64 v)
           return;
         if (index < adev->doorbell.num_kernel_doorbells)
-        atomic64_set((atomic64_t *)(adev->doorbell.ptr + index), v);
+        atomic64_set((atomic64_t *)(adev->doorbell.cpu_addr + 
 index), v);

      else
          DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index); }
  +/**
+ * amdgpu_doorbell_create_kernel_doorbells - Create kernel doorbells for graphics
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Creates doorbells for graphics driver usages.
+ * returns 0 on success, error otherwise.
+ */

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +6 -0
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev)
+{
+    int r;
+    int size;
+
+    /* Reserve first num_kernel_doorbells (page-aligned) for kernel 

ops */

Quoted 8 lines
Quoted 6 lines

+    size = ALIGN(adev->doorbell.num_kernel_doorbells * sizeof(u32), PAGE_SIZE);
+
+    /* Allocate an extra page for MES kernel usages (ring test) */ +    adev->mes.db_start_dw_offset = size / sizeof(u32);
+    size += PAGE_SIZE;

This is initialized but not used, so I suspect it will temporary break the MES test? Or is the MES just implicitly using the correct offset somehow?

I am not sure if I understand this comment properly, can you please elaborate ?

If you are talking about 'size', its just used below while creating the doorbell BO.

If you mean adev->mes.db_start_dw_offset, its being used in amdgpu_mes_kernel_doorbell_get/free and amdgpu_mes_doorbell_init

So both of the variables are being used.

- Shashank

Quoted 83 lines

Apart from that it looks good to me,
Christian.

Quoted 78 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +17 -0
+
+    r = amdgpu_bo_create_kernel(adev,
+                    size,
+                    PAGE_SIZE,
+                    AMDGPU_GEM_DOMAIN_DOORBELL,
+                    &adev->doorbell.kernel_doorbells,
+                    NULL,
+                    (void **)&adev->doorbell.cpu_addr);
+    if (r) {
+        DRM_ERROR("Failed to allocate kernel doorbells, err=%d\n", r);
+        return r;
+    }
+
+    adev->doorbell.num_kernel_doorbells = size / sizeof(u32);
+    return 0;
+}
+

  /*
   * GPU doorbell aperture helpers function.
   */

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +0 -1
@@ -127,7 +163,6 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev)
           adev->doorbell.base = 0;
           adev->doorbell.size = 0;
           adev->doorbell.num_kernel_doorbells = 0;
-        adev->doorbell.ptr = NULL;
           return 0;
       }
   @@ -156,12 +191,6 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev)

      if (adev->asic_type >= CHIP_VEGA10) adev->doorbell.num_kernel_doorbells += 0x400;
  -    adev->doorbell.ptr = ioremap(adev->doorbell.base,
-                     adev->doorbell.num_kernel_doorbells *
-                     sizeof(u32));
-    if (adev->doorbell.ptr == NULL)
-        return -ENOMEM;
-
      return 0;
  }
  @@ -174,6 +203,7 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev) */
  void amdgpu_doorbell_fini(struct amdgpu_device *adev)
  {
-    iounmap(adev->doorbell.ptr);
-    adev->doorbell.ptr = NULL;
+ amdgpu_bo_free_kernel(&adev->doorbell.kernel_doorbells,
+                  NULL,
+                  (void **)&adev->doorbell.cpu_addr);
  }

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +0 -0
index 547ec35691fa..a403418d5eac 100644
@@ -128,6 +128,9 @@ struct amdgpu_mes {
       int                             (*kiq_hw_init)(struct amdgpu_device *adev);
       int                             (*kiq_hw_fini)(struct amdgpu_device *adev);

  +    /* MES doorbells */
+    uint32_t            db_start_dw_offset;
+
      /* ip specific functions */
      const struct amdgpu_mes_funcs   *funcs;
  };

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +0 -0
index 6d1587bbda52..c48e97646541 100644
@@ -1854,6 +1854,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
           return r;
       }

  +    /* Create a boorbell page for kernel usages */ +    r = amdgpu_doorbell_create_kernel_doorbells(adev); +    if (r) {
+        DRM_ERROR("Failed to initialize kernel doorbells.\n"); +        return r;
+    }
+
      /* Initialize preemptible memory pool */
      r = amdgpu_preempt_mgr_init(adev);
      if (r) {

maintainer
#9 lore raw

Re: [PATCH v3 06/12] drm/amdgpu: create kernel doorbell pages

Am 21.06.23 um 11:10 schrieb Shashank Sharma:

Quoted 122 lines

Hey Christian,

Quoted 110 lines · Christian König

Am 20.06.23 um 19:16 schrieb Shashank Sharma:

Quoted 109 lines

This patch:
- creates a doorbell page for graphics driver usages.
- adds a few new varlables in adev->doorbell structure to keep track of kernel's doorbell-bo.
- removes the adev->doorbell.ptr variable, replaces it with kernel-doorbell-bo's cpu address.

V2: - Create doorbell BO directly, no wrappe functions (Alex)
     - no additional doorbell structure (Alex, Christian)
     - Use doorbell_cpu_ptr, remove ioremap (Christian, Alex)
     - Allocate one extra page of doorbells for MES (Alex)

Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |  8 ++- .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 56
++++++++++++++-----
  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |  3 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  7 +++

  4 files changed, 60 insertions(+), 14 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h +7 -1
index 783e2b8b086c..002899edb9d7 100644
@@ -31,10 +31,15 @@ struct amdgpu_doorbell {
       /* doorbell mmio */
       resource_size_t        base;
       resource_size_t        size;
-    u32 __iomem        *ptr;
         /* Number of doorbells reserved for amdgpu kernel driver */
       u32 num_kernel_doorbells;
+
+    /* Kernel doorbells */
+    struct amdgpu_bo *kernel_doorbells;
+
+    /* For CPU access of doorbells */
+    uint32_t *cpu_addr;
   };
     /* Reserved doorbells for amdgpu (including multimedia).
@@ -311,6 +316,7 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device 
 *adev, u32 index, u64 v);
    */
   int amdgpu_doorbell_init(struct amdgpu_device *adev);
   void amdgpu_doorbell_fini(struct amdgpu_device *adev);
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device 
 *adev);
     #define RDOORBELL32(index) amdgpu_mm_rdoorbell(adev, (index))

  #define WDOORBELL32(index, v) amdgpu_mm_wdoorbell(adev, (index), (v))

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +2 -2
index eb113e38f5e9..118f4bed32fd 100644
@@ -39,7 +39,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device 
 *adev, u32 index)
           return 0;
         if (index < adev->doorbell.num_kernel_doorbells)
-        return readl(adev->doorbell.ptr + index);
+        return readl(adev->doorbell.cpu_addr + index);
         DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);
       return 0;
@@ -61,7 +61,7 @@ void amdgpu_mm_wdoorbell(struct amdgpu_device 
 *adev, u32 index, u32 v)
           return;
         if (index < adev->doorbell.num_kernel_doorbells)
-        writel(v, adev->doorbell.ptr + index);
+        writel(v, adev->doorbell.cpu_addr + index);

      else
          DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index); }

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +2 -1
@@ -81,7 +81,7 @@ u64 amdgpu_mm_rdoorbell64(struct amdgpu_device 
 *adev, u32 index)
           return 0;
         if (index < adev->doorbell.num_kernel_doorbells)
-        return atomic64_read((atomic64_t *)(adev->doorbell.ptr + 
 index));
+        return atomic64_read((atomic64_t *)(adev->doorbell.cpu_addr 
+ index));
         DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);

      return 0;

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +1 -1
@@ -103,11 +103,47 @@ void amdgpu_mm_wdoorbell64(struct 
 amdgpu_device *adev, u32 index, u64 v)
           return;
         if (index < adev->doorbell.num_kernel_doorbells)
-        atomic64_set((atomic64_t *)(adev->doorbell.ptr + index), v);
+        atomic64_set((atomic64_t *)(adev->doorbell.cpu_addr + 
 index), v);

      else
          DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index); }
  +/**
+ * amdgpu_doorbell_create_kernel_doorbells - Create kernel doorbells for graphics
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Creates doorbells for graphics driver usages.
+ * returns 0 on success, error otherwise.
+ */
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +5 -0
+{
+    int r;
+    int size;
+
+    /* Reserve first num_kernel_doorbells (page-aligned) for kernel 

ops */

Quoted 8 lines
Quoted 6 lines

+    size = ALIGN(adev->doorbell.num_kernel_doorbells * sizeof(u32), PAGE_SIZE);
+
+    /* Allocate an extra page for MES kernel usages (ring test) */ +    adev->mes.db_start_dw_offset = size / sizeof(u32);
+    size += PAGE_SIZE;

This is initialized but not used, so I suspect it will temporary break the MES test? Or is the MES just implicitly using the correct offset somehow?

Quoted 5 lines

I am not sure if I understand this comment properly, can you please elaborate ?

If you are talking about 'size', its just used below while creating the doorbell BO.

If you mean adev->mes.db_start_dw_offset, its being used in amdgpu_mes_kernel_doorbell_get/free and amdgpu_mes_doorbell_init

I was talking about adev->mes.db_start_dw_offset. But using this in amdgpu_mes_kernel_doorbell_get/free comes in a later patch, doesn't it? If yes, doesn't that break the MES test?

Christian.

Quoted 89 lines

So both of the variables are being used.

- Shashank

Quoted 84 lines

Apart from that it looks good to me,
Christian.

Quoted 79 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +17 -0
+
+    r = amdgpu_bo_create_kernel(adev,
+                    size,
+                    PAGE_SIZE,
+                    AMDGPU_GEM_DOMAIN_DOORBELL,
+                    &adev->doorbell.kernel_doorbells,
+                    NULL,
+                    (void **)&adev->doorbell.cpu_addr);
+    if (r) {
+        DRM_ERROR("Failed to allocate kernel doorbells, err=%d\n", r);
+        return r;
+    }
+
+    adev->doorbell.num_kernel_doorbells = size / sizeof(u32);
+    return 0;
+}
+

  /*
   * GPU doorbell aperture helpers function.
   */

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +0 -1
@@ -127,7 +163,6 @@ int amdgpu_doorbell_init(struct amdgpu_device 
 *adev)
           adev->doorbell.base = 0;
           adev->doorbell.size = 0;
           adev->doorbell.num_kernel_doorbells = 0;
-        adev->doorbell.ptr = NULL;
           return 0;
       }

  @@ -156,12 +191,6 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev) if (adev->asic_type >= CHIP_VEGA10) adev->doorbell.num_kernel_doorbells += 0x400;
  -    adev->doorbell.ptr = ioremap(adev->doorbell.base,
-                     adev->doorbell.num_kernel_doorbells *
-                     sizeof(u32));
-    if (adev->doorbell.ptr == NULL)
-        return -ENOMEM;
-
      return 0;
  }
  @@ -174,6 +203,7 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev) */
  void amdgpu_doorbell_fini(struct amdgpu_device *adev)
  {
-    iounmap(adev->doorbell.ptr);
-    adev->doorbell.ptr = NULL;
+ amdgpu_bo_free_kernel(&adev->doorbell.kernel_doorbells,
+                  NULL,
+                  (void **)&adev->doorbell.cpu_addr);
  }

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +0 -0
index 547ec35691fa..a403418d5eac 100644
@@ -128,6 +128,9 @@ struct amdgpu_mes {
       int                             (*kiq_hw_init)(struct amdgpu_device *adev);
       int                             (*kiq_hw_fini)(struct amdgpu_device *adev);

  +    /* MES doorbells */
+    uint32_t            db_start_dw_offset;
+
      /* ip specific functions */
      const struct amdgpu_mes_funcs   *funcs;
  };

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +0 -0
index 6d1587bbda52..c48e97646541 100644
@@ -1854,6 +1854,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
           return r;
       }

  +    /* Create a boorbell page for kernel usages */ +    r = amdgpu_doorbell_create_kernel_doorbells(adev); +    if (r) {
+        DRM_ERROR("Failed to initialize kernel doorbells.\n"); +        return r;
+    }
+
      /* Initialize preemptible memory pool */
      r = amdgpu_preempt_mgr_init(adev);
      if (r) {

#10 lore raw

Re: [PATCH v3 06/12] drm/amdgpu: create kernel doorbell pages

Quoted 125 lines · Christian König

Am 21.06.23 um 11:10 schrieb Shashank Sharma:

Quoted 122 lines

Hey Christian,

Quoted 118 lines · Christian König

Am 20.06.23 um 19:16 schrieb Shashank Sharma:

Quoted 108 lines

This patch:
- creates a doorbell page for graphics driver usages.
- adds a few new varlables in adev->doorbell structure to keep track of kernel's doorbell-bo.
- removes the adev->doorbell.ptr variable, replaces it with kernel-doorbell-bo's cpu address.

V2: - Create doorbell BO directly, no wrappe functions (Alex)
     - no additional doorbell structure (Alex, Christian)
     - Use doorbell_cpu_ptr, remove ioremap (Christian, Alex)
     - Allocate one extra page of doorbells for MES (Alex)

Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |  8 ++- .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 56
++++++++++++++-----
  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |  3 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  7 +++

  4 files changed, 60 insertions(+), 14 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h +7 -1
index 783e2b8b086c..002899edb9d7 100644
@@ -31,10 +31,15 @@ struct amdgpu_doorbell {
       /* doorbell mmio */
       resource_size_t        base;
       resource_size_t        size;
-    u32 __iomem        *ptr;
         /* Number of doorbells reserved for amdgpu kernel driver */
       u32 num_kernel_doorbells;
+
+    /* Kernel doorbells */
+    struct amdgpu_bo *kernel_doorbells;
+
+    /* For CPU access of doorbells */
+    uint32_t *cpu_addr;
   };
     /* Reserved doorbells for amdgpu (including multimedia).
@@ -311,6 +316,7 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device 
 *adev, u32 index, u64 v);
    */
   int amdgpu_doorbell_init(struct amdgpu_device *adev);
   void amdgpu_doorbell_fini(struct amdgpu_device *adev);
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device 
 *adev);
     #define RDOORBELL32(index) amdgpu_mm_rdoorbell(adev, (index))

  #define WDOORBELL32(index, v) amdgpu_mm_wdoorbell(adev, (index), (v))

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +2 -2
index eb113e38f5e9..118f4bed32fd 100644
@@ -39,7 +39,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device 
 *adev, u32 index)
           return 0;
         if (index < adev->doorbell.num_kernel_doorbells)
-        return readl(adev->doorbell.ptr + index);
+        return readl(adev->doorbell.cpu_addr + index);
         DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);
       return 0;
@@ -61,7 +61,7 @@ void amdgpu_mm_wdoorbell(struct amdgpu_device 
 *adev, u32 index, u32 v)
           return;
         if (index < adev->doorbell.num_kernel_doorbells)
-        writel(v, adev->doorbell.ptr + index);
+        writel(v, adev->doorbell.cpu_addr + index);

      else
          DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index); }

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +1 -1
@@ -81,7 +81,7 @@ u64 amdgpu_mm_rdoorbell64(struct amdgpu_device 
 *adev, u32 index)
           return 0;
         if (index < adev->doorbell.num_kernel_doorbells)
-        return atomic64_read((atomic64_t *)(adev->doorbell.ptr + 
 index));
+        return atomic64_read((atomic64_t 
 *)(adev->doorbell.cpu_addr + index));
         DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);

      return 0;

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +1 -1
@@ -103,11 +103,47 @@ void amdgpu_mm_wdoorbell64(struct 
 amdgpu_device *adev, u32 index, u64 v)
           return;
         if (index < adev->doorbell.num_kernel_doorbells)
-        atomic64_set((atomic64_t *)(adev->doorbell.ptr + index), v);
+        atomic64_set((atomic64_t *)(adev->doorbell.cpu_addr + 
 index), v);

      else
          DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index); }
  +/**
+ * amdgpu_doorbell_create_kernel_doorbells - Create kernel doorbells for graphics
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Creates doorbells for graphics driver usages.
+ * returns 0 on success, error otherwise.
+ */
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +5 -0
+{
+    int r;
+    int size;
+
+    /* Reserve first num_kernel_doorbells (page-aligned) for 
Quoted 7 lines

kernel ops */
+    size = ALIGN(adev->doorbell.num_kernel_doorbells * sizeof(u32), PAGE_SIZE);
+
+    /* Allocate an extra page for MES kernel usages (ring test) */ +    adev->mes.db_start_dw_offset = size / sizeof(u32); +    size += PAGE_SIZE;

This is initialized but not used, so I suspect it will temporary break the MES test? Or is the MES just implicitly using the correct offset somehow?

Quoted 8 lines
Quoted 5 lines

I am not sure if I understand this comment properly, can you please elaborate ?

If you are talking about 'size', its just used below while creating the doorbell BO.

If you mean adev->mes.db_start_dw_offset, its being used in amdgpu_mes_kernel_doorbell_get/free and amdgpu_mes_doorbell_init

I was talking about adev->mes.db_start_dw_offset. But using this in amdgpu_mes_kernel_doorbell_get/free comes in a later patch, doesn't it? If yes, doesn't that break the MES test?

Ah, I get it now, and you are right, I though it would be better for reviewer to see all KGD kernel level pages being created in the same patch, but now I feel it would have been far better to do this in MES kernel specific patch.

I will move this code in MES kernel doorbells page.

- Shashank

Quoted 93 lines

Christian.

Quoted 90 lines

So both of the variables are being used.

- Shashank

Quoted 85 lines

Apart from that it looks good to me,
Christian.

Quoted 80 lines

+
+    r = amdgpu_bo_create_kernel(adev,
+                    size,
+                    PAGE_SIZE,
+                    AMDGPU_GEM_DOMAIN_DOORBELL,
+ &adev->doorbell.kernel_doorbells,
+                    NULL,
+                    (void **)&adev->doorbell.cpu_addr); +    if (r) {
+        DRM_ERROR("Failed to allocate kernel doorbells, err=%d\n", r);
+        return r;
+    }

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +5 -0
+
+    adev->doorbell.num_kernel_doorbells = size / sizeof(u32);
+    return 0;
+}
+

  /*
   * GPU doorbell aperture helpers function.
   */

drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +0 -1
@@ -127,7 +163,6 @@ int amdgpu_doorbell_init(struct amdgpu_device 
 *adev)
           adev->doorbell.base = 0;
           adev->doorbell.size = 0;
           adev->doorbell.num_kernel_doorbells = 0;
-        adev->doorbell.ptr = NULL;
           return 0;
       }

  @@ -156,12 +191,6 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev) if (adev->asic_type >= CHIP_VEGA10) adev->doorbell.num_kernel_doorbells += 0x400;
  -    adev->doorbell.ptr = ioremap(adev->doorbell.base,
-                     adev->doorbell.num_kernel_doorbells *
-                     sizeof(u32));
-    if (adev->doorbell.ptr == NULL)
-        return -ENOMEM;
-
      return 0;
  }
  @@ -174,6 +203,7 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev) */
  void amdgpu_doorbell_fini(struct amdgpu_device *adev)
  {
-    iounmap(adev->doorbell.ptr);
-    adev->doorbell.ptr = NULL;
+ amdgpu_bo_free_kernel(&adev->doorbell.kernel_doorbells,
+                  NULL,
+                  (void **)&adev->doorbell.cpu_addr);
  }

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +0 -0
index 547ec35691fa..a403418d5eac 100644
@@ -128,6 +128,9 @@ struct amdgpu_mes {
       int                             (*kiq_hw_init)(struct amdgpu_device *adev);
       int                             (*kiq_hw_fini)(struct amdgpu_device *adev);

  +    /* MES doorbells */
+    uint32_t            db_start_dw_offset;
+
      /* ip specific functions */
      const struct amdgpu_mes_funcs   *funcs;
  };

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +0 -0
index 6d1587bbda52..c48e97646541 100644
@@ -1854,6 +1854,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
           return r;
       }

  +    /* Create a boorbell page for kernel usages */ +    r = amdgpu_doorbell_create_kernel_doorbells(adev); +    if (r) {
+        DRM_ERROR("Failed to initialize kernel doorbells.\n"); +        return r;
+    }
+
      /* Initialize preemptible memory pool */
      r = amdgpu_preempt_mgr_init(adev);
      if (r) {

#11 lore raw

PATCH v4 6/12 drm/amdgpu: create kernel doorbell pages

In reply to Shashank Sharma

This patch:
- creates a doorbell page for graphics driver usages.
- adds a few new varlables in adev->doorbell structure to keep track of kernel's doorbell-bo.
- removes the adev->doorbell.ptr variable, replaces it with kernel-doorbell-bo's cpu address.

V2: - Create doorbell BO directly, no wrappe functions (Alex)
- no additional doorbell structure (Alex, Christian)
- Use doorbell_cpu_ptr, remove ioremap (Christian, Alex)
- Allocate one extra page of doorbells for MES (Alex)

V4: Move MES doorbell base init into MES related patch (Christian)

Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |  8 ++-
 .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 52 ++++++++++++++-----
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |  3 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  7 +++
 4 files changed, 56 insertions(+), 14 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h +7 -1
index 783e2b8b086c..002899edb9d7 100644
@@ -31,10 +31,15 @@ struct amdgpu_doorbell {
 	/* doorbell mmio */
 	resource_size_t		base;
 	resource_size_t		size;
-	u32 __iomem		*ptr;
 
 	/* Number of doorbells reserved for amdgpu kernel driver */
 	u32 num_kernel_doorbells;
+
+	/* Kernel doorbells */
+	struct amdgpu_bo *kernel_doorbells;
+
+	/* For CPU access of doorbells */
+	uint32_t *cpu_addr;
 };
 
 /* Reserved doorbells for amdgpu (including multimedia).
@@ -311,6 +316,7 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v);
  */
 int amdgpu_doorbell_init(struct amdgpu_device *adev);
 void amdgpu_doorbell_fini(struct amdgpu_device *adev);
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev);
 
 #define RDOORBELL32(index) amdgpu_mm_rdoorbell(adev, (index))
 #define WDOORBELL32(index, v) amdgpu_mm_wdoorbell(adev, (index), (v))
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +39 -13

96 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +3 -0
index 547ec35691fa..a403418d5eac 100644
@@ -128,6 +128,9 @@ struct amdgpu_mes {
 	int                             (*kiq_hw_init)(struct amdgpu_device *adev);
 	int                             (*kiq_hw_fini)(struct amdgpu_device *adev);
 
+	/* MES doorbells */
+	uint32_t			db_start_dw_offset;
+
 	/* ip specific functions */
 	const struct amdgpu_mes_funcs   *funcs;
 };
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +7 -0
index 6d1587bbda52..c48e97646541 100644
@@ -1854,6 +1854,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
 		return r;
 	}
 
+	/* Create a boorbell page for kernel usages */
+	r = amdgpu_doorbell_create_kernel_doorbells(adev);
+	if (r) {
+		DRM_ERROR("Failed to initialize kernel doorbells.\n");
+		return r;
+	}
+
 	/* Initialize preemptible memory pool */
 	r = amdgpu_preempt_mgr_init(adev);
 	if (r) {
Signature · 1 line
2.40.1
maintainer
#12 lore raw

Re: [PATCH v4 06/12] drm/amdgpu: create kernel doorbell pages

Am 21.06.23 um 14:44 schrieb Shashank Sharma:

Quoted 165 lines

This patch:
- creates a doorbell page for graphics driver usages.
- adds a few new varlables in adev->doorbell structure to keep track of kernel's doorbell-bo.
- removes the adev->doorbell.ptr variable, replaces it with kernel-doorbell-bo's cpu address.

V2: - Create doorbell BO directly, no wrappe functions (Alex)
- no additional doorbell structure (Alex, Christian)
- Use doorbell_cpu_ptr, remove ioremap (Christian, Alex)
- Allocate one extra page of doorbells for MES (Alex)

V4: Move MES doorbell base init into MES related patch (Christian)

Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |  8 ++-
  .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 52 ++++++++++++++-----
  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |  3 ++
  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  7 +++
  4 files changed, 56 insertions(+), 14 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h +7 -1
index 783e2b8b086c..002899edb9d7 100644
@@ -31,10 +31,15 @@ struct amdgpu_doorbell {
  	/* doorbell mmio */
  	resource_size_t		base;
  	resource_size_t		size;
-	u32 __iomem		*ptr;
 
  	/* Number of doorbells reserved for amdgpu kernel driver */
  	u32 num_kernel_doorbells;
+
+	/* Kernel doorbells */
+	struct amdgpu_bo *kernel_doorbells;
+
+	/* For CPU access of doorbells */
+	uint32_t *cpu_addr;
  };
 
  /* Reserved doorbells for amdgpu (including multimedia).
@@ -311,6 +316,7 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v);
   */
  int amdgpu_doorbell_init(struct amdgpu_device *adev);
  void amdgpu_doorbell_fini(struct amdgpu_device *adev);
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev);
 
  #define RDOORBELL32(index) amdgpu_mm_rdoorbell(adev, (index))
  #define WDOORBELL32(index, v) amdgpu_mm_wdoorbell(adev, (index), (v))
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +39 -13
index eb113e38f5e9..dd6744eb5875 100644
@@ -39,7 +39,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index)
  		return 0;
 
  	if (index < adev->doorbell.num_kernel_doorbells)
-		return readl(adev->doorbell.ptr + index);
+		return readl(adev->doorbell.cpu_addr + index);
 
  	DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);
  	return 0;
@@ -61,7 +61,7 @@ void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v)
  		return;
 
  	if (index < adev->doorbell.num_kernel_doorbells)
-		writel(v, adev->doorbell.ptr + index);
+		writel(v, adev->doorbell.cpu_addr + index);
  	else
  		DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index);
  }
@@ -81,7 +81,7 @@ u64 amdgpu_mm_rdoorbell64(struct amdgpu_device *adev, u32 index)
  		return 0;
 
  	if (index < adev->doorbell.num_kernel_doorbells)
-		return atomic64_read((atomic64_t *)(adev->doorbell.ptr + index));
+		return atomic64_read((atomic64_t *)(adev->doorbell.cpu_addr + index));
 
  	DRM_ERROR("reading beyond doorbell aperture: 0x%08x!\n", index);
  	return 0;
@@ -103,11 +103,43 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v)
  		return;
 
  	if (index < adev->doorbell.num_kernel_doorbells)
-		atomic64_set((atomic64_t *)(adev->doorbell.ptr + index), v);
+		atomic64_set((atomic64_t *)(adev->doorbell.cpu_addr + index), v);
  	else
  		DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index);
  }
 
+/**
+ * amdgpu_doorbell_create_kernel_doorbells - Create kernel doorbells for graphics
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Creates doorbells for graphics driver usages.
+ * returns 0 on success, error otherwise.
+ */
+int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev)
+{
+	int r;
+	int size;
+
+	/* Reserve first num_kernel_doorbells (page-aligned) for kernel ops */
+	size = ALIGN(adev->doorbell.num_kernel_doorbells * sizeof(u32), PAGE_SIZE);
+
+	r = amdgpu_bo_create_kernel(adev,
+				    size,
+				    PAGE_SIZE,
+				    AMDGPU_GEM_DOMAIN_DOORBELL,
+				    &adev->doorbell.kernel_doorbells,
+				    NULL,
+				    (void **)&adev->doorbell.cpu_addr);
+	if (r) {
+		DRM_ERROR("Failed to allocate kernel doorbells, err=%d\n", r);
+		return r;
+	}
+
+	adev->doorbell.num_kernel_doorbells = size / sizeof(u32);
+	return 0;
+}
+
  /*
   * GPU doorbell aperture helpers function.
   */
@@ -127,7 +159,6 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev)
  		adev->doorbell.base = 0;
  		adev->doorbell.size = 0;
  		adev->doorbell.num_kernel_doorbells = 0;
-		adev->doorbell.ptr = NULL;
  		return 0;
  	}
 
@@ -156,12 +187,6 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev)
  	if (adev->asic_type >= CHIP_VEGA10)
  		adev->doorbell.num_kernel_doorbells += 0x400;
 
-	adev->doorbell.ptr = ioremap(adev->doorbell.base,
-				     adev->doorbell.num_kernel_doorbells *
-				     sizeof(u32));
-	if (adev->doorbell.ptr == NULL)
-		return -ENOMEM;
-
  	return 0;
  }
 
@@ -174,6 +199,7 @@ int amdgpu_doorbell_init(struct amdgpu_device *adev)
   */
  void amdgpu_doorbell_fini(struct amdgpu_device *adev)
  {
-	iounmap(adev->doorbell.ptr);
-	adev->doorbell.ptr = NULL;
+	amdgpu_bo_free_kernel(&adev->doorbell.kernel_doorbells,
+			      NULL,
+			      (void **)&adev->doorbell.cpu_addr);
  }
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +0 -0
index 547ec35691fa..a403418d5eac 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h struct amdgpu_mes { · line 133

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +3 -0
@@ -128,6 +128,9 @@ struct amdgpu_mes {
  	int                             (*kiq_hw_init)(struct amdgpu_device *adev);
  	int                             (*kiq_hw_fini)(struct amdgpu_device *adev);
 
+	/* MES doorbells */
+	uint32_t			db_start_dw_offset;
+

This should probably go into patch #11 as well, apart from that the patch is Reviewed-by: Christian König <[email protected]>

Regards,
Christian.

Quoted 21 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +0 -0
  	/* ip specific functions */
  	const struct amdgpu_mes_funcs   *funcs;
  };
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +7 -0
index 6d1587bbda52..c48e97646541 100644
@@ -1854,6 +1854,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
  		return r;
  	}
 
+	/* Create a boorbell page for kernel usages */
+	r = amdgpu_doorbell_create_kernel_doorbells(adev);
+	if (r) {
+		DRM_ERROR("Failed to initialize kernel doorbells.\n");
+		return r;
+	}
+
  	/* Initialize preemptible memory pool */
  	r = amdgpu_preempt_mgr_init(adev);
  	if (r) {
#13 lore raw

PATCH v3 7/12 get absolute offset from doorbell index

In reply to Shashank Sharma

This patch adds a helper function which converts a doorbell's relative index in a BO to an absolute doorbell offset in the doorbell BAR.

V2: No space between the variable name doc (Luben)

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h  |  3 +++
 .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  | 21 +++++++++++++++++++
 2 files changed, 24 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h +3 -0
index 002899edb9d7..48e3c848ddfd 100644
@@ -317,6 +317,9 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v);
 int amdgpu_doorbell_init(struct amdgpu_device *adev);
 void amdgpu_doorbell_fini(struct amdgpu_device *adev);
 int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev);
+uint32_t amdgpu_doorbell_index_on_bar(struct amdgpu_device *adev,
+				       struct amdgpu_bo *db_bo,
+				       uint32_t doorbell_index);
 
 #define RDOORBELL32(index) amdgpu_mm_rdoorbell(adev, (index))
 #define WDOORBELL32(index, v) amdgpu_mm_wdoorbell(adev, (index), (v))
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +21 -0
index 118f4bed32fd..e1599fd72954 100644
@@ -108,6 +108,27 @@ void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v)
 		DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index);
 }
 
+/**
+ * amdgpu_doorbell_index_on_bar - Find doorbell's absolute offset in BAR
+ *
+ * @adev: amdgpu_device pointer
+ * @db_bo: doorbell object's bo
+ * @db_index: doorbell relative index in this doorbell object
+ *
+ * returns doorbell's absolute index in BAR
+ */
+uint32_t amdgpu_doorbell_index_on_bar(struct amdgpu_device *adev,
+				       struct amdgpu_bo *db_bo,
+				       uint32_t doorbell_index)
+{
+	int db_bo_offset;
+
+	db_bo_offset = amdgpu_bo_gpu_offset_no_check(db_bo);
+
+	/* doorbell index is 32 bit but doorbell's size is 64-bit, so *2 */
+	return db_bo_offset / sizeof(u32) + doorbell_index * 2;
+}
+
 /**
  * amdgpu_doorbell_create_kernel_doorbells - Create kernel doorbells for graphics
  *
Signature · 1 line
2.40.1
#14 lore raw

PATCH v3 8/12 drm/amdgpu: use doorbell manager for kfd kernel doorbells

In reply to Shashank Sharma

This patch:
- adds a doorbell bo in kfd device structure.
- creates doorbell page for kfd kernel usages.
- updates the get_kernel_doorbell and free_kernel_doorbell functions accordingly

V2: Do not use wrapper API, use direct amdgpu_create_kernel(Alex) V3:
- Move single variable declaration below (Christian)
- Add a to-do item to reuse the KGD kernel level doorbells for KFD for non-MES cases, instead of reserving one page (Felix)

Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>
Cc Felix Kuehling <[email protected]>
---
 drivers/gpu/drm/amd/amdkfd/kfd_device.c   |   2 -
 drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c | 109 +++++++---------------
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h     |   6 ++
 3 files changed, 39 insertions(+), 78 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_device.c +0 -2
index 00f528eb9812..36fbe9c840ee 100644
@@ -437,8 +437,6 @@ struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf)
 	atomic_set(&kfd->compute_profile, 0);
 
 	mutex_init(&kfd->doorbell_mutex);
-	memset(&kfd->doorbell_available_index, 0,
-		sizeof(kfd->doorbell_available_index));
 
 	atomic_set(&kfd->sram_ecc_flag, 0);
 
drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c +33 -76

147 lines not on this page · raw message

drivers/gpu/drm/amd/amdkfd/kfd_priv.h +6 -0
index 94a438956868..42c215782d72 100644
@@ -349,6 +349,12 @@ struct kfd_dev {
 
 	/* HMM page migration MEMORY_DEVICE_PRIVATE mapping */
 	struct dev_pagemap pgmap;
+
+	/* Kernel doorbells for KFD device */
+	struct amdgpu_bo *doorbells;
+
+	/* bitmap for dynamic doorbell allocation from this object */
+	unsigned long *doorbell_bitmap;
 };
 
 enum kfd_mempool {
Signature · 1 line
2.40.1
maintainer Reviewed-by on v3 8/12
#15 lore raw

Re: [PATCH v3 08/12] drm/amdgpu: use doorbell manager for kfd kernel doorbells

Quoted 12 lines · Shashank Sharma

This patch:
- adds a doorbell bo in kfd device structure.
- creates doorbell page for kfd kernel usages.
- updates the get_kernel_doorbell and free_kernel_doorbell functions accordingly

V2: Do not use wrapper API, use direct amdgpu_create_kernel(Alex) V3:
- Move single variable declaration below (Christian)
- Add a to-do item to reuse the KGD kernel level doorbells for KFD for non-MES cases, instead of reserving one page (Felix)

Quoted 4 lines
Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>
Cc Felix Kuehling <[email protected]>
Quoted 190 lines
---
  drivers/gpu/drm/amd/amdkfd/kfd_device.c   |   2 -
  drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c | 109 +++++++---------------
  drivers/gpu/drm/amd/amdkfd/kfd_priv.h     |   6 ++
  3 files changed, 39 insertions(+), 78 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_device.c +0 -2
index 00f528eb9812..36fbe9c840ee 100644
@@ -437,8 +437,6 @@ struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf)
  	atomic_set(&kfd->compute_profile, 0);
 
  	mutex_init(&kfd->doorbell_mutex);
-	memset(&kfd->doorbell_available_index, 0,
-		sizeof(kfd->doorbell_available_index));
 
  	atomic_set(&kfd->sram_ecc_flag, 0);
 
drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c +33 -76
index 38c9e1ca6691..f7d45057ed32 100644
@@ -61,81 +61,46 @@ size_t kfd_doorbell_process_slice(struct kfd_dev *kfd)
  /* Doorbell calculations for device init. */
  int kfd_doorbell_init(struct kfd_dev *kfd)
  {
-	size_t doorbell_start_offset;
-	size_t doorbell_aperture_size;
-	size_t doorbell_process_limit;
+	int size = PAGE_SIZE;
+	int r;
 
  	/*
-	 * With MES enabled, just set the doorbell base as it is needed
-	 * to calculate doorbell physical address.
-	 */
-	if (kfd->shared_resources.enable_mes) {
-		kfd->doorbell_base =
-			kfd->shared_resources.doorbell_physical_address;
-		return 0;
-	}
-
-	/*
-	 * We start with calculations in bytes because the input data might
-	 * only be byte-aligned.
-	 * Only after we have done the rounding can we assume any alignment.
+	 * Todo: KFD kernel level operations need only one doorbell for
+	 * ring test/HWS. So instead of reserving a whole page here for
+	 * kernel, reserve and consume a doorbell from existing KGD kernel
+	 * doorbell page.
  	 */
 
-	doorbell_start_offset =
-			roundup(kfd->shared_resources.doorbell_start_offset,
-					kfd_doorbell_process_slice(kfd));
-
-	doorbell_aperture_size =
-			rounddown(kfd->shared_resources.doorbell_aperture_size,
-					kfd_doorbell_process_slice(kfd));
-
-	if (doorbell_aperture_size > doorbell_start_offset)
-		doorbell_process_limit =
-			(doorbell_aperture_size - doorbell_start_offset) /
-						kfd_doorbell_process_slice(kfd);
-	else
-		return -ENOSPC;
-
-	if (!kfd->max_doorbell_slices ||
-	    doorbell_process_limit < kfd->max_doorbell_slices)
-		kfd->max_doorbell_slices = doorbell_process_limit;
-
-	kfd->doorbell_base = kfd->shared_resources.doorbell_physical_address +
-				doorbell_start_offset;
-
-	kfd->doorbell_base_dw_offset = doorbell_start_offset / sizeof(u32);
-
-	kfd->doorbell_kernel_ptr = ioremap(kfd->doorbell_base,
-					   kfd_doorbell_process_slice(kfd));
-
-	if (!kfd->doorbell_kernel_ptr)
+	/* Bitmap to dynamically allocate doorbells from kernel page */
+	kfd->doorbell_bitmap = bitmap_zalloc(size / sizeof(u32), GFP_KERNEL);
+	if (!kfd->doorbell_bitmap) {
+		DRM_ERROR("Failed to allocate kernel doorbell bitmap\n");
  		return -ENOMEM;
+	}
 
-	pr_debug("Doorbell initialization:\n");
-	pr_debug("doorbell base           == 0x%08lX\n",
-			(uintptr_t)kfd->doorbell_base);
-
-	pr_debug("doorbell_base_dw_offset      == 0x%08lX\n",
-			kfd->doorbell_base_dw_offset);
-
-	pr_debug("doorbell_process_limit  == 0x%08lX\n",
-			doorbell_process_limit);
-
-	pr_debug("doorbell_kernel_offset  == 0x%08lX\n",
-			(uintptr_t)kfd->doorbell_base);
-
-	pr_debug("doorbell aperture size  == 0x%08lX\n",
-			kfd->shared_resources.doorbell_aperture_size);
-
-	pr_debug("doorbell kernel address == %p\n", kfd->doorbell_kernel_ptr);
+	/* Alloc a doorbell page for KFD kernel usages */
+	r = amdgpu_bo_create_kernel(kfd->adev,
+				    size,
+				    PAGE_SIZE,
+				    AMDGPU_GEM_DOMAIN_DOORBELL,
+				    &kfd->doorbells,
+				    NULL,
+				    (void **)&kfd->doorbell_kernel_ptr);
+	if (r) {
+		pr_err("failed to allocate kernel doorbells\n");
+		bitmap_free(kfd->doorbell_bitmap);
+		return r;
+	}
 
+	pr_debug("Doorbell kernel address == %p\n", kfd->doorbell_kernel_ptr);
  	return 0;
  }
 
  void kfd_doorbell_fini(struct kfd_dev *kfd)
  {
-	if (kfd->doorbell_kernel_ptr)
-		iounmap(kfd->doorbell_kernel_ptr);
+	bitmap_free(kfd->doorbell_bitmap);
+	amdgpu_bo_free_kernel(&kfd->doorbells, NULL,
+			     (void **)&kfd->doorbell_kernel_ptr);
  }
 
  int kfd_doorbell_mmap(struct kfd_dev *dev, struct kfd_process *process,
@@ -188,22 +153,15 @@ void __iomem *kfd_get_kernel_doorbell(struct kfd_dev *kfd,
  	u32 inx;
 
  	mutex_lock(&kfd->doorbell_mutex);
-	inx = find_first_zero_bit(kfd->doorbell_available_index,
-					KFD_MAX_NUM_OF_QUEUES_PER_PROCESS);
+	inx = find_first_zero_bit(kfd->doorbell_bitmap, PAGE_SIZE / sizeof(u32));
 
-	__set_bit(inx, kfd->doorbell_available_index);
+	__set_bit(inx, kfd->doorbell_bitmap);
  	mutex_unlock(&kfd->doorbell_mutex);
 
  	if (inx >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS)
  		return NULL;
 
-	inx *= kfd->device_info.doorbell_size / sizeof(u32);
-
-	/*
-	 * Calculating the kernel doorbell offset using the first
-	 * doorbell page.
-	 */
-	*doorbell_off = kfd->doorbell_base_dw_offset + inx;
+	*doorbell_off = amdgpu_doorbell_index_on_bar(kfd->adev, kfd->doorbells, inx);
 
  	pr_debug("Get kernel queue doorbell\n"
  			"     doorbell offset   == 0x%08X\n"
@@ -217,11 +175,10 @@ void kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32 __iomem *db_addr)
  {
  	unsigned int inx;
 
-	inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr)
-		* sizeof(u32) / kfd->device_info.doorbell_size;
+	inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr);
 
  	mutex_lock(&kfd->doorbell_mutex);
-	__clear_bit(inx, kfd->doorbell_available_index);
+	__clear_bit(inx, kfd->doorbell_bitmap);
  	mutex_unlock(&kfd->doorbell_mutex);
  }
 
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +6 -0
index 94a438956868..42c215782d72 100644
@@ -349,6 +349,12 @@ struct kfd_dev {
 
  	/* HMM page migration MEMORY_DEVICE_PRIVATE mapping */
  	struct dev_pagemap pgmap;
+
+	/* Kernel doorbells for KFD device */
+	struct amdgpu_bo *doorbells;
+
+	/* bitmap for dynamic doorbell allocation from this object */
+	unsigned long *doorbell_bitmap;
  };
 
  enum kfd_mempool {
#16 lore raw

PATCH v3 9/12 drm/amdgpu: use doorbell manager for kfd process doorbells

In reply to Shashank Sharma

This patch:
- adds a doorbell object in kfd pdd structure.
- allocates doorbells for a process while creating its queue.
- frees the doorbells with pdd destroy.
- moves doorbell bitmap init function to kfd_doorbell.c

PS: This patch ensures that we don't break the existing KFD
functionality, but now KFD userspace library should also create doorbell pages as AMDGPU GEM objects using libdrm functions in userspace. The reference code for the same is available with AMDGPU Usermode queue libdrm MR. Once this is done, we will not need to create process doorbells in kernel.

V2: - Do not use doorbell wrapper API, use amdgpu_bo_create_kernel
instead (Alex).
- Do not use custom doorbell structure, instead use separate variables for bo and doorbell_bitmap (Alex)
V3:
- Do not allocate doorbell page with PDD, delay doorbell process page allocation until really needed (Felix)

---
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c      |  20 ++--
 .../drm/amd/amdkfd/kfd_device_queue_manager.c |   8 +-
 drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c     | 103 +++++++++++++-----
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h         |   9 +-
 drivers/gpu/drm/amd/amdkfd/kfd_process.c      |  40 +------
 .../amd/amdkfd/kfd_process_queue_manager.c    |  23 ++--
 6 files changed, 108 insertions(+), 95 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +11 -9
index 1b54a9aaae70..5d4f4fca793a 100644
@@ -327,10 +327,12 @@ static int kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p,
 		goto err_bind_process;
 	}
 
-	if (!pdd->doorbell_index &&
-	    kfd_alloc_process_doorbells(dev, &pdd->doorbell_index) < 0) {
-		err = -ENOMEM;
-		goto err_alloc_doorbells;
+	if (!pdd->qpd.proc_doorbells) {
+		err = kfd_alloc_process_doorbells(dev, pdd);
+		if (err) {
+			pr_debug("failed to allocate process doorbells\n");
+			goto err_bind_process;
+		}
 	}
 
 	/* Starting with GFX11, wptr BOs must be mapped to GART for MES to determine work
@@ -410,7 +412,6 @@ static int kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p,
 	if (wptr_bo)
 		amdgpu_amdkfd_free_gtt_mem(dev->adev, wptr_bo);
 err_wptr_map_gart:
-err_alloc_doorbells:
 err_bind_process:
 err_pdd:
 	mutex_unlock(&p->mutex);
@@ -2239,11 +2240,12 @@ static int criu_restore_devices(struct kfd_process *p,
 			goto exit;
 		}
 
-		if (!pdd->doorbell_index &&
-		    kfd_alloc_process_doorbells(pdd->dev, &pdd->doorbell_index) < 0) {
-			ret = -ENOMEM;
-			goto exit;
+		if (!pdd->qpd.proc_doorbells) {
+			ret = kfd_alloc_process_doorbells(dev, pdd);
+			if (ret)
+				goto exit;
 		}
+
 	}
 
 	/*
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +4 -4
index 7a95698d83f7..834f640cf807 100644
@@ -371,7 +371,7 @@ static int allocate_doorbell(struct qcm_process_device *qpd,
 			unsigned int found;
 
 			found = find_first_zero_bit(qpd->doorbell_bitmap,
-						KFD_MAX_NUM_OF_QUEUES_PER_PROCESS);
+						    KFD_MAX_NUM_OF_QUEUES_PER_PROCESS);
 			if (found >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) {
 				pr_debug("No doorbells available");
 				return -EBUSY;
@@ -381,9 +381,9 @@ static int allocate_doorbell(struct qcm_process_device *qpd,
 		}
 	}
 
-	q->properties.doorbell_off =
-		kfd_get_doorbell_dw_offset_in_bar(dev, qpd_to_pdd(qpd),
-					  q->doorbell_id);
+	q->properties.doorbell_off = amdgpu_doorbell_index_on_bar(dev->adev,
+								  qpd->proc_doorbells,
+								  q->doorbell_id);
 	return 0;
 }
 
drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c +76 -27

124 lines not on this page · raw message

drivers/gpu/drm/amd/amdkfd/kfd_priv.h +6 -3
index 42c215782d72..d0abfaccf8c1 100644
@@ -664,7 +664,10 @@ struct qcm_process_device {
 	uint64_t ib_base;
 	void *ib_kaddr;
 
-	/* doorbell resources per process per device */
+	/* doorbells for kfd process */
+	struct amdgpu_bo *proc_doorbells;
+
+	/* bitmap for dynamic doorbell allocation from the bo */
 	unsigned long *doorbell_bitmap;
 };
 
@@ -1013,9 +1016,9 @@ unsigned int kfd_get_doorbell_dw_offset_in_bar(struct kfd_dev *kfd,
 					unsigned int doorbell_id);
 phys_addr_t kfd_get_process_doorbells(struct kfd_process_device *pdd);
 int kfd_alloc_process_doorbells(struct kfd_dev *kfd,
-				unsigned int *doorbell_index);
+				 struct kfd_process_device *pdd);
 void kfd_free_process_doorbells(struct kfd_dev *kfd,
-				unsigned int doorbell_index);
+				 struct kfd_process_device *pdd);
 /* GTT Sub-Allocator */
 
 int kfd_gtt_sa_allocate(struct kfd_dev *kfd, unsigned int size,
drivers/gpu/drm/amd/amdkfd/kfd_process.c +1 -39
index 07a9eaf9b7d8..bb1281a4feef 100644
@@ -1037,10 +1037,9 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
 			free_pages((unsigned long)pdd->qpd.cwsr_kaddr,
 				get_order(KFD_CWSR_TBA_TMA_SIZE));
 
-		bitmap_free(pdd->qpd.doorbell_bitmap);
 		idr_destroy(&pdd->alloc_idr);
 
-		kfd_free_process_doorbells(pdd->dev, pdd->doorbell_index);
+		kfd_free_process_doorbells(pdd->dev, pdd);
 
 		if (pdd->dev->shared_resources.enable_mes)
 			amdgpu_amdkfd_free_gtt_mem(pdd->dev->adev,
@@ -1496,38 +1495,6 @@ static struct kfd_process *create_process(const struct task_struct *thread)
 	return ERR_PTR(err);
 }
 
-static int init_doorbell_bitmap(struct qcm_process_device *qpd,
-			struct kfd_dev *dev)
-{
-	unsigned int i;
-	int range_start = dev->shared_resources.non_cp_doorbells_start;
-	int range_end = dev->shared_resources.non_cp_doorbells_end;
-
-	if (!KFD_IS_SOC15(dev))
-		return 0;
-
-	qpd->doorbell_bitmap = bitmap_zalloc(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS,
-					     GFP_KERNEL);
-	if (!qpd->doorbell_bitmap)
-		return -ENOMEM;
-
-	/* Mask out doorbells reserved for SDMA, IH, and VCN on SOC15. */
-	pr_debug("reserved doorbell 0x%03x - 0x%03x\n", range_start, range_end);
-	pr_debug("reserved doorbell 0x%03x - 0x%03x\n",
-			range_start + KFD_QUEUE_DOORBELL_MIRROR_OFFSET,
-			range_end + KFD_QUEUE_DOORBELL_MIRROR_OFFSET);
-
-	for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS / 2; i++) {
-		if (i >= range_start && i <= range_end) {
-			__set_bit(i, qpd->doorbell_bitmap);
-			__set_bit(i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET,
-				  qpd->doorbell_bitmap);
-		}
-	}
-
-	return 0;
-}
-
 struct kfd_process_device *kfd_get_process_device_data(struct kfd_dev *dev,
 							struct kfd_process *p)
 {
@@ -1552,11 +1519,6 @@ struct kfd_process_device *kfd_create_process_device_data(struct kfd_dev *dev,
 	if (!pdd)
 		return NULL;
 
-	if (init_doorbell_bitmap(&pdd->qpd, dev)) {
-		pr_err("Failed to init doorbell for process\n");
-		goto err_free_pdd;
-	}
-
 	pdd->dev = dev;
 	INIT_LIST_HEAD(&pdd->qpd.queues_list);
 	INIT_LIST_HEAD(&pdd->qpd.priv_queue_list);
drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c +10 -13
index 4236539d9f93..5eff74015d51 100644
@@ -344,17 +344,20 @@ int pqm_create_queue(struct process_queue_manager *pqm,
 		goto err_create_queue;
 	}
 
-	if (q && p_doorbell_offset_in_process)
+	if (q && p_doorbell_offset_in_process) {
 		/* Return the doorbell offset within the doorbell page
 		 * to the caller so it can be passed up to user mode
 		 * (in bytes).
-		 * There are always 1024 doorbells per process, so in case
-		 * of 8-byte doorbells, there are two doorbell pages per
-		 * process.
+		 * relative doorbell index = Absolute doorbell index -
+		 * absolute index of first doorbell in the page.
 		 */
-		*p_doorbell_offset_in_process =
-			(q->properties.doorbell_off * sizeof(uint32_t)) &
-			(kfd_doorbell_process_slice(dev) - 1);
+		uint32_t first_db_index = amdgpu_doorbell_index_on_bar(pdd->dev->adev,
+								       pdd->qpd.proc_doorbells,
+								       0);
+
+		*p_doorbell_offset_in_process = (q->properties.doorbell_off
+						- first_db_index) * sizeof(uint32_t);
+	}
 
 	pr_debug("PQM After DQM create queue\n");
 
@@ -858,12 +861,6 @@ int kfd_criu_restore_queue(struct kfd_process *p,
 		goto exit;
 	}
 
-	if (!pdd->doorbell_index &&
-	    kfd_alloc_process_doorbells(pdd->dev, &pdd->doorbell_index) < 0) {
-		ret = -ENOMEM;
-		goto exit;
-	}
-
 	/* data stored in this order: mqd, ctl_stack */
 	mqd = q_extra_data;
 	ctl_stack = mqd + q_data->mqd_size;
Signature · 1 line
2.40.1
maintainer Reviewed-by on v3 9/12
#17 lore raw

Re: [PATCH v3 09/12] drm/amdgpu: use doorbell manager for kfd process doorbells

Quoted 22 lines · Shashank Sharma

This patch:
- adds a doorbell object in kfd pdd structure.
- allocates doorbells for a process while creating its queue.
- frees the doorbells with pdd destroy.
- moves doorbell bitmap init function to kfd_doorbell.c

PS: This patch ensures that we don't break the existing KFD
functionality, but now KFD userspace library should also create doorbell pages as AMDGPU GEM objects using libdrm functions in userspace. The reference code for the same is available with AMDGPU Usermode queue libdrm MR. Once this is done, we will not need to create process doorbells in kernel.

V2: - Do not use doorbell wrapper API, use amdgpu_bo_create_kernel
instead (Alex).
- Do not use custom doorbell structure, instead use separate variables for bo and doorbell_bitmap (Alex)
V3:
- Do not allocate doorbell page with PDD, delay doorbell process page allocation until really needed (Felix)

Quoted 5 lines
Quoted 350 lines
---
  drivers/gpu/drm/amd/amdkfd/kfd_chardev.c      |  20 ++--
  .../drm/amd/amdkfd/kfd_device_queue_manager.c |   8 +-
  drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c     | 103 +++++++++++++-----
  drivers/gpu/drm/amd/amdkfd/kfd_priv.h         |   9 +-
  drivers/gpu/drm/amd/amdkfd/kfd_process.c      |  40 +------
  .../amd/amdkfd/kfd_process_queue_manager.c    |  23 ++--
  6 files changed, 108 insertions(+), 95 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +11 -9
index 1b54a9aaae70..5d4f4fca793a 100644
@@ -327,10 +327,12 @@ static int kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p,
  		goto err_bind_process;
  	}
 
-	if (!pdd->doorbell_index &&
-	    kfd_alloc_process_doorbells(dev, &pdd->doorbell_index) < 0) {
-		err = -ENOMEM;
-		goto err_alloc_doorbells;
+	if (!pdd->qpd.proc_doorbells) {
+		err = kfd_alloc_process_doorbells(dev, pdd);
+		if (err) {
+			pr_debug("failed to allocate process doorbells\n");
+			goto err_bind_process;
+		}
  	}
 
  	/* Starting with GFX11, wptr BOs must be mapped to GART for MES to determine work
@@ -410,7 +412,6 @@ static int kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p,
  	if (wptr_bo)
  		amdgpu_amdkfd_free_gtt_mem(dev->adev, wptr_bo);
  err_wptr_map_gart:
-err_alloc_doorbells:
  err_bind_process:
  err_pdd:
  	mutex_unlock(&p->mutex);
@@ -2239,11 +2240,12 @@ static int criu_restore_devices(struct kfd_process *p,
  			goto exit;
  		}
 
-		if (!pdd->doorbell_index &&
-		    kfd_alloc_process_doorbells(pdd->dev, &pdd->doorbell_index) < 0) {
-			ret = -ENOMEM;
-			goto exit;
+		if (!pdd->qpd.proc_doorbells) {
+			ret = kfd_alloc_process_doorbells(dev, pdd);
+			if (ret)
+				goto exit;
  		}
+
  	}
 
  	/*
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +4 -4
index 7a95698d83f7..834f640cf807 100644
@@ -371,7 +371,7 @@ static int allocate_doorbell(struct qcm_process_device *qpd,
  			unsigned int found;
 
  			found = find_first_zero_bit(qpd->doorbell_bitmap,
-						KFD_MAX_NUM_OF_QUEUES_PER_PROCESS);
+						    KFD_MAX_NUM_OF_QUEUES_PER_PROCESS);
  			if (found >= KFD_MAX_NUM_OF_QUEUES_PER_PROCESS) {
  				pr_debug("No doorbells available");
  				return -EBUSY;
@@ -381,9 +381,9 @@ static int allocate_doorbell(struct qcm_process_device *qpd,
  		}
  	}
 
-	q->properties.doorbell_off =
-		kfd_get_doorbell_dw_offset_in_bar(dev, qpd_to_pdd(qpd),
-					  q->doorbell_id);
+	q->properties.doorbell_off = amdgpu_doorbell_index_on_bar(dev->adev,
+								  qpd->proc_doorbells,
+								  q->doorbell_id);
  	return 0;
  }
 
drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c +76 -27
index f7d45057ed32..c9ca21e1a99a 100644
@@ -232,48 +232,97 @@ uint64_t kfd_get_number_elems(struct kfd_dev *kfd)
 
  }
 
+static int init_doorbell_bitmap(struct qcm_process_device *qpd,
+				struct kfd_dev *dev)
+{
+	unsigned int i;
+	int range_start = dev->shared_resources.non_cp_doorbells_start;
+	int range_end = dev->shared_resources.non_cp_doorbells_end;
+
+	if (!KFD_IS_SOC15(dev))
+		return 0;
+
+	/* Mask out doorbells reserved for SDMA, IH, and VCN on SOC15. */
+	pr_debug("reserved doorbell 0x%03x - 0x%03x\n", range_start, range_end);
+	pr_debug("reserved doorbell 0x%03x - 0x%03x\n",
+			range_start + KFD_QUEUE_DOORBELL_MIRROR_OFFSET,
+			range_end + KFD_QUEUE_DOORBELL_MIRROR_OFFSET);
+
+	for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS / 2; i++) {
+		if (i >= range_start && i <= range_end) {
+			__set_bit(i, qpd->doorbell_bitmap);
+			__set_bit(i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET,
+				  qpd->doorbell_bitmap);
+		}
+	}
+
+	return 0;
+}
+
  phys_addr_t kfd_get_process_doorbells(struct kfd_process_device *pdd)
  {
-	if (!pdd->doorbell_index) {
-		int r = kfd_alloc_process_doorbells(pdd->dev,
-						    &pdd->doorbell_index);
-		if (r < 0)
+	struct amdgpu_device *adev = pdd->dev->adev;
+	uint32_t first_db_index;
+
+	if (!pdd->qpd.proc_doorbells) {
+		if (kfd_alloc_process_doorbells(pdd->dev, pdd))
+			/* phys_addr_t 0 is error */
  			return 0;
  	}
 
-	return pdd->dev->doorbell_base +
-		pdd->doorbell_index * kfd_doorbell_process_slice(pdd->dev);
+	first_db_index = amdgpu_doorbell_index_on_bar(adev, pdd->qpd.proc_doorbells, 0);
+	return adev->doorbell.base + first_db_index * sizeof(uint32_t);
  }
 
-int kfd_alloc_process_doorbells(struct kfd_dev *kfd, unsigned int *doorbell_index)
+int kfd_alloc_process_doorbells(struct kfd_dev *kfd, struct kfd_process_device *pdd)
  {
-	int r = 0;
+	int r;
+	struct qcm_process_device *qpd = &pdd->qpd;
 
-	if (!kfd->shared_resources.enable_mes)
-		r = ida_simple_get(&kfd->doorbell_ida, 1,
-				   kfd->max_doorbell_slices, GFP_KERNEL);
-	else
-		r = amdgpu_mes_alloc_process_doorbells(
-				(struct amdgpu_device *)kfd->adev,
-				doorbell_index);
+	/* Allocate bitmap for dynamic doorbell allocation */
+	qpd->doorbell_bitmap = bitmap_zalloc(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS,
+					     GFP_KERNEL);
+	if (!qpd->doorbell_bitmap) {
+		DRM_ERROR("Failed to allocate process doorbell bitmap\n");
+		return -ENOMEM;
+	}
 
-	if (r > 0)
-		*doorbell_index = r;
+	r = init_doorbell_bitmap(&pdd->qpd, kfd);
+	if (r) {
+		DRM_ERROR("Failed to initialize process doorbells\n");
+		r = -ENOMEM;
+		goto err;
+	}
+
+	/* Allocate doorbells for this process */
+	r = amdgpu_bo_create_kernel(kfd->adev,
+				    kfd_doorbell_process_slice(kfd),
+				    PAGE_SIZE,
+				    AMDGPU_GEM_DOMAIN_DOORBELL,
+				    &qpd->proc_doorbells,
+				    NULL,
+				    NULL);
+	if (r) {
+		DRM_ERROR("Failed to allocate process doorbells\n");
+		goto err;
+	}
 
-	if (r < 0)
-		pr_err("Failed to allocate process doorbells\n");
+	return 0;
 
+err:
+	bitmap_free(qpd->doorbell_bitmap);
+	qpd->doorbell_bitmap = NULL;
  	return r;
  }
 
-void kfd_free_process_doorbells(struct kfd_dev *kfd, unsigned int doorbell_index)
+void kfd_free_process_doorbells(struct kfd_dev *kfd, struct kfd_process_device *pdd)
  {
-	if (doorbell_index) {
-		if (!kfd->shared_resources.enable_mes)
-			ida_simple_remove(&kfd->doorbell_ida, doorbell_index);
-		else
-			amdgpu_mes_free_process_doorbells(
-					(struct amdgpu_device *)kfd->adev,
-					doorbell_index);
+	struct qcm_process_device *qpd = &pdd->qpd;
+
+	if (qpd->doorbell_bitmap) {
+		bitmap_free(qpd->doorbell_bitmap);
+		qpd->doorbell_bitmap = NULL;
  	}
+
+	amdgpu_bo_free_kernel(&qpd->proc_doorbells, NULL, NULL);
  }
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +6 -3
index 42c215782d72..d0abfaccf8c1 100644
@@ -664,7 +664,10 @@ struct qcm_process_device {
  	uint64_t ib_base;
  	void *ib_kaddr;
 
-	/* doorbell resources per process per device */
+	/* doorbells for kfd process */
+	struct amdgpu_bo *proc_doorbells;
+
+	/* bitmap for dynamic doorbell allocation from the bo */
  	unsigned long *doorbell_bitmap;
  };
 
@@ -1013,9 +1016,9 @@ unsigned int kfd_get_doorbell_dw_offset_in_bar(struct kfd_dev *kfd,
  					unsigned int doorbell_id);
  phys_addr_t kfd_get_process_doorbells(struct kfd_process_device *pdd);
  int kfd_alloc_process_doorbells(struct kfd_dev *kfd,
-				unsigned int *doorbell_index);
+				 struct kfd_process_device *pdd);
  void kfd_free_process_doorbells(struct kfd_dev *kfd,
-				unsigned int doorbell_index);
+				 struct kfd_process_device *pdd);
  /* GTT Sub-Allocator */
 
  int kfd_gtt_sa_allocate(struct kfd_dev *kfd, unsigned int size,
drivers/gpu/drm/amd/amdkfd/kfd_process.c +1 -39
index 07a9eaf9b7d8..bb1281a4feef 100644
@@ -1037,10 +1037,9 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
  			free_pages((unsigned long)pdd->qpd.cwsr_kaddr,
  				get_order(KFD_CWSR_TBA_TMA_SIZE));
 
-		bitmap_free(pdd->qpd.doorbell_bitmap);
  		idr_destroy(&pdd->alloc_idr);
 
-		kfd_free_process_doorbells(pdd->dev, pdd->doorbell_index);
+		kfd_free_process_doorbells(pdd->dev, pdd);
 
  		if (pdd->dev->shared_resources.enable_mes)
  			amdgpu_amdkfd_free_gtt_mem(pdd->dev->adev,
@@ -1496,38 +1495,6 @@ static struct kfd_process *create_process(const struct task_struct *thread)
  	return ERR_PTR(err);
  }
 
-static int init_doorbell_bitmap(struct qcm_process_device *qpd,
-			struct kfd_dev *dev)
-{
-	unsigned int i;
-	int range_start = dev->shared_resources.non_cp_doorbells_start;
-	int range_end = dev->shared_resources.non_cp_doorbells_end;
-
-	if (!KFD_IS_SOC15(dev))
-		return 0;
-
-	qpd->doorbell_bitmap = bitmap_zalloc(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS,
-					     GFP_KERNEL);
-	if (!qpd->doorbell_bitmap)
-		return -ENOMEM;
-
-	/* Mask out doorbells reserved for SDMA, IH, and VCN on SOC15. */
-	pr_debug("reserved doorbell 0x%03x - 0x%03x\n", range_start, range_end);
-	pr_debug("reserved doorbell 0x%03x - 0x%03x\n",
-			range_start + KFD_QUEUE_DOORBELL_MIRROR_OFFSET,
-			range_end + KFD_QUEUE_DOORBELL_MIRROR_OFFSET);
-
-	for (i = 0; i < KFD_MAX_NUM_OF_QUEUES_PER_PROCESS / 2; i++) {
-		if (i >= range_start && i <= range_end) {
-			__set_bit(i, qpd->doorbell_bitmap);
-			__set_bit(i + KFD_QUEUE_DOORBELL_MIRROR_OFFSET,
-				  qpd->doorbell_bitmap);
-		}
-	}
-
-	return 0;
-}
-
  struct kfd_process_device *kfd_get_process_device_data(struct kfd_dev *dev,
  							struct kfd_process *p)
  {
@@ -1552,11 +1519,6 @@ struct kfd_process_device *kfd_create_process_device_data(struct kfd_dev *dev,
  	if (!pdd)
  		return NULL;
 
-	if (init_doorbell_bitmap(&pdd->qpd, dev)) {
-		pr_err("Failed to init doorbell for process\n");
-		goto err_free_pdd;
-	}
-
  	pdd->dev = dev;
  	INIT_LIST_HEAD(&pdd->qpd.queues_list);
  	INIT_LIST_HEAD(&pdd->qpd.priv_queue_list);
drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c +10 -13
index 4236539d9f93..5eff74015d51 100644
@@ -344,17 +344,20 @@ int pqm_create_queue(struct process_queue_manager *pqm,
  		goto err_create_queue;
  	}
 
-	if (q && p_doorbell_offset_in_process)
+	if (q && p_doorbell_offset_in_process) {
  		/* Return the doorbell offset within the doorbell page
  		 * to the caller so it can be passed up to user mode
  		 * (in bytes).
-		 * There are always 1024 doorbells per process, so in case
-		 * of 8-byte doorbells, there are two doorbell pages per
-		 * process.
+		 * relative doorbell index = Absolute doorbell index -
+		 * absolute index of first doorbell in the page.
  		 */
-		*p_doorbell_offset_in_process =
-			(q->properties.doorbell_off * sizeof(uint32_t)) &
-			(kfd_doorbell_process_slice(dev) - 1);
+		uint32_t first_db_index = amdgpu_doorbell_index_on_bar(pdd->dev->adev,
+								       pdd->qpd.proc_doorbells,
+								       0);
+
+		*p_doorbell_offset_in_process = (q->properties.doorbell_off
+						- first_db_index) * sizeof(uint32_t);
+	}
 
  	pr_debug("PQM After DQM create queue\n");
 
@@ -858,12 +861,6 @@ int kfd_criu_restore_queue(struct kfd_process *p,
  		goto exit;
  	}
 
-	if (!pdd->doorbell_index &&
-	    kfd_alloc_process_doorbells(pdd->dev, &pdd->doorbell_index) < 0) {
-		ret = -ENOMEM;
-		goto exit;
-	}
-
  	/* data stored in this order: mqd, ctl_stack */
  	mqd = q_extra_data;
  	ctl_stack = mqd + q_data->mqd_size;
#18 lore raw

PATCH v3 10/12 drm/amdgpu: remove unused functions and variables

In reply to Shashank Sharma

This patch removes some variables and functions from KFD doorbell handling code, which are no more required since doorbell manager is handling doorbell calculations.

Cc Alex Deucher <[email protected]>
Cc Christian Koenig <[email protected]>
Cc Felix Kuehling <[email protected]>
---
 drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c | 32 -----------------------
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h     | 12 ---------
 2 files changed, 44 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c +0 -32
index c9ca21e1a99a..5423d717ccbb 100644
@@ -200,38 +200,6 @@ void write_kernel_doorbell64(void __iomem *db, u64 value)
 	}
 }
 
-unsigned int kfd_get_doorbell_dw_offset_in_bar(struct kfd_dev *kfd,
-					struct kfd_process_device *pdd,
-					unsigned int doorbell_id)
-{
-	/*
-	 * doorbell_base_dw_offset accounts for doorbells taken by KGD.
-	 * index * kfd_doorbell_process_slice/sizeof(u32) adjusts to
-	 * the process's doorbells. The offset returned is in dword
-	 * units regardless of the ASIC-dependent doorbell size.
-	 */
-	if (!kfd->shared_resources.enable_mes)
-		return kfd->doorbell_base_dw_offset +
-			pdd->doorbell_index
-			* kfd_doorbell_process_slice(kfd) / sizeof(u32) +
-			doorbell_id *
-			kfd->device_info.doorbell_size / sizeof(u32);
-	else
-		return amdgpu_mes_get_doorbell_dw_offset_in_bar(
-				(struct amdgpu_device *)kfd->adev,
-				pdd->doorbell_index, doorbell_id);
-}
-
-uint64_t kfd_get_number_elems(struct kfd_dev *kfd)
-{
-	uint64_t num_of_elems = (kfd->shared_resources.doorbell_aperture_size -
-				kfd->shared_resources.doorbell_start_offset) /
-					kfd_doorbell_process_slice(kfd) + 1;
-
-	return num_of_elems;
-
-}
-
 static int init_doorbell_bitmap(struct qcm_process_device *qpd,
 				struct kfd_dev *dev)
 {
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +0 -12
index d0abfaccf8c1..edbaba845d06 100644
@@ -260,15 +260,6 @@ struct kfd_dev {
 
 	unsigned int id;		/* topology stub index */
 
-	phys_addr_t doorbell_base;	/* Start of actual doorbells used by
-					 * KFD. It is aligned for mapping
-					 * into user mode
-					 */
-	size_t doorbell_base_dw_offset;	/* Offset from the start of the PCI
-					 * doorbell BAR to the first KFD
-					 * doorbell in dwords. GFX reserves
-					 * the segment before this offset.
-					 */
 	u32 __iomem *doorbell_kernel_ptr; /* This is a pointer for a doorbells
 					   * page used by kernel queue
 					   */
@@ -279,8 +270,6 @@ struct kfd_dev {
 
 	const struct kfd2kgd_calls *kfd2kgd;
 	struct mutex doorbell_mutex;
-	DECLARE_BITMAP(doorbell_available_index,
-			KFD_MAX_NUM_OF_QUEUES_PER_PROCESS);
 
 	void *gtt_mem;
 	uint64_t gtt_start_gpu_addr;
@@ -757,7 +746,6 @@ struct kfd_process_device {
 	struct attribute attr_evict;
 
 	struct kobject *kobj_stats;
-	unsigned int doorbell_index;
 
 	/*
 	 * @cu_occupancy: Reports occupancy of Compute Units (CU) of a process
Signature · 1 line
2.40.1
#19 lore raw

PATCH v3 11/12 drm/amdgpu: use doorbell mgr for MES kernel doorbells

In reply to Shashank Sharma

This patch:
- Removes the existing doorbell management code, and its variables from the doorbell_init function, it will be done in doorbell manager now.
- uses the doorbell page created for MES kernel level needs (doorbells for MES self tests)
- current MES code was allocating MES doorbells in MES process context, but those were getting written using kernel doorbell calls. This patch instead allocates a MES kernel doorbell for this (in add_hw_queue).

V2: Create an extra page of doorbells for MES during kernel doorbell
creation (Alex)

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c | 94 ++++++++++---------------
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h |  3 +
 2 files changed, 40 insertions(+), 57 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +37 -57
index f0f00466b59f..01c3d27512d9 100644
@@ -67,91 +67,70 @@ unsigned int amdgpu_mes_get_doorbell_dw_offset_in_bar(
 		doorbell_id * 2);
 }
 
-static int amdgpu_mes_queue_doorbell_get(struct amdgpu_device *adev,
+static int amdgpu_mes_kernel_doorbell_get(struct amdgpu_device *adev,
 					 struct amdgpu_mes_process *process,
 					 int ip_type, uint64_t *doorbell_index)
 {
 	unsigned int offset, found;
+	struct amdgpu_mes *mes = &adev->mes;
 
-	if (ip_type == AMDGPU_RING_TYPE_SDMA) {
+	if (ip_type == AMDGPU_RING_TYPE_SDMA)
 		offset = adev->doorbell_index.sdma_engine[0];
-		found = find_next_zero_bit(process->doorbell_bitmap,
-					   AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS,
-					   offset);
-	} else {
-		found = find_first_zero_bit(process->doorbell_bitmap,
-					    AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS);
-	}
+	else
+		offset = 0;
 
-	if (found >= AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS) {
+	found = find_next_zero_bit(mes->doorbell_bitmap, mes->num_mes_dbs, offset);
+	if (found >= mes->num_mes_dbs) {
 		DRM_WARN("No doorbell available\n");
 		return -ENOSPC;
 	}
 
-	set_bit(found, process->doorbell_bitmap);
-
-	*doorbell_index = amdgpu_mes_get_doorbell_dw_offset_in_bar(adev,
-				process->doorbell_index, found);
+	set_bit(found, mes->doorbell_bitmap);
 
+	/* Get the absolute doorbell index on BAR */
+	*doorbell_index = mes->db_start_dw_offset + found * 2;
 	return 0;
 }
 
-static void amdgpu_mes_queue_doorbell_free(struct amdgpu_device *adev,
+static void amdgpu_mes_kernel_doorbell_free(struct amdgpu_device *adev,
 					   struct amdgpu_mes_process *process,
 					   uint32_t doorbell_index)
 {
-	unsigned int old, doorbell_id;
-
-	doorbell_id = doorbell_index -
-		(process->doorbell_index *
-		 amdgpu_mes_doorbell_process_slice(adev)) / sizeof(u32);
-	doorbell_id /= 2;
+	unsigned int old, rel_index;
+	struct amdgpu_mes *mes = &adev->mes;
 
-	old = test_and_clear_bit(doorbell_id, process->doorbell_bitmap);
+	/* Find the relative index of the doorbell in this object */
+	rel_index = (doorbell_index - mes->db_start_dw_offset) / 2;
+	old = test_and_clear_bit(rel_index, mes->doorbell_bitmap);
 	WARN_ON(!old);
 }
 
 static int amdgpu_mes_doorbell_init(struct amdgpu_device *adev)
 {
-	size_t doorbell_start_offset;
-	size_t doorbell_aperture_size;
-	size_t doorbell_process_limit;
-	size_t aggregated_doorbell_start;
 	int i;
+	struct amdgpu_mes *mes = &adev->mes;
 
-	aggregated_doorbell_start = (adev->doorbell_index.max_assignment + 1) * sizeof(u32);
-	aggregated_doorbell_start =
-		roundup(aggregated_doorbell_start, PAGE_SIZE);
-
-	doorbell_start_offset = aggregated_doorbell_start + PAGE_SIZE;
-	doorbell_start_offset =
-		roundup(doorbell_start_offset,
-			amdgpu_mes_doorbell_process_slice(adev));
-
-	doorbell_aperture_size = adev->doorbell.size;
-	doorbell_aperture_size =
-			rounddown(doorbell_aperture_size,
-				  amdgpu_mes_doorbell_process_slice(adev));
-
-	if (doorbell_aperture_size > doorbell_start_offset)
-		doorbell_process_limit =
-			(doorbell_aperture_size - doorbell_start_offset) /
-			amdgpu_mes_doorbell_process_slice(adev);
-	else
-		return -ENOSPC;
-
-	adev->mes.doorbell_id_offset = doorbell_start_offset / sizeof(u32);
-	adev->mes.max_doorbell_slices = doorbell_process_limit;
+	/* Bitmap for dynamic allocation of kernel doorbells */
+	mes->doorbell_bitmap = bitmap_zalloc(PAGE_SIZE / sizeof(u32), GFP_KERNEL);
+	if (!mes->doorbell_bitmap) {
+		DRM_ERROR("Failed to allocate MES doorbell bitmap\n");
+		return -ENOMEM;
+	}
 
-	/* allocate Qword range for aggregated doorbell */
-	for (i = 0; i < AMDGPU_MES_PRIORITY_NUM_LEVELS; i++)
-		adev->mes.aggregated_doorbells[i] =
-			aggregated_doorbell_start / sizeof(u32) + i * 2;
+	mes->num_mes_dbs = PAGE_SIZE / AMDGPU_ONE_DOORBELL_SIZE;
+	for (i = 0; i < AMDGPU_MES_PRIORITY_NUM_LEVELS; i++) {
+		adev->mes.aggregated_doorbells[i] = mes->db_start_dw_offset + i * 2;
+		set_bit(i, mes->doorbell_bitmap);
+	}
 
-	DRM_INFO("max_doorbell_slices=%zu\n", doorbell_process_limit);
 	return 0;
 }
 
+static void amdgpu_mes_doorbell_free(struct amdgpu_device *adev)
+{
+	bitmap_free(adev->mes.doorbell_bitmap);
+}
+
 int amdgpu_mes_init(struct amdgpu_device *adev)
 {
 	int i, r;
@@ -250,6 +229,7 @@ void amdgpu_mes_fini(struct amdgpu_device *adev)
 	amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs);
 	amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs);
 	amdgpu_device_wb_free(adev, adev->mes.read_val_offs);
+	amdgpu_mes_doorbell_free(adev);
 
 	idr_destroy(&adev->mes.pasid_idr);
 	idr_destroy(&adev->mes.gang_id_idr);
@@ -679,7 +659,7 @@ int amdgpu_mes_add_hw_queue(struct amdgpu_device *adev, int gang_id,
 	*queue_id = queue->queue_id = r;
 
 	/* allocate a doorbell index for the queue */
-	r = amdgpu_mes_queue_doorbell_get(adev, gang->process,
+	r = amdgpu_mes_kernel_doorbell_get(adev, gang->process,
 					  qprops->queue_type,
 					  &qprops->doorbell_off);
 	if (r)
@@ -737,7 +717,7 @@ int amdgpu_mes_add_hw_queue(struct amdgpu_device *adev, int gang_id,
 	return 0;
 
 clean_up_doorbell:
-	amdgpu_mes_queue_doorbell_free(adev, gang->process,
+	amdgpu_mes_kernel_doorbell_free(adev, gang->process,
 				       qprops->doorbell_off);
 clean_up_queue_id:
 	spin_lock_irqsave(&adev->mes.queue_id_lock, flags);
@@ -792,7 +772,7 @@ int amdgpu_mes_remove_hw_queue(struct amdgpu_device *adev, int queue_id)
 			  queue_id);
 
 	list_del(&queue->list);
-	amdgpu_mes_queue_doorbell_free(adev, gang->process,
+	amdgpu_mes_kernel_doorbell_free(adev, gang->process,
 				       queue->doorbell_off);
 	amdgpu_mes_unlock(&adev->mes);
 
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +3 -0
index a403418d5eac..2c604e04fd5c 100644
@@ -27,6 +27,7 @@
 #include "amdgpu_irq.h"
 #include "kgd_kfd_interface.h"
 #include "amdgpu_gfx.h"
+#include "amdgpu_doorbell.h"
 #include <linux/sched/mm.h>
 
 #define AMDGPU_MES_MAX_COMPUTE_PIPES        8
@@ -130,6 +131,8 @@ struct amdgpu_mes {
 
 	/* MES doorbells */
 	uint32_t			db_start_dw_offset;
+	uint32_t			num_mes_dbs;
+	unsigned long			*doorbell_bitmap;
 
 	/* ip specific functions */
 	const struct amdgpu_mes_funcs   *funcs;
Signature · 1 line
2.40.1
maintainer
#20 lore raw

Re: [PATCH v3 11/12] drm/amdgpu: use doorbell mgr for MES kernel doorbells

Am 20.06.23 um 19:16 schrieb Shashank Sharma:

Quoted 10 lines

This patch:
- Removes the existing doorbell management code, and its variables from the doorbell_init function, it will be done in doorbell manager now.
- uses the doorbell page created for MES kernel level needs (doorbells for MES self tests)
- current MES code was allocating MES doorbells in MES process context, but those were getting written using kernel doorbell calls. This patch instead allocates a MES kernel doorbell for this (in add_hw_queue).

Quoted 2 lines

V2: Create an extra page of doorbells for MES during kernel doorbell
creation (Alex)

As discussed on the other patch, this should probably be merged with patch #6 or otherwise we break the MES test in between.

Apart from that this patch set looks good to me now and is most likely ready to land.

Christian.

Quoted 195 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c | 94 ++++++++++---------------
  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h |  3 +
  2 files changed, 40 insertions(+), 57 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +37 -57
index f0f00466b59f..01c3d27512d9 100644
@@ -67,91 +67,70 @@ unsigned int amdgpu_mes_get_doorbell_dw_offset_in_bar(
  		doorbell_id * 2);
  }
 
-static int amdgpu_mes_queue_doorbell_get(struct amdgpu_device *adev,
+static int amdgpu_mes_kernel_doorbell_get(struct amdgpu_device *adev,
  					 struct amdgpu_mes_process *process,
  					 int ip_type, uint64_t *doorbell_index)
  {
  	unsigned int offset, found;
+	struct amdgpu_mes *mes = &adev->mes;
 
-	if (ip_type == AMDGPU_RING_TYPE_SDMA) {
+	if (ip_type == AMDGPU_RING_TYPE_SDMA)
  		offset = adev->doorbell_index.sdma_engine[0];
-		found = find_next_zero_bit(process->doorbell_bitmap,
-					   AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS,
-					   offset);
-	} else {
-		found = find_first_zero_bit(process->doorbell_bitmap,
-					    AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS);
-	}
+	else
+		offset = 0;
 
-	if (found >= AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS) {
+	found = find_next_zero_bit(mes->doorbell_bitmap, mes->num_mes_dbs, offset);
+	if (found >= mes->num_mes_dbs) {
  		DRM_WARN("No doorbell available\n");
  		return -ENOSPC;
  	}
 
-	set_bit(found, process->doorbell_bitmap);
-
-	*doorbell_index = amdgpu_mes_get_doorbell_dw_offset_in_bar(adev,
-				process->doorbell_index, found);
+	set_bit(found, mes->doorbell_bitmap);
 
+	/* Get the absolute doorbell index on BAR */
+	*doorbell_index = mes->db_start_dw_offset + found * 2;
  	return 0;
  }
 
-static void amdgpu_mes_queue_doorbell_free(struct amdgpu_device *adev,
+static void amdgpu_mes_kernel_doorbell_free(struct amdgpu_device *adev,
  					   struct amdgpu_mes_process *process,
  					   uint32_t doorbell_index)
  {
-	unsigned int old, doorbell_id;
-
-	doorbell_id = doorbell_index -
-		(process->doorbell_index *
-		 amdgpu_mes_doorbell_process_slice(adev)) / sizeof(u32);
-	doorbell_id /= 2;
+	unsigned int old, rel_index;
+	struct amdgpu_mes *mes = &adev->mes;
 
-	old = test_and_clear_bit(doorbell_id, process->doorbell_bitmap);
+	/* Find the relative index of the doorbell in this object */
+	rel_index = (doorbell_index - mes->db_start_dw_offset) / 2;
+	old = test_and_clear_bit(rel_index, mes->doorbell_bitmap);
  	WARN_ON(!old);
  }
 
  static int amdgpu_mes_doorbell_init(struct amdgpu_device *adev)
  {
-	size_t doorbell_start_offset;
-	size_t doorbell_aperture_size;
-	size_t doorbell_process_limit;
-	size_t aggregated_doorbell_start;
  	int i;
+	struct amdgpu_mes *mes = &adev->mes;
 
-	aggregated_doorbell_start = (adev->doorbell_index.max_assignment + 1) * sizeof(u32);
-	aggregated_doorbell_start =
-		roundup(aggregated_doorbell_start, PAGE_SIZE);
-
-	doorbell_start_offset = aggregated_doorbell_start + PAGE_SIZE;
-	doorbell_start_offset =
-		roundup(doorbell_start_offset,
-			amdgpu_mes_doorbell_process_slice(adev));
-
-	doorbell_aperture_size = adev->doorbell.size;
-	doorbell_aperture_size =
-			rounddown(doorbell_aperture_size,
-				  amdgpu_mes_doorbell_process_slice(adev));
-
-	if (doorbell_aperture_size > doorbell_start_offset)
-		doorbell_process_limit =
-			(doorbell_aperture_size - doorbell_start_offset) /
-			amdgpu_mes_doorbell_process_slice(adev);
-	else
-		return -ENOSPC;
-
-	adev->mes.doorbell_id_offset = doorbell_start_offset / sizeof(u32);
-	adev->mes.max_doorbell_slices = doorbell_process_limit;
+	/* Bitmap for dynamic allocation of kernel doorbells */
+	mes->doorbell_bitmap = bitmap_zalloc(PAGE_SIZE / sizeof(u32), GFP_KERNEL);
+	if (!mes->doorbell_bitmap) {
+		DRM_ERROR("Failed to allocate MES doorbell bitmap\n");
+		return -ENOMEM;
+	}
 
-	/* allocate Qword range for aggregated doorbell */
-	for (i = 0; i < AMDGPU_MES_PRIORITY_NUM_LEVELS; i++)
-		adev->mes.aggregated_doorbells[i] =
-			aggregated_doorbell_start / sizeof(u32) + i * 2;
+	mes->num_mes_dbs = PAGE_SIZE / AMDGPU_ONE_DOORBELL_SIZE;
+	for (i = 0; i < AMDGPU_MES_PRIORITY_NUM_LEVELS; i++) {
+		adev->mes.aggregated_doorbells[i] = mes->db_start_dw_offset + i * 2;
+		set_bit(i, mes->doorbell_bitmap);
+	}
 
-	DRM_INFO("max_doorbell_slices=%zu\n", doorbell_process_limit);
  	return 0;
  }
 
+static void amdgpu_mes_doorbell_free(struct amdgpu_device *adev)
+{
+	bitmap_free(adev->mes.doorbell_bitmap);
+}
+
  int amdgpu_mes_init(struct amdgpu_device *adev)
  {
  	int i, r;
@@ -250,6 +229,7 @@ void amdgpu_mes_fini(struct amdgpu_device *adev)
  	amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs);
  	amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs);
  	amdgpu_device_wb_free(adev, adev->mes.read_val_offs);
+	amdgpu_mes_doorbell_free(adev);
 
  	idr_destroy(&adev->mes.pasid_idr);
  	idr_destroy(&adev->mes.gang_id_idr);
@@ -679,7 +659,7 @@ int amdgpu_mes_add_hw_queue(struct amdgpu_device *adev, int gang_id,
  	*queue_id = queue->queue_id = r;
 
  	/* allocate a doorbell index for the queue */
-	r = amdgpu_mes_queue_doorbell_get(adev, gang->process,
+	r = amdgpu_mes_kernel_doorbell_get(adev, gang->process,
  					  qprops->queue_type,
  					  &qprops->doorbell_off);
  	if (r)
@@ -737,7 +717,7 @@ int amdgpu_mes_add_hw_queue(struct amdgpu_device *adev, int gang_id,
  	return 0;
 
  clean_up_doorbell:
-	amdgpu_mes_queue_doorbell_free(adev, gang->process,
+	amdgpu_mes_kernel_doorbell_free(adev, gang->process,
  				       qprops->doorbell_off);
  clean_up_queue_id:
  	spin_lock_irqsave(&adev->mes.queue_id_lock, flags);
@@ -792,7 +772,7 @@ int amdgpu_mes_remove_hw_queue(struct amdgpu_device *adev, int queue_id)
  			  queue_id);
 
  	list_del(&queue->list);
-	amdgpu_mes_queue_doorbell_free(adev, gang->process,
+	amdgpu_mes_kernel_doorbell_free(adev, gang->process,
  				       queue->doorbell_off);
  	amdgpu_mes_unlock(&adev->mes);
 
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +3 -0
index a403418d5eac..2c604e04fd5c 100644
@@ -27,6 +27,7 @@
  #include "amdgpu_irq.h"
  #include "kgd_kfd_interface.h"
  #include "amdgpu_gfx.h"
+#include "amdgpu_doorbell.h"
  #include <linux/sched/mm.h>
 
  #define AMDGPU_MES_MAX_COMPUTE_PIPES        8
@@ -130,6 +131,8 @@ struct amdgpu_mes {
 
  	/* MES doorbells */
  	uint32_t			db_start_dw_offset;
+	uint32_t			num_mes_dbs;
+	unsigned long			*doorbell_bitmap;
 
  	/* ip specific functions */
  	const struct amdgpu_mes_funcs   *funcs;
#21 lore raw

Re: [PATCH v3 11/12] drm/amdgpu: use doorbell mgr for MES kernel doorbells

Quoted 11 lines · Christian König

Am 20.06.23 um 19:16 schrieb Shashank Sharma:

Quoted 10 lines

This patch:
- Removes the existing doorbell management code, and its variables from the doorbell_init function, it will be done in doorbell manager now.
- uses the doorbell page created for MES kernel level needs (doorbells for MES self tests)
- current MES code was allocating MES doorbells in MES process context, but those were getting written using kernel doorbell calls. This patch instead allocates a MES kernel doorbell for this (in add_hw_queue).

Quoted 5 lines
Quoted 2 lines

V2: Create an extra page of doorbells for MES during kernel doorbell creation (Alex)

As discussed on the other patch, this should probably be merged with patch #6 or otherwise we break the MES test in between.

I have instead moved the MES init part here in this patch, so that all MES code changes are in this same patch, whereas patch 6 contains only the base kernel doorbell object creation code.

- Shashank

Quoted 187 lines

Apart from that this patch set looks good to me now and is most likely ready to land.

Christian.

Quoted 182 lines

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c | 94 ++++++++++--------------- drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h |  3 +

  2 files changed, 40 insertions(+), 57 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +0 -0
index f0f00466b59f..01c3d27512d9 100644
@@ -67,91 +67,70 @@ unsigned int 

amdgpu_mes_get_doorbell_dw_offset_in_bar(
          doorbell_id * 2);
  }
  -static int amdgpu_mes_queue_doorbell_get(struct amdgpu_device *adev, +static int amdgpu_mes_kernel_doorbell_get(struct amdgpu_device *adev, struct amdgpu_mes_process *process,
                       int ip_type, uint64_t *doorbell_index)
  {
      unsigned int offset, found;
+    struct amdgpu_mes *mes = &adev->mes;
  -    if (ip_type == AMDGPU_RING_TYPE_SDMA) {
+    if (ip_type == AMDGPU_RING_TYPE_SDMA)
          offset = adev->doorbell_index.sdma_engine[0];
-        found = find_next_zero_bit(process->doorbell_bitmap,
- AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS,
-                       offset);
-    } else {
-        found = find_first_zero_bit(process->doorbell_bitmap,
- AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS);
-    }
+    else
+        offset = 0;
  -    if (found >= AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS) {
+    found = find_next_zero_bit(mes->doorbell_bitmap,
mes->num_mes_dbs, offset);
+    if (found >= mes->num_mes_dbs) {
          DRM_WARN("No doorbell available\n");
          return -ENOSPC;
      }
  -    set_bit(found, process->doorbell_bitmap);
-
-    *doorbell_index = amdgpu_mes_get_doorbell_dw_offset_in_bar(adev,
-                process->doorbell_index, found);
+    set_bit(found, mes->doorbell_bitmap);
  +    /* Get the absolute doorbell index on BAR */
+    *doorbell_index = mes->db_start_dw_offset + found * 2;
      return 0;
  }
  -static void amdgpu_mes_queue_doorbell_free(struct amdgpu_device *adev, +static void amdgpu_mes_kernel_doorbell_free(struct amdgpu_device *adev, struct amdgpu_mes_process *process,
                         uint32_t doorbell_index)
  {
-    unsigned int old, doorbell_id;
-
-    doorbell_id = doorbell_index -
-        (process->doorbell_index *
-         amdgpu_mes_doorbell_process_slice(adev)) / sizeof(u32);
-    doorbell_id /= 2;
+    unsigned int old, rel_index;
+    struct amdgpu_mes *mes = &adev->mes;
  -    old = test_and_clear_bit(doorbell_id, process->doorbell_bitmap);
+    /* Find the relative index of the doorbell in this object */
+    rel_index = (doorbell_index - mes->db_start_dw_offset) / 2;
+    old = test_and_clear_bit(rel_index, mes->doorbell_bitmap); WARN_ON(!old);
  }
    static int amdgpu_mes_doorbell_init(struct amdgpu_device *adev)
  {
-    size_t doorbell_start_offset;
-    size_t doorbell_aperture_size;
-    size_t doorbell_process_limit;
-    size_t aggregated_doorbell_start;
      int i;
+    struct amdgpu_mes *mes = &adev->mes;
  -    aggregated_doorbell_start = (adev->doorbell_index.max_assignment + 1) * sizeof(u32); -    aggregated_doorbell_start =
-        roundup(aggregated_doorbell_start, PAGE_SIZE);

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +2 -20
-
-    doorbell_start_offset = aggregated_doorbell_start + PAGE_SIZE;
-    doorbell_start_offset =
-        roundup(doorbell_start_offset,
-            amdgpu_mes_doorbell_process_slice(adev));
-
-    doorbell_aperture_size = adev->doorbell.size;
-    doorbell_aperture_size =
-            rounddown(doorbell_aperture_size,
-                  amdgpu_mes_doorbell_process_slice(adev));
-
-    if (doorbell_aperture_size > doorbell_start_offset)
-        doorbell_process_limit =
-            (doorbell_aperture_size - doorbell_start_offset) /
-            amdgpu_mes_doorbell_process_slice(adev);
-    else
-        return -ENOSPC;
-
-    adev->mes.doorbell_id_offset = doorbell_start_offset / sizeof(u32);
-    adev->mes.max_doorbell_slices = doorbell_process_limit;
+    /* Bitmap for dynamic allocation of kernel doorbells */
+    mes->doorbell_bitmap = bitmap_zalloc(PAGE_SIZE / sizeof(u32), 

GFP_KERNEL);
+    if (!mes->doorbell_bitmap) {
+        DRM_ERROR("Failed to allocate MES doorbell bitmap\n"); +        return -ENOMEM;
+    }
  -    /* allocate Qword range for aggregated doorbell */ -    for (i = 0; i < AMDGPU_MES_PRIORITY_NUM_LEVELS; i++) -        adev->mes.aggregated_doorbells[i] =
-            aggregated_doorbell_start / sizeof(u32) + i * 2; +    mes->num_mes_dbs = PAGE_SIZE / AMDGPU_ONE_DOORBELL_SIZE; +    for (i = 0; i < AMDGPU_MES_PRIORITY_NUM_LEVELS; i++) { +        adev->mes.aggregated_doorbells[i] = mes->db_start_dw_offset
+ i * 2;
+        set_bit(i, mes->doorbell_bitmap);
+    }
  -    DRM_INFO("max_doorbell_slices=%zu\n", doorbell_process_limit); return 0;
  }
  +static void amdgpu_mes_doorbell_free(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +4 -0
+{
+    bitmap_free(adev->mes.doorbell_bitmap);
+}
+

  int amdgpu_mes_init(struct amdgpu_device *adev) {
      int i, r;

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +2 -1
@@ -250,6 +229,7 @@ void amdgpu_mes_fini(struct amdgpu_device *adev)
       amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs);
       amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs);
       amdgpu_device_wb_free(adev, adev->mes.read_val_offs);
+    amdgpu_mes_doorbell_free(adev);
         idr_destroy(&adev->mes.pasid_idr);
       idr_destroy(&adev->mes.gang_id_idr);
@@ -679,7 +659,7 @@ int amdgpu_mes_add_hw_queue(struct amdgpu_device 
 *adev, int gang_id,
       *queue_id = queue->queue_id = r;
         /* allocate a doorbell index for the queue */
-    r = amdgpu_mes_queue_doorbell_get(adev, gang->process,
+    r = amdgpu_mes_kernel_doorbell_get(adev, gang->process,
                         qprops->queue_type,
                         &qprops->doorbell_off);
       if (r)
@@ -737,7 +717,7 @@ int amdgpu_mes_add_hw_queue(struct amdgpu_device 
 *adev, int gang_id,
       return 0;

    clean_up_doorbell:
-    amdgpu_mes_queue_doorbell_free(adev, gang->process, +    amdgpu_mes_kernel_doorbell_free(adev, gang->process, qprops->doorbell_off); clean_up_queue_id:
      spin_lock_irqsave(&adev->mes.queue_id_lock, flags);

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +1 -1
@@ -792,7 +772,7 @@ int amdgpu_mes_remove_hw_queue(struct 
 amdgpu_device *adev, int queue_id)
                 queue_id);
         list_del(&queue->list);
-    amdgpu_mes_queue_doorbell_free(adev, gang->process,
+    amdgpu_mes_kernel_doorbell_free(adev, gang->process,
                          queue->doorbell_off);
       amdgpu_mes_unlock(&adev->mes);

  diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h index a403418d5eac..2c604e04fd5c 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +3 -0
@@ -27,6 +27,7 @@
   #include "amdgpu_irq.h"
   #include "kgd_kfd_interface.h"
   #include "amdgpu_gfx.h"
+#include "amdgpu_doorbell.h"
   #include <linux/sched/mm.h>
     #define AMDGPU_MES_MAX_COMPUTE_PIPES        8
@@ -130,6 +131,8 @@ struct amdgpu_mes {
         /* MES doorbells */
       uint32_t            db_start_dw_offset;
+    uint32_t            num_mes_dbs;
+    unsigned long            *doorbell_bitmap;
         /* ip specific functions */
       const struct amdgpu_mes_funcs   *funcs;
#22 lore raw

PATCH v4 11/12 drm/amdgpu: use doorbell mgr for MES kernel doorbells

In reply to Shashank Sharma

This patch:
- Removes the existing doorbell management code, and its variables from the doorbell_init function, it will be done in doorbell manager now.
- uses the doorbell page created for MES kernel level needs (doorbells for MES self tests)
- current MES code was allocating MES doorbells in MES process context, but those were getting written using kernel doorbell calls. This patch instead allocates a MES kernel doorbell for this (in add_hw_queue).

V2: Create an extra page of doorbells for MES during kernel doorbell
creation (Alex)

---
 .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c  |  4 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c       | 94 ++++++++-----------
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h       |  3 +
 3 files changed, 44 insertions(+), 57 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c +4 -0
index ba2b2f4cad47..e1599fd72954 100644
@@ -145,6 +145,10 @@ int amdgpu_doorbell_create_kernel_doorbells(struct amdgpu_device *adev)
 	/* Reserve first num_kernel_doorbells (page-aligned) for kernel ops */
 	size = ALIGN(adev->doorbell.num_kernel_doorbells * sizeof(u32), PAGE_SIZE);
 
+	/* Allocate an extra page for MES kernel usages (ring test) */
+	adev->mes.db_start_dw_offset = size / sizeof(u32);
+	size += PAGE_SIZE;
+
 	r = amdgpu_bo_create_kernel(adev,
 				    size,
 				    PAGE_SIZE,
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +37 -57

155 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +3 -0
index a403418d5eac..2c604e04fd5c 100644
@@ -27,6 +27,7 @@
 #include "amdgpu_irq.h"
 #include "kgd_kfd_interface.h"
 #include "amdgpu_gfx.h"
+#include "amdgpu_doorbell.h"
 #include <linux/sched/mm.h>
 
 #define AMDGPU_MES_MAX_COMPUTE_PIPES        8
@@ -130,6 +131,8 @@ struct amdgpu_mes {
 
 	/* MES doorbells */
 	uint32_t			db_start_dw_offset;
+	uint32_t			num_mes_dbs;
+	unsigned long			*doorbell_bitmap;
 
 	/* ip specific functions */
 	const struct amdgpu_mes_funcs   *funcs;
Signature · 1 line
2.40.1
#23 lore raw

PATCH v3 12/12 drm/amdgpu: cleanup MES process level doorbells

In reply to Shashank Sharma

MES allocates process level doorbells, but there is no userspace client to consume it. It was only being used for the MES ring tests (in kernel), and was written by kernel doorbell write.

The previous patch of this series has changed the MES ring test code to use kernel level MES doorbells. This patch now cleans up the process level doorbell allocation code which is not required.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c | 54 +------------------------
 drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h | 10 -----
 2 files changed, 1 insertion(+), 63 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c +1 -53
index 01c3d27512d9..8ab558ceafe8 100644
@@ -38,35 +38,6 @@ int amdgpu_mes_doorbell_process_slice(struct amdgpu_device *adev)
 		       PAGE_SIZE);
 }
 
-int amdgpu_mes_alloc_process_doorbells(struct amdgpu_device *adev,
-				      unsigned int *doorbell_index)
-{
-	int r = ida_simple_get(&adev->mes.doorbell_ida, 2,
-			       adev->mes.max_doorbell_slices,
-			       GFP_KERNEL);
-	if (r > 0)
-		*doorbell_index = r;
-
-	return r;
-}
-
-void amdgpu_mes_free_process_doorbells(struct amdgpu_device *adev,
-				      unsigned int doorbell_index)
-{
-	if (doorbell_index)
-		ida_simple_remove(&adev->mes.doorbell_ida, doorbell_index);
-}
-
-unsigned int amdgpu_mes_get_doorbell_dw_offset_in_bar(
-					struct amdgpu_device *adev,
-					uint32_t doorbell_index,
-					unsigned int doorbell_id)
-{
-	return ((doorbell_index *
-		amdgpu_mes_doorbell_process_slice(adev)) / sizeof(u32) +
-		doorbell_id * 2);
-}
-
 static int amdgpu_mes_kernel_doorbell_get(struct amdgpu_device *adev,
 					 struct amdgpu_mes_process *process,
 					 int ip_type, uint64_t *doorbell_index)
@@ -258,15 +229,6 @@ int amdgpu_mes_create_process(struct amdgpu_device *adev, int pasid,
 		return -ENOMEM;
 	}
 
-	process->doorbell_bitmap =
-		kzalloc(DIV_ROUND_UP(AMDGPU_MES_MAX_NUM_OF_QUEUES_PER_PROCESS,
-				     BITS_PER_BYTE), GFP_KERNEL);
-	if (!process->doorbell_bitmap) {
-		DRM_ERROR("failed to allocate doorbell bitmap\n");
-		kfree(process);
-		return -ENOMEM;
-	}
-
 	/* allocate the process context bo and map it */
 	r = amdgpu_bo_create_kernel(adev, AMDGPU_MES_PROC_CTX_SIZE, PAGE_SIZE,
 				    AMDGPU_GEM_DOMAIN_GTT,
@@ -293,15 +255,6 @@ int amdgpu_mes_create_process(struct amdgpu_device *adev, int pasid,
 		goto clean_up_ctx;
 	}
 
-	/* allocate the starting doorbell index of the process */
-	r = amdgpu_mes_alloc_process_doorbells(adev, &process->doorbell_index);
-	if (r < 0) {
-		DRM_ERROR("failed to allocate doorbell for process\n");
-		goto clean_up_pasid;
-	}
-
-	DRM_DEBUG("process doorbell index = %d\n", process->doorbell_index);
-
 	INIT_LIST_HEAD(&process->gang_list);
 	process->vm = vm;
 	process->pasid = pasid;
@@ -311,15 +264,12 @@ int amdgpu_mes_create_process(struct amdgpu_device *adev, int pasid,
 	amdgpu_mes_unlock(&adev->mes);
 	return 0;
 
-clean_up_pasid:
-	idr_remove(&adev->mes.pasid_idr, pasid);
-	amdgpu_mes_unlock(&adev->mes);
 clean_up_ctx:
+	amdgpu_mes_unlock(&adev->mes);
 	amdgpu_bo_free_kernel(&process->proc_ctx_bo,
 			      &process->proc_ctx_gpu_addr,
 			      &process->proc_ctx_cpu_ptr);
 clean_up_memory:
-	kfree(process->doorbell_bitmap);
 	kfree(process);
 	return r;
 }
@@ -365,7 +315,6 @@ void amdgpu_mes_destroy_process(struct amdgpu_device *adev, int pasid)
 		idr_remove(&adev->mes.gang_id_idr, gang->gang_id);
 	}
 
-	amdgpu_mes_free_process_doorbells(adev, process->doorbell_index);
 	idr_remove(&adev->mes.pasid_idr, pasid);
 	amdgpu_mes_unlock(&adev->mes);
 
@@ -387,7 +336,6 @@ void amdgpu_mes_destroy_process(struct amdgpu_device *adev, int pasid)
 	amdgpu_bo_free_kernel(&process->proc_ctx_bo,
 			      &process->proc_ctx_gpu_addr,
 			      &process->proc_ctx_cpu_ptr);
-	kfree(process->doorbell_bitmap);
 	kfree(process);
 }
 
drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h +0 -10
index 2c604e04fd5c..dc78c9e0697b 100644
@@ -77,7 +77,6 @@ struct amdgpu_mes {
 	uint32_t			kiq_version;
 
 	uint32_t                        total_max_queue;
-	uint32_t                        doorbell_id_offset;
 	uint32_t                        max_doorbell_slices;
 
 	uint64_t                        default_process_quantum;
@@ -148,7 +147,6 @@ struct amdgpu_mes_process {
 	uint64_t 		process_quantum;
 	struct 			list_head gang_list;
 	uint32_t 		doorbell_index;
-	unsigned long 		*doorbell_bitmap;
 	struct mutex		doorbell_lock;
 };
 
@@ -368,14 +366,6 @@ int amdgpu_mes_ctx_unmap_meta_data(struct amdgpu_device *adev,
 
 int amdgpu_mes_self_test(struct amdgpu_device *adev);
 
-int amdgpu_mes_alloc_process_doorbells(struct amdgpu_device *adev,
-					unsigned int *doorbell_index);
-void amdgpu_mes_free_process_doorbells(struct amdgpu_device *adev,
-					unsigned int doorbell_index);
-unsigned int amdgpu_mes_get_doorbell_dw_offset_in_bar(
-					struct amdgpu_device *adev,
-					uint32_t doorbell_index,
-					unsigned int doorbell_id);
 int amdgpu_mes_doorbell_process_slice(struct amdgpu_device *adev);
 
 /*
Signature · 1 line
2.40.1