Skip to contentSkip to conversation

Raven support for KFD v2

PATCH In review Felix Kuehling 4 participants 15 messages Jul 13, 2018 – Jul 30, 2018 7 messages not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Raven support for KFD v2

Raven refers to Ryzen APUs with integrated GFXv9 GPU. This patch series completes Raven support for KFD:

* fix up memory banks info from CRAT
* support different number of SDMA engines
* workaround IOMMUv2 PPR issues
* add device info

v2: Updated patch descriptions, no code changes

Yong Zhao (6):
drm/amdkfd: Consolidate duplicate memory banks info in topology drm/amdkfd: Make SDMA engine number an ASIC-dependent variable drm/amdkfd: Avoid flooding dmesg on Raven due to IOMMU issues drm/amdkfd: Workaround to accommodate Raven too many PPR issue drm/amdkfd: Optimize out some duplicated code in
kfd_signal_iommu_event()
drm/amdkfd: Enable Raven for KFD

 drivers/gpu/drm/amd/amdkfd/kfd_crat.c              | 57 +++++++++++++++++-----
 drivers/gpu/drm/amd/amdkfd/kfd_device.c            | 28 +++++++++++
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c  | 29 +++++++----
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.h  |  6 +--
 drivers/gpu/drm/amd/amdkfd/kfd_events.c            | 47 ++++++++++--------
 drivers/gpu/drm/amd/amdkfd/kfd_iommu.c             |  2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h              |  1 +
 .../gpu/drm/amd/amdkfd/kfd_process_queue_manager.c |  3 +-
 8 files changed, 126 insertions(+), 47 deletions(-)
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1531513068-3805-1-git-send-email-Felix.Kuehling-5C7GfCeVMHo@public.gmane.org

#2 lore raw

PATCH 1/6 drm/amdkfd: Consolidate duplicate memory banks info in topology

From: Yong Zhao <[email protected]>

If there are several memory banks that has the same properties in CRAT, we aggregate them into one memory bank. This cleans up memory banks on APUs (e.g. Raven) where the CRAT reports each memory channel as a separate bank. This only confuses user mode, which only deals with virtual memory.

---
 drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 57 ++++++++++++++++++++++++++++-------
 1 file changed, 46 insertions(+), 11 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_crat.c +46 -11
index 296b3f2..ee49960 100644
@@ -189,6 +189,21 @@ static int kfd_parse_subtype_cu(struct crat_subtype_computeunit *cu,
 	return 0;
 }
 
+static struct kfd_mem_properties *
+find_subtype_mem(uint32_t heap_type, uint32_t flags, uint32_t width,
+		struct kfd_topology_device *dev)
+{
+	struct kfd_mem_properties *props;
+
+	list_for_each_entry(props, &dev->mem_props, list) {
+		if (props->heap_type == heap_type
+				&& props->flags == flags
+				&& props->width == width)
+			return props;
+	}
+
+	return NULL;
+}
 /* kfd_parse_subtype_mem - parse memory subtypes and attach it to correct
  * topology device present in the device_list
  */
@@ -197,36 +212,56 @@ static int kfd_parse_subtype_mem(struct crat_subtype_memory *mem,
 {
 	struct kfd_mem_properties *props;
 	struct kfd_topology_device *dev;
+	uint32_t heap_type;
+	uint64_t size_in_bytes;
+	uint32_t flags = 0;
+	uint32_t width;
 
 	pr_debug("Found memory entry in CRAT table with proximity_domain=%d\n",
 			mem->proximity_domain);
 	list_for_each_entry(dev, device_list, list) {
 		if (mem->proximity_domain == dev->proximity_domain) {
-			props = kfd_alloc_struct(props);
-			if (!props)
-				return -ENOMEM;
-
 			/* We're on GPU node */
 			if (dev->node_props.cpu_cores_count == 0) {
 				/* APU */
 				if (mem->visibility_type == 0)
-					props->heap_type =
+					heap_type =
 						HSA_MEM_HEAP_TYPE_FB_PRIVATE;
 				/* dGPU */
 				else
-					props->heap_type = mem->visibility_type;
+					heap_type = mem->visibility_type;
 			} else
-				props->heap_type = HSA_MEM_HEAP_TYPE_SYSTEM;
+				heap_type = HSA_MEM_HEAP_TYPE_SYSTEM;
 
 			if (mem->flags & CRAT_MEM_FLAGS_HOT_PLUGGABLE)
-				props->flags |= HSA_MEM_FLAGS_HOT_PLUGGABLE;
+				flags |= HSA_MEM_FLAGS_HOT_PLUGGABLE;
 			if (mem->flags & CRAT_MEM_FLAGS_NON_VOLATILE)
-				props->flags |= HSA_MEM_FLAGS_NON_VOLATILE;
+				flags |= HSA_MEM_FLAGS_NON_VOLATILE;
 
-			props->size_in_bytes =
+			size_in_bytes =
 				((uint64_t)mem->length_high << 32) +
 							mem->length_low;
-			props->width = mem->width;
+			width = mem->width;
+
+			/* Multiple banks of the same type are aggregated into
+			 * one. User mode doesn't care about multiple physical
+			 * memory segments. It's managed as a single virtual
+			 * heap for user mode.
+			 */
+			props = find_subtype_mem(heap_type, flags, width, dev);
+			if (props) {
+				props->size_in_bytes += size_in_bytes;
+				break;
+			}
+
+			props = kfd_alloc_struct(props);
+			if (!props)
+				return -ENOMEM;
+
+			props->heap_type = heap_type;
+			props->flags = flags;
+			props->size_in_bytes = size_in_bytes;
+			props->width = width;
 
 			dev->node_props.mem_banks_count++;
 			list_add_tail(&props->list, &dev->mem_props);
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#3 lore raw

PATCH 2/6 drm/amdkfd: Make SDMA engine number an ASIC-dependent variable

In reply to a message not in the archive

From: Yong Zhao <[email protected]>

On Raven there is only one SDMA engine instead of previously assumed two, so we need to adapt our code to this new scenario.

---
 drivers/gpu/drm/amd/amdkfd/kfd_device.c            | 12 +++++++++
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c  | 29 +++++++++++++++-------
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.h  |  6 ++---
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h              |  1 +
 .../gpu/drm/amd/amdkfd/kfd_process_queue_manager.c |  3 +--
 5 files changed, 36 insertions(+), 15 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_device.c +12 -0
index 8faa8db..572235c 100644
@@ -52,6 +52,7 @@ static const struct kfd_device_info kaveri_device_info = {
 	.supports_cwsr = false,
 	.needs_iommu_device = true,
 	.needs_pci_atomics = false,
+	.num_sdma_engines = 2,
 };
 
 static const struct kfd_device_info carrizo_device_info = {
@@ -67,6 +68,7 @@ static const struct kfd_device_info carrizo_device_info = {
 	.supports_cwsr = true,
 	.needs_iommu_device = true,
 	.needs_pci_atomics = false,
+	.num_sdma_engines = 2,
 };
 #endif
 
@@ -83,6 +85,7 @@ static const struct kfd_device_info hawaii_device_info = {
 	.supports_cwsr = false,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = false,
+	.num_sdma_engines = 2,
 };
 
 static const struct kfd_device_info tonga_device_info = {
@@ -97,6 +100,7 @@ static const struct kfd_device_info tonga_device_info = {
 	.supports_cwsr = false,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = true,
+	.num_sdma_engines = 2,
 };
 
 static const struct kfd_device_info tonga_vf_device_info = {
@@ -111,6 +115,7 @@ static const struct kfd_device_info tonga_vf_device_info = {
 	.supports_cwsr = false,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = false,
+	.num_sdma_engines = 2,
 };
 
 static const struct kfd_device_info fiji_device_info = {
@@ -125,6 +130,7 @@ static const struct kfd_device_info fiji_device_info = {
 	.supports_cwsr = true,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = true,
+	.num_sdma_engines = 2,
 };
 
 static const struct kfd_device_info fiji_vf_device_info = {
@@ -139,6 +145,7 @@ static const struct kfd_device_info fiji_vf_device_info = {
 	.supports_cwsr = true,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = false,
+	.num_sdma_engines = 2,
 };
 
 
@@ -154,6 +161,7 @@ static const struct kfd_device_info polaris10_device_info = {
 	.supports_cwsr = true,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = true,
+	.num_sdma_engines = 2,
 };
 
 static const struct kfd_device_info polaris10_vf_device_info = {
@@ -168,6 +176,7 @@ static const struct kfd_device_info polaris10_vf_device_info = {
 	.supports_cwsr = true,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = false,
+	.num_sdma_engines = 2,
 };
 
 static const struct kfd_device_info polaris11_device_info = {
@@ -182,6 +191,7 @@ static const struct kfd_device_info polaris11_device_info = {
 	.supports_cwsr = true,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = true,
+	.num_sdma_engines = 2,
 };
 
 static const struct kfd_device_info vega10_device_info = {
@@ -196,6 +206,7 @@ static const struct kfd_device_info vega10_device_info = {
 	.supports_cwsr = true,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = false,
+	.num_sdma_engines = 2,
 };
 
 static const struct kfd_device_info vega10_vf_device_info = {
@@ -210,6 +221,7 @@ static const struct kfd_device_info vega10_vf_device_info = {
 	.supports_cwsr = true,
 	.needs_iommu_device = false,
 	.needs_pci_atomics = false,
+	.num_sdma_engines = 2,
 };
 
 
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +20 -9
index 97c9f10..ace94d6 100644
@@ -101,6 +101,17 @@ unsigned int get_pipes_per_mec(struct device_queue_manager *dqm)
 	return dqm->dev->shared_resources.num_pipe_per_mec;
 }
 
+static unsigned int get_num_sdma_engines(struct device_queue_manager *dqm)
+{
+	return dqm->dev->device_info->num_sdma_engines;
+}
+
+unsigned int get_num_sdma_queues(struct device_queue_manager *dqm)
+{
+	return dqm->dev->device_info->num_sdma_engines
+			* KFD_SDMA_QUEUES_PER_ENGINE;
+}
+
 void program_sh_mem_settings(struct device_queue_manager *dqm,
 					struct qcm_process_device *qpd)
 {
@@ -855,7 +866,7 @@ static int initialize_nocpsch(struct device_queue_manager *dqm)
 	}
 
 	dqm->vmid_bitmap = (1 << dqm->dev->vm_info.vmid_num_kfd) - 1;
-	dqm->sdma_bitmap = (1 << CIK_SDMA_QUEUES) - 1;
+	dqm->sdma_bitmap = (1 << get_num_sdma_queues(dqm)) - 1;
 
 	return 0;
 }
@@ -903,7 +914,7 @@ static int allocate_sdma_queue(struct device_queue_manager *dqm,
 static void deallocate_sdma_queue(struct device_queue_manager *dqm,
 				unsigned int sdma_queue_id)
 {
-	if (sdma_queue_id >= CIK_SDMA_QUEUES)
+	if (sdma_queue_id >= get_num_sdma_queues(dqm))
 		return;
 	dqm->sdma_bitmap |= (1 << sdma_queue_id);
 }
@@ -923,8 +934,8 @@ static int create_sdma_queue_nocpsch(struct device_queue_manager *dqm,
 	if (retval)
 		return retval;
 
-	q->properties.sdma_queue_id = q->sdma_id / CIK_SDMA_QUEUES_PER_ENGINE;
-	q->properties.sdma_engine_id = q->sdma_id % CIK_SDMA_QUEUES_PER_ENGINE;
+	q->properties.sdma_queue_id = q->sdma_id / get_num_sdma_engines(dqm);
+	q->properties.sdma_engine_id = q->sdma_id % get_num_sdma_engines(dqm);
 
 	retval = allocate_doorbell(qpd, q);
 	if (retval)
@@ -1011,7 +1022,7 @@ static int initialize_cpsch(struct device_queue_manager *dqm)
 	dqm->queue_count = dqm->processes_count = 0;
 	dqm->sdma_queue_count = 0;
 	dqm->active_runlist = false;
-	dqm->sdma_bitmap = (1 << CIK_SDMA_QUEUES) - 1;
+	dqm->sdma_bitmap = (1 << get_num_sdma_queues(dqm)) - 1;
 
 	INIT_WORK(&dqm->hw_exception_work, kfd_process_hw_exception);
 
@@ -1142,9 +1153,9 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
 		if (retval)
 			goto out_unlock;
 		q->properties.sdma_queue_id =
-			q->sdma_id / CIK_SDMA_QUEUES_PER_ENGINE;
+			q->sdma_id / get_num_sdma_engines(dqm);
 		q->properties.sdma_engine_id =
-			q->sdma_id % CIK_SDMA_QUEUES_PER_ENGINE;
+			q->sdma_id % get_num_sdma_engines(dqm);
 	}
 
 	retval = allocate_doorbell(qpd, q);
@@ -1791,8 +1802,8 @@ int dqm_debugfs_hqds(struct seq_file *m, void *data)
 		}
 	}
 
-	for (pipe = 0; pipe < CIK_SDMA_ENGINE_NUM; pipe++) {
-		for (queue = 0; queue < CIK_SDMA_QUEUES_PER_ENGINE; queue++) {
+	for (pipe = 0; pipe < get_num_sdma_engines(dqm); pipe++) {
+		for (queue = 0; queue < KFD_SDMA_QUEUES_PER_ENGINE; queue++) {
 			r = dqm->dev->kfd2kgd->hqd_sdma_dump(
 				dqm->dev->kgd, pipe, queue, &dump, &n_regs);
 			if (r)
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h +2 -4
index 52e708c..00da316 100644
@@ -33,10 +33,7 @@
 
 #define KFD_UNMAP_LATENCY_MS			(4000)
 #define QUEUE_PREEMPT_DEFAULT_TIMEOUT_MS (2 * KFD_UNMAP_LATENCY_MS + 1000)
-
-#define CIK_SDMA_QUEUES				(4)
-#define CIK_SDMA_QUEUES_PER_ENGINE		(2)
-#define CIK_SDMA_ENGINE_NUM			(2)
+#define KFD_SDMA_QUEUES_PER_ENGINE		(2)
 
 struct device_process_node {
 	struct qcm_process_device *qpd;
@@ -214,6 +211,7 @@ void program_sh_mem_settings(struct device_queue_manager *dqm,
 unsigned int get_queues_num(struct device_queue_manager *dqm);
 unsigned int get_queues_per_pipe(struct device_queue_manager *dqm);
 unsigned int get_pipes_per_mec(struct device_queue_manager *dqm);
+unsigned int get_num_sdma_queues(struct device_queue_manager *dqm);
 
 static inline unsigned int get_sh_mem_bases_32(struct kfd_process_device *pdd)
 {
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +1 -0
index 37d179e..ca83254 100644
@@ -203,6 +203,7 @@ struct kfd_device_info {
 	bool supports_cwsr;
 	bool needs_iommu_device;
 	bool needs_pci_atomics;
+	unsigned int num_sdma_engines;
 };
 
 struct kfd_mem_obj {
drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c +1 -2
index 1303b14..eb4e5fb 100644
@@ -186,8 +186,7 @@ int pqm_create_queue(struct process_queue_manager *pqm,
 
 	switch (type) {
 	case KFD_QUEUE_TYPE_SDMA:
-		if (dev->dqm->queue_count >=
-			CIK_SDMA_QUEUES_PER_ENGINE * CIK_SDMA_ENGINE_NUM) {
+		if (dev->dqm->queue_count >= get_num_sdma_queues(dev->dqm)) {
 			pr_err("Over-subscription is not allowed for SDMA.\n");
 			retval = -EPERM;
 			goto err_create_queue;
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#4 lore raw

PATCH 3/6 drm/amdkfd: Avoid flooding dmesg on Raven due to IOMMU issues

In reply to a message not in the archive

From: Yong Zhao <[email protected]>

On Raven Invalid PPRs (peripheral page requests) can be reported because multiple PPRs can be still queued when memory is freed. Apply a rate limit to avoid flooding the log in this case.

---
 drivers/gpu/drm/amd/amdkfd/kfd_iommu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdkfd/kfd_iommu.c +1 -1
index c718179..7a61f38 100644
@@ -190,7 +190,7 @@ static int iommu_invalid_ppr_cb(struct pci_dev *pdev, int pasid,
 {
 	struct kfd_dev *dev;
 
-	dev_warn(kfd_device,
+	dev_warn_ratelimited(kfd_device,
 			"Invalid PPR device %x:%x.%x pasid %d address 0x%lX flags 0x%X",
 			PCI_BUS_NUM(pdev->devfn),
 			PCI_SLOT(pdev->devfn),
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#5 lore raw

PATCH 4/6 drm/amdkfd: Workaround to accommodate Raven too many PPR issue

In reply to a message not in the archive

From: Yong Zhao <[email protected]>

On Raven multiple PPRs can be queued up by the hardware. When the first of those requests is handled by the IOMMU driver, the memory access succeeds. After that the application may be done with the memory and unmap it. At that point the page table entries are invalidated, but there are still outstanding duplicate PPRs for those addresses. When the IOMMU driver processes those duplicate requests, it finds invalid page table entries and triggers an invalid PPR fault.

As a workaround, don't signal invalid PPR faults on Raven to avoid segfaulting applications that haven't done anything wrong. As a side effect, real GPU memory access faults may go unnoticed by the application.

---
 drivers/gpu/drm/amd/amdkfd/kfd_events.c | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_events.c +16 -5
index 820133c..4dcacce 100644
@@ -932,13 +932,24 @@ void kfd_signal_iommu_event(struct kfd_dev *dev, unsigned int pasid,
 	up_read(&mm->mmap_sem);
 	mmput(mm);
 
-	mutex_lock(&p->event_mutex);
+	pr_debug("notpresent %d, noexecute %d, readonly %d\n",
+			memory_exception_data.failure.NotPresent,
+			memory_exception_data.failure.NoExecute,
+			memory_exception_data.failure.ReadOnly);
 
-	/* Lookup events by type and signal them */
-	lookup_events_by_type_and_signal(p, KFD_EVENT_TYPE_MEMORY,
-			&memory_exception_data);
+	/* Workaround on Raven to not kill the process when memory is freed
+	 * before IOMMU is able to finish processing all the excessive PPRs
+	 */
+	if (dev->device_info->asic_family != CHIP_RAVEN) {
+		mutex_lock(&p->event_mutex);
+
+		/* Lookup events by type and signal them */
+		lookup_events_by_type_and_signal(p, KFD_EVENT_TYPE_MEMORY,
+				&memory_exception_data);
+
+		mutex_unlock(&p->event_mutex);
+	}
 
-	mutex_unlock(&p->event_mutex);
 	kfd_unref_process(p);
 }
 #endif /* KFD_SUPPORT_IOMMU_V2 */
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#6 lore raw

PATCH 5/6 drm/amdkfd: Optimize out some duplicated code in kfd_signal_iommu_event()

In reply to a message not in the archive

From: Yong Zhao <[email protected]>

memory_exception_data is already initialized for not-present faults. It only needs to be overridden for permission faults.

---
 drivers/gpu/drm/amd/amdkfd/kfd_events.c | 26 +++++++++++---------------
 1 file changed, 11 insertions(+), 15 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_events.c +11 -15
index 4dcacce..e9f0e0a 100644
@@ -911,22 +911,18 @@ void kfd_signal_iommu_event(struct kfd_dev *dev, unsigned int pasid,
 	memory_exception_data.failure.NotPresent = 1;
 	memory_exception_data.failure.NoExecute = 0;
 	memory_exception_data.failure.ReadOnly = 0;
-	if (vma) {
-		if (vma->vm_start > address) {
-			memory_exception_data.failure.NotPresent = 1;
-			memory_exception_data.failure.NoExecute = 0;
+	if (vma && address >= vma->vm_start) {
+		memory_exception_data.failure.NotPresent = 0;
+
+		if (is_write_requested && !(vma->vm_flags & VM_WRITE))
+			memory_exception_data.failure.ReadOnly = 1;
+		else
 			memory_exception_data.failure.ReadOnly = 0;
-		} else {
-			memory_exception_data.failure.NotPresent = 0;
-			if (is_write_requested && !(vma->vm_flags & VM_WRITE))
-				memory_exception_data.failure.ReadOnly = 1;
-			else
-				memory_exception_data.failure.ReadOnly = 0;
-			if (is_execute_requested && !(vma->vm_flags & VM_EXEC))
-				memory_exception_data.failure.NoExecute = 1;
-			else
-				memory_exception_data.failure.NoExecute = 0;
-		}
+
+		if (is_execute_requested && !(vma->vm_flags & VM_EXEC))
+			memory_exception_data.failure.NoExecute = 1;
+		else
+			memory_exception_data.failure.NoExecute = 0;
 	}
 
 	up_read(&mm->mmap_sem);
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#7 lore raw

PATCH 6/6 drm/amdkfd: Enable Raven for KFD

In reply to a message not in the archive

From: Yong Zhao <[email protected]>

Add DID and kfd_device_info for Raven.

---
 drivers/gpu/drm/amd/amdkfd/kfd_device.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
drivers/gpu/drm/amd/amdkfd/kfd_device.c +16 -0
index 572235c..1b04871 100644
@@ -70,6 +70,21 @@ static const struct kfd_device_info carrizo_device_info = {
 	.needs_pci_atomics = false,
 	.num_sdma_engines = 2,
 };
+
+static const struct kfd_device_info raven_device_info = {
+	.asic_family = CHIP_RAVEN,
+	.max_pasid_bits = 16,
+	.max_no_of_hqd  = 24,
+	.doorbell_size  = 8,
+	.ih_ring_entry_size = 8 * sizeof(uint32_t),
+	.event_interrupt_class = &event_interrupt_class_v9,
+	.num_of_watch_points = 4,
+	.mqd_size_aligned = MQD_SIZE_ALIGNED,
+	.supports_cwsr = true,
+	.needs_iommu_device = true,
+	.needs_pci_atomics = true,
+	.num_sdma_engines = 1,
+};
 #endif
 
 static const struct kfd_device_info hawaii_device_info = {
@@ -259,6 +274,7 @@ static const struct kfd_deviceid supported_devices[] = {
 	{ 0x9875, &carrizo_device_info },	/* Carrizo */
 	{ 0x9876, &carrizo_device_info },	/* Carrizo */
 	{ 0x9877, &carrizo_device_info },	/* Carrizo */
+	{ 0x15DD, &raven_device_info },		/* Raven */
 #endif
 	{ 0x67A0, &hawaii_device_info },	/* Hawaii */
 	{ 0x67A1, &hawaii_device_info },	/* Hawaii */
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1531513068-3805-7-git-send-email-Felix.Kuehling-5C7GfCeVMHo@public.gmane.org

#9 lore raw

Re: [PATCH 6/6] drm/amdkfd: Enable Raven for KFD

Dear Felix, dear Yong,

Quoted 39 lines · Felix Kühling

From: Yong Zhao <[email protected]>

Add DID and kfd_device_info for Raven.

---
 drivers/gpu/drm/amd/amdkfd/kfd_device.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
drivers/gpu/drm/amd/amdkfd/kfd_device.c +15 -0
index 572235c..1b04871 100644
@@ -70,6 +70,21 @@ static const struct kfd_device_info carrizo_device_info = {
 	.needs_pci_atomics = false,
 	.num_sdma_engines = 2,
 };
+
+static const struct kfd_device_info raven_device_info = {
+	.asic_family = CHIP_RAVEN,
+	.max_pasid_bits = 16,
+	.max_no_of_hqd  = 24,
+	.doorbell_size  = 8,
+	.ih_ring_entry_size = 8 * sizeof(uint32_t),
+	.event_interrupt_class = &event_interrupt_class_v9,
+	.num_of_watch_points = 4,
+	.mqd_size_aligned = MQD_SIZE_ALIGNED,
+	.supports_cwsr = true,
+	.needs_iommu_device = true,
+	.needs_pci_atomics = true,
+	.num_sdma_engines = 1,
+};
 #endif
 
 static const struct kfd_device_info hawaii_device_info = {

drivers/gpu/drm/amd/amdkfd/kfd_device.c static const struct kfd_deviceid supported_devices[] = { · line 280

Quoted 7 lines
drivers/gpu/drm/amd/amdkfd/kfd_device.c +1 -0
@@ -259,6 +274,7 @@ static const struct kfd_deviceid supported_devices[] = {
 	{ 0x9875, &carrizo_device_info },	/* Carrizo */
 	{ 0x9876, &carrizo_device_info },	/* Carrizo */
 	{ 0x9877, &carrizo_device_info },	/* Carrizo */
+	{ 0x15DD, &raven_device_info },		/* Raven */
 #endif
 	{ 0x67A0, &hawaii_device_info },	/* Hawaii */
 	{ 0x67A1, &hawaii_device_info },	/* Hawaii */

I built the branch *amdkfd-next* from [1], but on the MIS B350M MORTAR (MS-7A37) with BIOS 1.G1 05/17/2018 it is not loaded.

```
$ uname -a
Linux tokeiihto 4.18.0-rc3-01368-gb5aa3f4aef72 #51 SMP Mon Jul 30 13:11:36 UTC 2018 x86_64 GNU/Linux
$ grep HSA /boot/config-4.18.0-rc3-01368-gb5aa3f4aef72
CONFIG_HSA_AMD=m
$ sudo dmesg | grep -i kfd
[   23.206628] calling  kfd_module_init+0x0/0x1000 [amdkfd] @ 365
[   23.209082] kfd kfd: Initialized module
[   23.209667] initcall kfd_module_init+0x0/0x1000 [amdkfd] returned 0 after 2960 usecs
[   23.212852] kfd kfd: DID 15dd is missing in supported_devices
[   23.212856] kfd kfd: kgd2kfd_probe failed
$ /sbin/modinfo amdkfd
filename:       /lib/modules/4.18.0-rc3-01368-gb5aa3f4aef72/kernel/drivers/gpu/drm/amd/amdkfd/amdkfd.ko
version:        0.7.2
license:        GPL and additional rights
description:    Standalone HSA driver for AMD's GPUs
author:         AMD Inc. and others
srcversion:     9AAD0B6E4510ACF8D20A888
depends:
retpoline:      Y 
intree:         Y 
name:           amdkfd
vermagic:       4.18.0-rc3-01368-gb5aa3f4aef72 SMP mod_unload 
sig_id:         PKCS#7
[…]
```

Any hints, what I am doing wrong?

Kind regards,

Paul

PS: Output of lspci:

```
$ lspci -nn
00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15d0]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Device [1022:15d1]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:15d3]
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:15db]
00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:15dc]
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e8]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e9]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ea]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15eb]
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ec]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ed]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ee]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ef]
15:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset USB 3.1 xHCI Controller [1022:43bb] (rev 02)
15:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset SATA Controller [1022:43b7] (rev 02)
15:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b2] (rev 02)
16:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
16:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
16:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
18:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
38:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] [1002:15dd] (rev c8)
38:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:15de]
38:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Device [1022:15df]
38:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e0]
38:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e1]
38:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e3]
39:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 61)
```

[1]: git://people.freedesktop.org/~gabbayo/linux.git

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): a4f4f2e1-5f2f-2ca4-ce74-e3e76690c7bb-KUpvgZVWgV9o1qOY/[email protected]

#11 lore raw

Re: [PATCH 6/6] drm/amdkfd: Enable Raven for KFD

Do you have IOMMUv2 enabled on this chipset ?
If not, amdkfd won't include raven support.

Oded

Quoted 129 lines · Paul Menzel

Dear Felix, dear Yong,

Quoted 39 lines · Felix Kühling

From: Yong Zhao <[email protected]>

Add DID and kfd_device_info for Raven.

---
 drivers/gpu/drm/amd/amdkfd/kfd_device.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
drivers/gpu/drm/amd/amdkfd/kfd_device.c +15 -0
index 572235c..1b04871 100644
@@ -70,6 +70,21 @@ static const struct kfd_device_info carrizo_device_info = {
      .needs_pci_atomics = false,
      .num_sdma_engines = 2,
 };
+
+static const struct kfd_device_info raven_device_info = {
+     .asic_family = CHIP_RAVEN,
+     .max_pasid_bits = 16,
+     .max_no_of_hqd  = 24,
+     .doorbell_size  = 8,
+     .ih_ring_entry_size = 8 * sizeof(uint32_t),
+     .event_interrupt_class = &event_interrupt_class_v9,
+     .num_of_watch_points = 4,
+     .mqd_size_aligned = MQD_SIZE_ALIGNED,
+     .supports_cwsr = true,
+     .needs_iommu_device = true,
+     .needs_pci_atomics = true,
+     .num_sdma_engines = 1,
+};
 #endif
 
 static const struct kfd_device_info hawaii_device_info = {

drivers/gpu/drm/amd/amdkfd/kfd_device.c static const struct kfd_deviceid supported_devices[] = { · line 280

Quoted 7 lines
drivers/gpu/drm/amd/amdkfd/kfd_device.c +1 -0
@@ -259,6 +274,7 @@ static const struct kfd_deviceid supported_devices[] = {
      { 0x9875, &carrizo_device_info },       /* Carrizo */
      { 0x9876, &carrizo_device_info },       /* Carrizo */
      { 0x9877, &carrizo_device_info },       /* Carrizo */
+     { 0x15DD, &raven_device_info },         /* Raven */
 #endif
      { 0x67A0, &hawaii_device_info },        /* Hawaii */
      { 0x67A1, &hawaii_device_info },        /* Hawaii */

I built the branch *amdkfd-next* from [1], but on the MIS B350M MORTAR (MS-7A37) with BIOS 1.G1 05/17/2018 it is not loaded.

```
$ uname -a
Linux tokeiihto 4.18.0-rc3-01368-gb5aa3f4aef72 #51 SMP Mon Jul 30 13:11:36 UTC 2018 x86_64 GNU/Linux
$ grep HSA /boot/config-4.18.0-rc3-01368-gb5aa3f4aef72
CONFIG_HSA_AMD=m
$ sudo dmesg | grep -i kfd
[   23.206628] calling  kfd_module_init+0x0/0x1000 [amdkfd] @ 365
[   23.209082] kfd kfd: Initialized module
[   23.209667] initcall kfd_module_init+0x0/0x1000 [amdkfd] returned 0 after 2960 usecs
[   23.212852] kfd kfd: DID 15dd is missing in supported_devices
[   23.212856] kfd kfd: kgd2kfd_probe failed
$ /sbin/modinfo amdkfd
filename:       /lib/modules/4.18.0-rc3-01368-gb5aa3f4aef72/kernel/drivers/gpu/drm/amd/amdkfd/amdkfd.ko
version:        0.7.2
license:        GPL and additional rights
description:    Standalone HSA driver for AMD's GPUs
author:         AMD Inc. and others
srcversion:     9AAD0B6E4510ACF8D20A888
depends:
retpoline:      Y
intree:         Y
name:           amdkfd
vermagic:       4.18.0-rc3-01368-gb5aa3f4aef72 SMP mod_unload
sig_id:         PKCS#7
[…]
```

Any hints, what I am doing wrong?

Kind regards,

Paul

PS: Output of lspci:

```
$ lspci -nn
00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15d0]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Device [1022:15d1]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:15d3]
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:15db]
00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:15dc]
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e8]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e9]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ea]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15eb]
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ec]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ed]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ee]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ef]
15:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset USB 3.1 xHCI Controller [1022:43bb] (rev 02)
15:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset SATA Controller [1022:43b7] (rev 02)
15:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b2] (rev 02)
16:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
16:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
16:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
18:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
38:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] [1002:15dd] (rev c8)
38:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:15de]
38:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Device [1022:15df]
38:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e0]
38:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e1]
38:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e3]
39:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 61)
```
Quoted 2 lines

[1]: git://people.freedesktop.org/~gabbayo/linux.git

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CAFCwf11vL6YTr6uzTvFACLRtYWh5LrpbSr4UGf5W3Jt+RZzmdw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#13 lore raw

Raven: KFD/HSA depends on AMD_IOMMU_V2 (was: [PATCH 6/6] drm/amdkfd: Enable Raven for KFD)

Dear Oded,

Quoted 1 line · Oded Gabbay

Do you have IOMMUv2 enabled on this chipset?

No, I did not. With AMD_IOMMU_V2 selected, the device is found.

```
$ grep AMD_IOMMU /boot/config-4.18.0-rc3-01368-gb5aa3f4aef72 
CONFIG_AMD_IOMMU=y
CONFIG_AMD_IOMMU_V2=m
$ sudo dmesg | grep -i kfd
[sudo] password for paul: 
[   20.767499] calling  kfd_module_init+0x0/0x1000 [amdkfd] @ 365
[   20.770567] kfd kfd: Initialized module
[   20.771195] initcall kfd_module_init+0x0/0x1000 [amdkfd] returned 0 after 3598 usecs
[   20.893432] Modules linked in: amdkfd amd_iommu_v2 snd_hda_codec_realtek crct10dif_pclmul crc32_pclmul snd_hda_codec_hdmi snd_hda_codec_generic ghash_clmulni_intel snd_hda_intel amdgpu(+) i2c_piix4 r8169 k10temp snd_hda_codec snd_hda_core mii snd_hwdep snd_pcm snd_timer snd chash soundcore gpu_sched wmi video crc32c_intel xhci_pci xhci_hcd
[   20.948465] kfd kfd: Allocated 3969056 bytes on gart
[   20.948663] kfd kfd: added device 1002:15dd
```
Quoted 1 line

If not, amdkfd won't include raven support.

Hmm, then the warning is very confusing. Also, shouldn’t the build system ensure that the dependencies are selected?

Kind regards,

Paul

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 23116928-594a-99be-a613-fcd89f3c73eb-KUpvgZVWgV9o1qOY/[email protected]

#15 lore raw

Re: Raven: KFD/HSA depends on AMD_IOMMU_V2 (was: [PATCH 6/6] drm/amdkfd: Enable Raven for KFD)

Quoted 22 lines · Paul Menzel

Dear Oded,

Quoted 1 line · Oded Gabbay

Do you have IOMMUv2 enabled on this chipset?

No, I did not. With AMD_IOMMU_V2 selected, the device is found.

```
$ grep AMD_IOMMU /boot/config-4.18.0-rc3-01368-gb5aa3f4aef72
CONFIG_AMD_IOMMU=y
CONFIG_AMD_IOMMU_V2=m
$ sudo dmesg | grep -i kfd
[sudo] password for paul:
[   20.767499] calling  kfd_module_init+0x0/0x1000 [amdkfd] @ 365
[   20.770567] kfd kfd: Initialized module
[   20.771195] initcall kfd_module_init+0x0/0x1000 [amdkfd] returned 0 after 3598 usecs
[   20.893432] Modules linked in: amdkfd amd_iommu_v2 snd_hda_codec_realtek crct10dif_pclmul crc32_pclmul snd_hda_codec_hdmi snd_hda_codec_generic ghash_clmulni_intel snd_hda_intel amdgpu(+) i2c_piix4 r8169 k10temp snd_hda_codec snd_hda_core mii snd_hwdep snd_pcm snd_timer snd chash soundcore gpu_sched wmi video crc32c_intel xhci_pci xhci_hcd
[   20.948465] kfd kfd: Allocated 3969056 bytes on gart
[   20.948663] kfd kfd: added device 1002:15dd
```
Quoted 4 lines
Quoted 1 line

If not, amdkfd won't include raven support.

Hmm, then the warning is very confusing. Also, shouldn’t the build system ensure that the dependencies are selected?

It's only required for APUs. If you use a dGPU in a intel system for example, you may not want the AMD IOMMU driver.

Alex
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CADnq5_OMn3eHQTFeqSi9VExQmEEsi7hzwedPgPgWG_0PXiFNew-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#17 lore raw

Re: Raven: KFD/HSA depends on AMD_IOMMU_V2

Dear Alex,

Quoted 21 lines · Alex Deucher
Quoted 19 lines · Paul Menzel
Quoted 1 line · Oded Gabbay

Do you have IOMMUv2 enabled on this chipset?

No, I did not. With AMD_IOMMU_V2 selected, the device is found.

```
$ grep AMD_IOMMU /boot/config-4.18.0-rc3-01368-gb5aa3f4aef72
CONFIG_AMD_IOMMU=y
CONFIG_AMD_IOMMU_V2=m
$ sudo dmesg | grep -i kfd
[sudo] password for paul:
[   20.767499] calling  kfd_module_init+0x0/0x1000 [amdkfd] @ 365
[   20.770567] kfd kfd: Initialized module
[   20.771195] initcall kfd_module_init+0x0/0x1000 [amdkfd] returned 0 after 3598 usecs
[   20.893432] Modules linked in: amdkfd amd_iommu_v2 snd_hda_codec_realtek crct10dif_pclmul crc32_pclmul snd_hda_codec_hdmi snd_hda_codec_generic ghash_clmulni_intel snd_hda_intel amdgpu(+) i2c_piix4 r8169 k10temp snd_hda_codec snd_hda_core mii snd_hwdep snd_pcm snd_timer snd chash soundcore gpu_sched wmi video crc32c_intel xhci_pci xhci_hcd
[   20.948465] kfd kfd: Allocated 3969056 bytes on gart
[   20.948663] kfd kfd: added device 1002:15dd
```
Quoted 7 lines
Quoted 4 lines
Quoted 1 line

If not, amdkfd won't include raven support.

Hmm, then the warning is very confusing. Also, shouldn’t the build system ensure that the dependencies are selected?

It's only required for APUs. If you use a dGPU in a intel system for example, you may not want the AMD IOMMU driver.

Yes, I understand that, but couldn’t that be documented in the Kconfig help text?

It looks like *imply* is used already, but that is not shown in menuconfig, if I am not mistaken.

Quoted 7 lines

config HSA_AMD
tristate "HSA kernel driver for AMD GPU devices" depends on DRM_AMDGPU && X86_64
imply AMD_IOMMU_V2
select MMU_NOTIFIER
help
Enable this if you want to use HSA features on AMD GPU devices.

Kind regards,

Paul

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#18 lore raw

Re: [PATCH 6/6] drm/amdkfd: Enable Raven for KFD

In reply to a message not in the archive

I'm using that branch myself successfully. For Raven (and other APU) support, make sure you have AMD_IOMMU_V2 enabled in your kernel config.

Regards,
  Felix

Quoted 127 lines · Paul Menzel

Dear Felix, dear Yong,

Quoted 39 lines · Felix Kühling

From: Yong Zhao <[email protected]>

Add DID and kfd_device_info for Raven.

---
 drivers/gpu/drm/amd/amdkfd/kfd_device.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
drivers/gpu/drm/amd/amdkfd/kfd_device.c +15 -0
index 572235c..1b04871 100644
@@ -70,6 +70,21 @@ static const struct kfd_device_info carrizo_device_info = {
 	.needs_pci_atomics = false,
 	.num_sdma_engines = 2,
 };
+
+static const struct kfd_device_info raven_device_info = {
+	.asic_family = CHIP_RAVEN,
+	.max_pasid_bits = 16,
+	.max_no_of_hqd  = 24,
+	.doorbell_size  = 8,
+	.ih_ring_entry_size = 8 * sizeof(uint32_t),
+	.event_interrupt_class = &event_interrupt_class_v9,
+	.num_of_watch_points = 4,
+	.mqd_size_aligned = MQD_SIZE_ALIGNED,
+	.supports_cwsr = true,
+	.needs_iommu_device = true,
+	.needs_pci_atomics = true,
+	.num_sdma_engines = 1,
+};
 #endif
 
 static const struct kfd_device_info hawaii_device_info = {

drivers/gpu/drm/amd/amdkfd/kfd_device.c static const struct kfd_deviceid supported_devices[] = { · line 280

Quoted 7 lines
drivers/gpu/drm/amd/amdkfd/kfd_device.c +1 -0
@@ -259,6 +274,7 @@ static const struct kfd_deviceid supported_devices[] = {
 	{ 0x9875, &carrizo_device_info },	/* Carrizo */
 	{ 0x9876, &carrizo_device_info },	/* Carrizo */
 	{ 0x9877, &carrizo_device_info },	/* Carrizo */
+	{ 0x15DD, &raven_device_info },		/* Raven */
 #endif
 	{ 0x67A0, &hawaii_device_info },	/* Hawaii */
 	{ 0x67A1, &hawaii_device_info },	/* Hawaii */

I built the branch *amdkfd-next* from [1], but on the MIS B350M MORTAR (MS-7A37) with BIOS 1.G1 05/17/2018 it is not loaded.

```
$ uname -a
Linux tokeiihto 4.18.0-rc3-01368-gb5aa3f4aef72 #51 SMP Mon Jul 30 13:11:36 UTC 2018 x86_64 GNU/Linux
$ grep HSA /boot/config-4.18.0-rc3-01368-gb5aa3f4aef72
CONFIG_HSA_AMD=m
$ sudo dmesg | grep -i kfd
[   23.206628] calling  kfd_module_init+0x0/0x1000 [amdkfd] @ 365
[   23.209082] kfd kfd: Initialized module
[   23.209667] initcall kfd_module_init+0x0/0x1000 [amdkfd] returned 0 after 2960 usecs
[   23.212852] kfd kfd: DID 15dd is missing in supported_devices
[   23.212856] kfd kfd: kgd2kfd_probe failed
$ /sbin/modinfo amdkfd
filename:       /lib/modules/4.18.0-rc3-01368-gb5aa3f4aef72/kernel/drivers/gpu/drm/amd/amdkfd/amdkfd.ko
version:        0.7.2
license:        GPL and additional rights
description:    Standalone HSA driver for AMD's GPUs
author:         AMD Inc. and others
srcversion:     9AAD0B6E4510ACF8D20A888
depends:
retpoline:      Y 
intree:         Y 
name:           amdkfd
vermagic:       4.18.0-rc3-01368-gb5aa3f4aef72 SMP mod_unload 
sig_id:         PKCS#7
[…]
```

Any hints, what I am doing wrong?

Kind regards,

Paul

PS: Output of lspci:

```
$ lspci -nn
00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15d0]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Device [1022:15d1]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:15d3]
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:15db]
00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:15dc]
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e8]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e9]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ea]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15eb]
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ec]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ed]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ee]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:15ef]
15:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset USB 3.1 xHCI Controller [1022:43bb] (rev 02)
15:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset SATA Controller [1022:43b7] (rev 02)
15:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b2] (rev 02)
16:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
16:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
16:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
18:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
38:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] [1002:15dd] (rev c8)
38:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:15de]
38:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Device [1022:15df]
38:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e0]
38:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e1]
38:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Device [1022:15e3]
39:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 61)
```
Quoted 2 lines

[1]: git://people.freedesktop.org/~gabbayo/linux.git

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Acked-by
#19 lore raw
In reply to a message not in the archive
Quoted 9 lines · Felix Kuehling

Raven refers to Ryzen APUs with integrated GFXv9 GPU. This patch series completes Raven support for KFD:

* fix up memory banks info from CRAT
* support different number of SDMA engines
* workaround IOMMUv2 PPR issues
* add device info

v2: Updated patch descriptions, no code changes

Looks good. For the remaining patches:

Quoted 20 lines

Yong Zhao (6):
drm/amdkfd: Consolidate duplicate memory banks info in topology drm/amdkfd: Make SDMA engine number an ASIC-dependent variable drm/amdkfd: Avoid flooding dmesg on Raven due to IOMMU issues drm/amdkfd: Workaround to accommodate Raven too many PPR issue drm/amdkfd: Optimize out some duplicated code in
kfd_signal_iommu_event()
drm/amdkfd: Enable Raven for KFD

 drivers/gpu/drm/amd/amdkfd/kfd_crat.c              | 57 +++++++++++++++++-----
 drivers/gpu/drm/amd/amdkfd/kfd_device.c            | 28 +++++++++++
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c  | 29 +++++++----
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.h  |  6 +--
 drivers/gpu/drm/amd/amdkfd/kfd_events.c            | 47 ++++++++++--------
 drivers/gpu/drm/amd/amdkfd/kfd_iommu.c             |  2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h              |  1 +
 .../gpu/drm/amd/amdkfd/kfd_process_queue_manager.c |  3 +-
 8 files changed, 126 insertions(+), 47 deletions(-)
Quoted 7 lines

--
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CADnq5_PxwAAfWHEv8RmtRwkk80OrvnS9dVF=kFnjTcQT7Ypd3Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#21 lore raw

Hi Felix,
Thanks for the patch-set. Applied to -next.

Oded

Quoted 10 lines · Alex Deucher
Quoted 8 lines · Felix Kuehling

Raven refers to Ryzen APUs with integrated GFXv9 GPU. This patch series completes Raven support for KFD:

* fix up memory banks info from CRAT
* support different number of SDMA engines
* workaround IOMMUv2 PPR issues
* add device info

Quoted 32 lines
Quoted 1 line

v2: Updated patch descriptions, no code changes

Looks good. For the remaining patches:

Quoted 27 lines

Yong Zhao (6):
drm/amdkfd: Consolidate duplicate memory banks info in topology drm/amdkfd: Make SDMA engine number an ASIC-dependent variable drm/amdkfd: Avoid flooding dmesg on Raven due to IOMMU issues drm/amdkfd: Workaround to accommodate Raven too many PPR issue drm/amdkfd: Optimize out some duplicated code in
kfd_signal_iommu_event()
drm/amdkfd: Enable Raven for KFD

 drivers/gpu/drm/amd/amdkfd/kfd_crat.c              | 57 +++++++++++++++++-----
 drivers/gpu/drm/amd/amdkfd/kfd_device.c            | 28 +++++++++++
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c  | 29 +++++++----
 .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.h  |  6 +--
 drivers/gpu/drm/amd/amdkfd/kfd_events.c            | 47 ++++++++++--------
 drivers/gpu/drm/amd/amdkfd/kfd_iommu.c             |  2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h              |  1 +
 .../gpu/drm/amd/amdkfd/kfd_process_queue_manager.c |  3 +-
 8 files changed, 126 insertions(+), 47 deletions(-)

--
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx