Skip to contentSkip to conversation

Enable new L1 security for Vega10 SR-IOV

PATCH In review Trigger Huang 2 participants 13 messages May 8, 2019 – May 9, 2019 3 messages not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Enable new L1 security for Vega10 SR-IOV

To support new Vega10 SR-IOV L1 security, KMD need some modifications
1: Due to the new features supported in FW(PSP, RLC, etc),
for register access during initialization, we have more modes:
1), request PSP to program
2), request RLC to program
3), request SR-IOV host driver to program and skip
programming them in amdgpu
4), Legacy MMIO access
We will try to read the firmware version to see which mode is support

2: If PSP FW support to program some registers, such as IH,
we need:
1), initialize PSP before IH
2), send the specific command to PSP

3: Support VMR ring. VMR ring, compared with physical platform
TMR ring, the program sequence are nearly the same, but we will use another register set, mmMP0_SMN_C2PMSG_101/102/103 to communicate with PSP

4: Skip programming some registers in guest KMD
As some registers are processed by new L1 security, amdgpu on VF will fail to program them, and don’t worry, these registers will be programmed on the SR-IOV host driver side.

5: Call RLC to program some registers in instead of MMIO
With new L1 policy, some registers can’t be programmed in SR-IOV VF amdgpu with MMIO. Fortunately, new RLC firmware will support to program them with specific program sequence, which are described in the patch commit message

Trigger Huang (9):
drm/amdgpu: init vega10 SR-IOV reg access mode drm/amdgpu: initialize PSP before IH under SR-IOV drm/amdgpu: Add new PSP cmd GFX_CMD_ID_PROG_REG drm/amdgpu: implement PSP cmd GFX_CMD_ID_PROG_REG drm/amdgpu: call psp to progrm ih cntl in SR-IOV drm/amdgpu: Support PSP VMR ring for Vega10 VF drm/amdgpu: Skip setting some regs under Vega10 VF drm/amdgpu: add basic func for RLC program reg drm/amdgpu: RLC to program regs for Vega10 SR-IOV

 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c |  30 ++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c        |   4 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c           |  28 +++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h           |  11 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c          |  43 +++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h          |  12 ++
 drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c             | 114 ++++++++++---------
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c          |  20 ++--
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c             |   3 +
 drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c           |  25 ++++-
 drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c             |  19 ++++
 drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h           |   8 ++
 drivers/gpu/drm/amd/amdgpu/psp_v3_1.c             | 131 ++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c            |  14 ++-
 drivers/gpu/drm/amd/amdgpu/soc15.c                |  52 ++++++---
 drivers/gpu/drm/amd/amdgpu/soc15_common.h         |  57 +++++++++-
 drivers/gpu/drm/amd/amdgpu/vega10_ih.c            |  91 +++++++++++++--
 17 files changed, 514 insertions(+), 148 deletions(-)
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1557283541-7877-1-git-send-email-Trigger.Huang-5C7GfCeVMHo@public.gmane.org

#2 lore raw

PATCH 1/9 drm/amdgpu: init vega10 SR-IOV reg access mode

Set different register access mode according to the features provided by firmware

Change-Id Ia03e25a5a3b188f66363a0af487edfa21aafefc5
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  3 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c   | 43 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h   | 12 +++++++++
 drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c      | 19 +++++++++++++
 4 files changed, 77 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
index 3ff8899..615e775 100644
@@ -1532,6 +1532,9 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev)
 		r = amdgpu_virt_request_full_gpu(adev, true);
 		if (r)
 			return -EAGAIN;
+
+		/* query the reg access mode at the very beginning */
+		amdgpu_virt_init_reg_access_mode(adev);
 	}
 
 	adev->pm.pp_feature = amdgpu_pp_feature_mask;
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c +43 -0

47 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +12 -0
index 722deef..eb4cb1b 100644
@@ -48,6 +48,12 @@ struct amdgpu_vf_error_buffer {
 	uint64_t data[AMDGPU_VF_ERROR_ENTRY_SIZE];
 };
 
+/* According to the fw feature, some new reg access modes are supported */
+#define AMDGPU_VIRT_REG_ACCESS_LEGACY		(1 << 0) /* directly mmio */
+#define AMDGPU_VIRT_REG_ACCESS_PSP_PRG_IH	(1 << 1) /* by PSP */
+#define AMDGPU_VIRT_REG_ACCESS_RLC			(1 << 2) /* by RLC */
+#define AMDGPU_VIRT_REG_SKIP_SEETING		(1 << 3) /* Skip setting reg */
+
 /**
  * struct amdgpu_virt_ops - amdgpu device virt operations
  */
@@ -57,6 +63,7 @@ struct amdgpu_virt_ops {
 	int (*reset_gpu)(struct amdgpu_device *adev);
 	int (*wait_reset)(struct amdgpu_device *adev);
 	void (*trans_msg)(struct amdgpu_device *adev, u32 req, u32 data1, u32 data2, u32 data3);
+	void (*init_reg_access_mode)(struct amdgpu_device *adev);
 };
 
 /*
@@ -252,6 +259,7 @@ struct amdgpu_virt {
 	struct amdgpu_vf_error_buffer   vf_errors;
 	struct amdgpu_virt_fw_reserve	fw_reserve;
 	uint32_t gim_feature;
+	uint32_t reg_access_mode;
 };
 
 #define amdgpu_sriov_enabled(adev) \
@@ -295,5 +303,9 @@ int amdgpu_virt_fw_reserve_get_checksum(void *obj, unsigned long obj_size,
 					unsigned int key,
 					unsigned int chksum);
 void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev);
+void amdgpu_virt_init_reg_access_mode(struct amdgpu_device *adev);
+bool amdgpu_virt_support_psp_prg_ih_reg(struct amdgpu_device *adev);
+bool amdgpu_virt_support_rlc_prg_reg(struct amdgpu_device *adev);
+bool amdgpu_virt_support_skip_setting(struct amdgpu_device *adev);
 
 #endif
drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c +19 -0
index 73851eb..f548e22 100644
@@ -26,6 +26,7 @@
 #include "nbio/nbio_6_1_sh_mask.h"
 #include "gc/gc_9_0_offset.h"
 #include "gc/gc_9_0_sh_mask.h"
+#include "mp/mp_9_0_offset.h"
 #include "soc15.h"
 #include "vega10_ih.h"
 #include "soc15_common.h"
@@ -369,10 +370,28 @@ void xgpu_ai_mailbox_put_irq(struct amdgpu_device *adev)
 	amdgpu_irq_put(adev, &adev->virt.rcv_irq, 0);
 }
 
+static void xgpu_ai_init_reg_access_mode(struct amdgpu_device *adev)
+{
+	uint32_t rlc_fw_ver = RREG32_SOC15(GC, 0, mmRLC_GPM_GENERAL_6);
+	uint32_t sos_fw_ver = RREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_58);
+
+	adev->virt.reg_access_mode = AMDGPU_VIRT_REG_ACCESS_LEGACY;
+
+	if (rlc_fw_ver >= 0x5d)
+		adev->virt.reg_access_mode |= AMDGPU_VIRT_REG_ACCESS_RLC;
+
+	if (sos_fw_ver >= 0x80455)
+		adev->virt.reg_access_mode |= AMDGPU_VIRT_REG_ACCESS_PSP_PRG_IH;
+
+	if (sos_fw_ver >= 0x8045b)
+		adev->virt.reg_access_mode |= AMDGPU_VIRT_REG_SKIP_SEETING;
+}
+
 const struct amdgpu_virt_ops xgpu_ai_virt_ops = {
 	.req_full_gpu	= xgpu_ai_request_full_gpu_access,
 	.rel_full_gpu	= xgpu_ai_release_full_gpu_access,
 	.reset_gpu = xgpu_ai_request_reset,
 	.wait_reset = NULL,
 	.trans_msg = xgpu_ai_mailbox_trans_msg,
+	.init_reg_access_mode = xgpu_ai_init_reg_access_mode,
 };
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#3 lore raw

PATCH 2/9 drm/amdgpu: initialize PSP before IH under SR-IOV

In reply to a message not in the archive

In order to support new PSP feature that PSP may provide interface to program IH CNTL register, initialize PSP before IH under Vega10 SR-IOV VF

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  1 +
 drivers/gpu/drm/amd/amdgpu/soc15.c         | 24 ++++++++++++++++++------
 2 files changed, 19 insertions(+), 6 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
index 615e775..dacd128 100644
@@ -1580,6 +1580,7 @@ static int amdgpu_device_ip_hw_init_phase1(struct amdgpu_device *adev)
 		if (adev->ip_blocks[i].status.hw)
 			continue;
 		if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_COMMON ||
+		    (amdgpu_sriov_vf(adev) && (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_PSP)) ||
 		    adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_IH) {
 			r = adev->ip_blocks[i].version->funcs->hw_init(adev);
 			if (r) {
drivers/gpu/drm/amd/amdgpu/soc15.c +18 -6
index 77493a0f..9150e93 100644
@@ -597,12 +597,24 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev)
 	case CHIP_VEGA20:
 		amdgpu_device_ip_block_add(adev, &vega10_common_ip_block);
 		amdgpu_device_ip_block_add(adev, &gmc_v9_0_ip_block);
-		amdgpu_device_ip_block_add(adev, &vega10_ih_ip_block);
-		if (likely(adev->firmware.load_type == AMDGPU_FW_LOAD_PSP)) {
-			if (adev->asic_type == CHIP_VEGA20)
-				amdgpu_device_ip_block_add(adev, &psp_v11_0_ip_block);
-			else
-				amdgpu_device_ip_block_add(adev, &psp_v3_1_ip_block);
+
+		/* For Vega10 SR-IOV, PSP need to be initialized before IH */
+		if (amdgpu_sriov_vf(adev)) {
+			if (likely(adev->firmware.load_type == AMDGPU_FW_LOAD_PSP)) {
+				if (adev->asic_type == CHIP_VEGA20)
+					amdgpu_device_ip_block_add(adev, &psp_v11_0_ip_block);
+				else
+					amdgpu_device_ip_block_add(adev, &psp_v3_1_ip_block);
+			}
+			amdgpu_device_ip_block_add(adev, &vega10_ih_ip_block);
+		} else {
+			amdgpu_device_ip_block_add(adev, &vega10_ih_ip_block);
+			if (likely(adev->firmware.load_type == AMDGPU_FW_LOAD_PSP)) {
+				if (adev->asic_type == CHIP_VEGA20)
+					amdgpu_device_ip_block_add(adev, &psp_v11_0_ip_block);
+				else
+					amdgpu_device_ip_block_add(adev, &psp_v3_1_ip_block);
+			}
 		}
 		amdgpu_device_ip_block_add(adev, &gfx_v9_0_ip_block);
 		amdgpu_device_ip_block_add(adev, &sdma_v4_0_ip_block);
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#4 lore raw

PATCH 3/9 drm/amdgpu: Add new PSP cmd GFX_CMD_ID_PROG_REG

In reply to a message not in the archive

Add new PSP command GFX_CMD_ID_PROG_REG definition

Change-Id I685baa2a219cac60417c2aa609cd3d6b9ff2b0cf
---
 drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h +8 -0
index 2f79765..7f8edc6 100644
@@ -94,6 +94,7 @@ enum psp_gfx_cmd_id
     GFX_CMD_ID_SAVE_RESTORE = 0x00000008,   /* save/restore HW IP FW */
     GFX_CMD_ID_SETUP_VMR    = 0x00000009,   /* setup VMR region */
     GFX_CMD_ID_DESTROY_VMR  = 0x0000000A,   /* destroy VMR region */
+    GFX_CMD_ID_PROG_REG     = 0x0000000B,   /* program regs */
 };
 
 
@@ -217,6 +218,12 @@ struct psp_gfx_cmd_save_restore_ip_fw
     enum psp_gfx_fw_type    fw_type;              /* FW type */
 };
 
+/* Command to setup register program */
+struct psp_gfx_cmd_reg_prog {
+	uint32_t	reg_value;
+	uint32_t	reg_id;
+};
+
 /* All GFX ring buffer commands. */
 union psp_gfx_commands
 {
@@ -226,6 +233,7 @@ union psp_gfx_commands
     struct psp_gfx_cmd_setup_tmr        cmd_setup_tmr;
     struct psp_gfx_cmd_load_ip_fw       cmd_load_ip_fw;
     struct psp_gfx_cmd_save_restore_ip_fw cmd_save_restore_ip_fw;
+    struct psp_gfx_cmd_reg_prog       cmd_setup_reg_prog;
 };
 
 
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#5 lore raw

PATCH 4/9 drm/amdgpu: implement PSP cmd GFX_CMD_ID_PROG_REG

In reply to a message not in the archive

Add implementation to program regs by PSP, currently the following IH registers are supported:
IH_RB_CNTL
IH_RB_CNTL_RING1
IH_RB_CNTL_RING2

Change-Id I8e777f1080043066843d3962d3635e7075ecf21b
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 28 ++++++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 11 ++++++++++-
 2 files changed, 38 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +28 -0
index 32388b5..b3bc0f8 100644
@@ -286,6 +286,34 @@ static int psp_asd_load(struct psp_context *psp)
 	return ret;
 }
 
+static void psp_prep_reg_prog_cmd_buf(struct psp_gfx_cmd_resp *cmd,
+		uint32_t id, uint32_t value)
+{
+	cmd->cmd_id = GFX_CMD_ID_PROG_REG;
+	cmd->cmd.cmd_setup_reg_prog.reg_value = value;
+	cmd->cmd.cmd_setup_reg_prog.reg_id = id;
+}
+
+int psp_reg_program(struct psp_context *psp, enum psp_reg_prog_id reg,
+		uint32_t value)
+{
+	struct psp_gfx_cmd_resp *cmd = NULL;
+	int ret = 0;
+
+	if (reg >= PSP_REG_LAST)
+		return -EINVAL;
+
+	cmd = kzalloc(sizeof(struct psp_gfx_cmd_resp), GFP_KERNEL);
+	if (!cmd)
+		return -ENOMEM;
+
+	psp_prep_reg_prog_cmd_buf(cmd, reg, value);
+	ret = psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr);
+
+	kfree(cmd);
+	return ret;
+}
+
 static void psp_prep_xgmi_ta_load_cmd_buf(struct psp_gfx_cmd_resp *cmd,
 					  uint64_t xgmi_ta_mc, uint64_t xgmi_mc_shared,
 					  uint32_t xgmi_ta_size, uint32_t shared_size)
drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h +10 -1
index cde113f..23d90b2 100644
@@ -62,6 +62,14 @@ struct psp_ring
 	uint32_t			ring_size;
 };
 
+/* More registers may will be supported */
+enum psp_reg_prog_id {
+	PSP_REG_IH_RB_CNTL        = 0,  /* register IH_RB_CNTL */
+	PSP_REG_IH_RB_CNTL_RING1  = 1,  /* register IH_RB_CNTL_RING1 */
+	PSP_REG_IH_RB_CNTL_RING2  = 2,  /* register IH_RB_CNTL_RING2 */
+	PSP_REG_LAST
+};
+
 struct psp_funcs
 {
 	int (*init_microcode)(struct psp_context *psp);
@@ -250,5 +258,6 @@ int psp_ras_enable_features(struct psp_context *psp,
 		union ta_ras_cmd_input *info, bool enable);
 
 extern const struct amdgpu_ip_block_version psp_v11_0_ip_block;
-
+int psp_reg_program(struct psp_context *psp, enum psp_reg_prog_id reg,
+		uint32_t value);
 #endif
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#6 lore raw

PATCH 5/9 drm/amdgpu: call psp to progrm ih cntl in SR-IOV

In reply to a message not in the archive

call psp to progrm ih cntl in SR-IOV if supported

Change-Id I466dd66926221e764cbcddca48b1f0fe5cd798b4
---
 drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 91 ++++++++++++++++++++++++++++++----
 1 file changed, 82 insertions(+), 9 deletions(-)
drivers/gpu/drm/amd/amdgpu/vega10_ih.c +82 -9
index 1b2f69a..fbb1ed8 100644
@@ -48,14 +48,29 @@ static void vega10_ih_enable_interrupts(struct amdgpu_device *adev)
 
 	ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, RB_ENABLE, 1);
 	ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, ENABLE_INTR, 1);
-	WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+	if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+		if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL, ih_rb_cntl)) {
+			DRM_ERROR("PSP program IH_RB_CNTL failed!\n");
+			return;
+		}
+	} else {
+		WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+	}
 	adev->irq.ih.enabled = true;
 
 	if (adev->irq.ih1.ring_size) {
 		ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1);
 		ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL_RING1,
 					   RB_ENABLE, 1);
-		WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+		if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+			if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING1,
+						ih_rb_cntl)) {
+				DRM_ERROR("program IH_RB_CNTL_RING1 failed!\n");
+				return;
+			}
+		} else {
+			WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+		}
 		adev->irq.ih1.enabled = true;
 	}
 
@@ -63,7 +78,15 @@ static void vega10_ih_enable_interrupts(struct amdgpu_device *adev)
 		ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2);
 		ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL_RING2,
 					   RB_ENABLE, 1);
-		WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+		if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+			if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING2,
+						ih_rb_cntl)) {
+				DRM_ERROR("program IH_RB_CNTL_RING2 failed!\n");
+				return;
+			}
+		} else {
+			WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+		}
 		adev->irq.ih2.enabled = true;
 	}
 }
@@ -81,7 +104,15 @@ static void vega10_ih_disable_interrupts(struct amdgpu_device *adev)
 
 	ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, RB_ENABLE, 0);
 	ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, ENABLE_INTR, 0);
-	WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+	if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+		if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL, ih_rb_cntl)) {
+			DRM_ERROR("PSP program IH_RB_CNTL failed!\n");
+			return;
+		}
+	} else {
+		WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+	}
+
 	/* set rptr, wptr to 0 */
 	WREG32_SOC15(OSSSYS, 0, mmIH_RB_RPTR, 0);
 	WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR, 0);
@@ -92,7 +123,15 @@ static void vega10_ih_disable_interrupts(struct amdgpu_device *adev)
 		ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1);
 		ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL_RING1,
 					   RB_ENABLE, 0);
-		WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+		if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+			if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING1,
+						ih_rb_cntl)) {
+				DRM_ERROR("program IH_RB_CNTL_RING1 failed!\n");
+				return;
+			}
+		} else {
+			WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+		}
 		/* set rptr, wptr to 0 */
 		WREG32_SOC15(OSSSYS, 0, mmIH_RB_RPTR_RING1, 0);
 		WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_RING1, 0);
@@ -104,7 +143,16 @@ static void vega10_ih_disable_interrupts(struct amdgpu_device *adev)
 		ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2);
 		ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL_RING2,
 					   RB_ENABLE, 0);
-		WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+		if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+			if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING2,
+						ih_rb_cntl)) {
+				DRM_ERROR("program IH_RB_CNTL_RING2 failed!\n");
+				return;
+			}
+		} else {
+			WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+		}
+
 		/* set rptr, wptr to 0 */
 		WREG32_SOC15(OSSSYS, 0, mmIH_RB_RPTR_RING2, 0);
 		WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_RING2, 0);
@@ -187,7 +235,15 @@ static int vega10_ih_irq_init(struct amdgpu_device *adev)
 	ih_rb_cntl = vega10_ih_rb_cntl(ih, ih_rb_cntl);
 	ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, RPTR_REARM,
 				   !!adev->irq.msi_enabled);
-	WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+
+	if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+		if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL, ih_rb_cntl)) {
+			DRM_ERROR("PSP program IH_RB_CNTL failed!\n");
+			return -ETIMEDOUT;
+		}
+	} else {
+		WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+	}
 
 	/* set the writeback address whether it's enabled or not */
 	WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_ADDR_LO,
@@ -214,7 +270,15 @@ static int vega10_ih_irq_init(struct amdgpu_device *adev)
 					   WPTR_OVERFLOW_ENABLE, 0);
 		ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL,
 					   RB_FULL_DRAIN_ENABLE, 1);
-		WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+		if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+			if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING1,
+						ih_rb_cntl)) {
+				DRM_ERROR("program IH_RB_CNTL_RING1 failed!\n");
+				return -ETIMEDOUT;
+			}
+		} else {
+			WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+		}
 
 		/* set rptr, wptr to 0 */
 		WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_RING1, 0);
@@ -232,7 +296,16 @@ static int vega10_ih_irq_init(struct amdgpu_device *adev)
 
 		ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2);
 		ih_rb_cntl = vega10_ih_rb_cntl(ih, ih_rb_cntl);
-		WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+
+		if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+			if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING2,
+						ih_rb_cntl)) {
+				DRM_ERROR("program IH_RB_CNTL_RING2 failed!\n");
+				return -ETIMEDOUT;
+			}
+		} else {
+			WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+		}
 
 		/* set rptr, wptr to 0 */
 		WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_RING2, 0);
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1557283541-7877-6-git-send-email-Trigger.Huang-5C7GfCeVMHo@public.gmane.org

Reviewed-by
#8 lore raw

Re: [PATCH 5/9] drm/amdgpu: call psp to progrm ih cntl in SR-IOV

Quoted 2 lines · Trigger Huang

call psp to progrm ih cntl in SR-IOV if supported

typo in subject and description:
progrm -> program
With that fixed:

Quoted 167 lines
Change-Id I466dd66926221e764cbcddca48b1f0fe5cd798b4
---
 drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 91 ++++++++++++++++++++++++++++++----
 1 file changed, 82 insertions(+), 9 deletions(-)
drivers/gpu/drm/amd/amdgpu/vega10_ih.c +82 -10
index 1b2f69a..fbb1ed8 100644
@@ -48,14 +48,29 @@ static void vega10_ih_enable_interrupts(struct amdgpu_device *adev)
 
        ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, RB_ENABLE, 1);
        ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, ENABLE_INTR, 1);
-       WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+       if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+               if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL, ih_rb_cntl)) {
+                       DRM_ERROR("PSP program IH_RB_CNTL failed!\n");
+                       return;
+               }
+       } else {
+               WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+       }
        adev->irq.ih.enabled = true;
 
        if (adev->irq.ih1.ring_size) {
                ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1);
                ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL_RING1,
                                           RB_ENABLE, 1);
-               WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+               if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+                       if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING1,
+                                               ih_rb_cntl)) {
+                               DRM_ERROR("program IH_RB_CNTL_RING1 failed!\n");
+                               return;
+                       }
+               } else {
+                       WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+               }
                adev->irq.ih1.enabled = true;
        }
 
@@ -63,7 +78,15 @@ static void vega10_ih_enable_interrupts(struct amdgpu_device *adev)
                ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2);
                ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL_RING2,
                                           RB_ENABLE, 1);
-               WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+               if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+                       if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING2,
+                                               ih_rb_cntl)) {
+                               DRM_ERROR("program IH_RB_CNTL_RING2 failed!\n");
+                               return;
+                       }
+               } else {
+                       WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+               }
                adev->irq.ih2.enabled = true;
        }
 }
@@ -81,7 +104,15 @@ static void vega10_ih_disable_interrupts(struct amdgpu_device *adev)
 
        ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, RB_ENABLE, 0);
        ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, ENABLE_INTR, 0);
-       WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+       if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+               if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL, ih_rb_cntl)) {
+                       DRM_ERROR("PSP program IH_RB_CNTL failed!\n");
+                       return;
+               }
+       } else {
+               WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+       }
+
        /* set rptr, wptr to 0 */
        WREG32_SOC15(OSSSYS, 0, mmIH_RB_RPTR, 0);
        WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR, 0);
@@ -92,7 +123,15 @@ static void vega10_ih_disable_interrupts(struct amdgpu_device *adev)
                ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1);
                ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL_RING1,
                                           RB_ENABLE, 0);
-               WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+               if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+                       if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING1,
+                                               ih_rb_cntl)) {
+                               DRM_ERROR("program IH_RB_CNTL_RING1 failed!\n");
+                               return;
+                       }
+               } else {
+                       WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+               }
                /* set rptr, wptr to 0 */
                WREG32_SOC15(OSSSYS, 0, mmIH_RB_RPTR_RING1, 0);
                WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_RING1, 0);
@@ -104,7 +143,16 @@ static void vega10_ih_disable_interrupts(struct amdgpu_device *adev)
                ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2);
                ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL_RING2,
                                           RB_ENABLE, 0);
-               WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+               if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+                       if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING2,
+                                               ih_rb_cntl)) {
+                               DRM_ERROR("program IH_RB_CNTL_RING2 failed!\n");
+                               return;
+                       }
+               } else {
+                       WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+               }
+
                /* set rptr, wptr to 0 */
                WREG32_SOC15(OSSSYS, 0, mmIH_RB_RPTR_RING2, 0);
                WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_RING2, 0);
@@ -187,7 +235,15 @@ static int vega10_ih_irq_init(struct amdgpu_device *adev)
        ih_rb_cntl = vega10_ih_rb_cntl(ih, ih_rb_cntl);
        ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL, RPTR_REARM,
                                   !!adev->irq.msi_enabled);
-       WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+
+       if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+               if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL, ih_rb_cntl)) {
+                       DRM_ERROR("PSP program IH_RB_CNTL failed!\n");
+                       return -ETIMEDOUT;
+               }
+       } else {
+               WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL, ih_rb_cntl);
+       }
 
        /* set the writeback address whether it's enabled or not */
        WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_ADDR_LO,
@@ -214,7 +270,15 @@ static int vega10_ih_irq_init(struct amdgpu_device *adev)
                                           WPTR_OVERFLOW_ENABLE, 0);
                ih_rb_cntl = REG_SET_FIELD(ih_rb_cntl, IH_RB_CNTL,
                                           RB_FULL_DRAIN_ENABLE, 1);
-               WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+               if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+                       if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING1,
+                                               ih_rb_cntl)) {
+                               DRM_ERROR("program IH_RB_CNTL_RING1 failed!\n");
+                               return -ETIMEDOUT;
+                       }
+               } else {
+                       WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING1, ih_rb_cntl);
+               }
 
                /* set rptr, wptr to 0 */
                WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_RING1, 0);
@@ -232,7 +296,16 @@ static int vega10_ih_irq_init(struct amdgpu_device *adev)
 
                ih_rb_cntl = RREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2);
                ih_rb_cntl = vega10_ih_rb_cntl(ih, ih_rb_cntl);
-               WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+
+               if (amdgpu_virt_support_psp_prg_ih_reg(adev)) {
+                       if (psp_reg_program(&adev->psp, PSP_REG_IH_RB_CNTL_RING2,
+                                               ih_rb_cntl)) {
+                               DRM_ERROR("program IH_RB_CNTL_RING2 failed!\n");
+                               return -ETIMEDOUT;
+                       }
+               } else {
+                       WREG32_SOC15(OSSSYS, 0, mmIH_RB_CNTL_RING2, ih_rb_cntl);
+               }
 
                /* set rptr, wptr to 0 */
                WREG32_SOC15(OSSSYS, 0, mmIH_RB_WPTR_RING2, 0);
--

2.7.4

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#9 lore raw

PATCH 6/9 drm/amdgpu: Support PSP VMR ring for Vega10 VF

In reply to a message not in the archive

Add VMR ring support for Vega10 SR-IOV VF if PSP supported

Change-Id I1990e4c9babdac95d9797e7870569c1c6f630585
---
 drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 131 +++++++++++++++++++++++++---------
 1 file changed, 99 insertions(+), 32 deletions(-)
drivers/gpu/drm/amd/amdgpu/psp_v3_1.c +99 -32
index 143f0fa..3f58277 100644
@@ -50,6 +50,10 @@ MODULE_FIRMWARE("amdgpu/vega12_asd.bin");
 
 static uint32_t sos_old_versions[] = {1517616, 1510592, 1448594, 1446554};
 
+static bool psp_v3_1_support_vmr_ring(struct psp_context *psp);
+static int psp_v3_1_ring_stop(struct psp_context *psp,
+			      enum psp_ring_type ring_type);
+
 static int psp_v3_1_init_microcode(struct psp_context *psp)
 {
 	struct amdgpu_device *adev = psp->adev;
@@ -296,27 +300,57 @@ static int psp_v3_1_ring_create(struct psp_context *psp,
 
 	psp_v3_1_reroute_ih(psp);
 
-	/* Write low address of the ring to C2PMSG_69 */
-	psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr);
-	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_69, psp_ring_reg);
-	/* Write high address of the ring to C2PMSG_70 */
-	psp_ring_reg = upper_32_bits(ring->ring_mem_mc_addr);
-	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_70, psp_ring_reg);
-	/* Write size of ring to C2PMSG_71 */
-	psp_ring_reg = ring->ring_size;
-	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_71, psp_ring_reg);
-	/* Write the ring initialization command to C2PMSG_64 */
-	psp_ring_reg = ring_type;
-	psp_ring_reg = psp_ring_reg << 16;
-	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_64, psp_ring_reg);
-
-	/* there might be handshake issue with hardware which needs delay */
-	mdelay(20);
-
-	/* Wait for response flag (bit 31) in C2PMSG_64 */
-	ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, mmMP0_SMN_C2PMSG_64),
-			   0x80000000, 0x8000FFFF, false);
+	if (psp_v3_1_support_vmr_ring(psp)) {
+		ret = psp_v3_1_ring_stop(psp, ring_type);
+		if (ret) {
+			DRM_ERROR("psp_v3_1_ring_stop_sriov failed!\n");
+			return ret;
+		}
+
+		/* Write low address of the ring to C2PMSG_102 */
+		psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr);
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_102, psp_ring_reg);
+		/* Write high address of the ring to C2PMSG_103 */
+		psp_ring_reg = upper_32_bits(ring->ring_mem_mc_addr);
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_103, psp_ring_reg);
+		/* No size initialization for sriov  */
+		/* Write the ring initialization command to C2PMSG_101 */
+		psp_ring_reg = ring_type;
+		psp_ring_reg = psp_ring_reg << 16;
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_101, psp_ring_reg);
+
+		/* there might be hardware handshake issue which needs delay */
+		mdelay(20);
+
+		/* Wait for response flag (bit 31) in C2PMSG_101 */
+		ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0,
+					mmMP0_SMN_C2PMSG_101), 0x80000000,
+					0x8000FFFF, false);
+	} else {
+
+		/* Write low address of the ring to C2PMSG_69 */
+		psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr);
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_69, psp_ring_reg);
+		/* Write high address of the ring to C2PMSG_70 */
+		psp_ring_reg = upper_32_bits(ring->ring_mem_mc_addr);
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_70, psp_ring_reg);
+		/* Write size of ring to C2PMSG_71 */
+		psp_ring_reg = ring->ring_size;
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_71, psp_ring_reg);
+		/* Write the ring initialization command to C2PMSG_64 */
+		psp_ring_reg = ring_type;
+		psp_ring_reg = psp_ring_reg << 16;
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_64, psp_ring_reg);
+
+		/* there might be hardware handshake issue which needs delay */
+		mdelay(20);
+
+		/* Wait for response flag (bit 31) in C2PMSG_64 */
+		ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0,
+					mmMP0_SMN_C2PMSG_64), 0x80000000,
+					0x8000FFFF, false);
 
+	}
 	return ret;
 }
 
@@ -327,16 +361,31 @@ static int psp_v3_1_ring_stop(struct psp_context *psp,
 	unsigned int psp_ring_reg = 0;
 	struct amdgpu_device *adev = psp->adev;
 
-	/* Write the ring destroy command to C2PMSG_64 */
-	psp_ring_reg = 3 << 16;
-	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_64, psp_ring_reg);
-
-	/* there might be handshake issue with hardware which needs delay */
-	mdelay(20);
-
-	/* Wait for response flag (bit 31) in C2PMSG_64 */
-	ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, mmMP0_SMN_C2PMSG_64),
-			   0x80000000, 0x80000000, false);
+	if (psp_v3_1_support_vmr_ring(psp)) {
+		/* Write the Destroy GPCOM ring command to C2PMSG_101 */
+		psp_ring_reg = GFX_CTRL_CMD_ID_DESTROY_GPCOM_RING;
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_101, psp_ring_reg);
+
+		/* there might be handshake issue which needs delay */
+		mdelay(20);
+
+		/* Wait for response flag (bit 31) in C2PMSG_101 */
+		ret = psp_wait_for(psp,
+				SOC15_REG_OFFSET(MP0, 0, mmMP0_SMN_C2PMSG_101),
+				0x80000000, 0x80000000, false);
+	} else {
+		/* Write the ring destroy command to C2PMSG_64 */
+		psp_ring_reg = 3 << 16;
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_64, psp_ring_reg);
+
+		/* there might be handshake issue which needs delay */
+		mdelay(20);
+
+		/* Wait for response flag (bit 31) in C2PMSG_64 */
+		ret = psp_wait_for(psp,
+				SOC15_REG_OFFSET(MP0, 0, mmMP0_SMN_C2PMSG_64),
+				0x80000000, 0x80000000, false);
+	}
 
 	return ret;
 }
@@ -375,7 +424,10 @@ static int psp_v3_1_cmd_submit(struct psp_context *psp,
 	uint32_t rb_frame_size_dw = sizeof(struct psp_gfx_rb_frame) / 4;
 
 	/* KM (GPCOM) prepare write pointer */
-	psp_write_ptr_reg = RREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_67);
+	if (psp_v3_1_support_vmr_ring(psp))
+		psp_write_ptr_reg = RREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_102);
+	else
+		psp_write_ptr_reg = RREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_67);
 
 	/* Update KM RB frame pointer to new frame */
 	/* write_frame ptr increments by size of rb_frame in bytes */
@@ -404,7 +456,13 @@ static int psp_v3_1_cmd_submit(struct psp_context *psp,
 
 	/* Update the write Pointer in DWORDs */
 	psp_write_ptr_reg = (psp_write_ptr_reg + rb_frame_size_dw) % ring_size_dw;
-	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_67, psp_write_ptr_reg);
+	if (psp_v3_1_support_vmr_ring(psp)) {
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_102, psp_write_ptr_reg);
+		/* send interrupt to PSP for SRIOV ring write pointer update */
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_101,
+					GFX_CTRL_CMD_ID_CONSUME_CMD);
+	} else
+		WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_67, psp_write_ptr_reg);
 
 	return 0;
 }
@@ -574,6 +632,14 @@ static int psp_v3_1_mode1_reset(struct psp_context *psp)
 	return 0;
 }
 
+static bool psp_v3_1_support_vmr_ring(struct psp_context *psp)
+{
+	if (amdgpu_sriov_vf(psp->adev) && psp->sos_fw_version >= 0x80455)
+		return true;
+
+	return false;
+}
+
 static const struct psp_funcs psp_v3_1_funcs = {
 	.init_microcode = psp_v3_1_init_microcode,
 	.bootloader_load_sysdrv = psp_v3_1_bootloader_load_sysdrv,
@@ -586,6 +652,7 @@ static const struct psp_funcs psp_v3_1_funcs = {
 	.compare_sram_data = psp_v3_1_compare_sram_data,
 	.smu_reload_quirk = psp_v3_1_smu_reload_quirk,
 	.mode1_reset = psp_v3_1_mode1_reset,
+	.support_vmr_ring = psp_v3_1_support_vmr_ring,
 };
 
 void psp_v3_1_set_psp_funcs(struct psp_context *psp)
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#10 lore raw

PATCH 7/9 drm/amdgpu: Skip setting some regs under Vega10 VF

In reply to a message not in the archive

For Vega10 SR-IOV VF, skip setting some regs due to:
1, host will program thme
2, avoid VF register programming violations

Change-Id Id43e7fca7775035be47696c67a74ad418403036b
---
 drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c   | 14 ++++++++------
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c   |  3 +++
 drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c | 25 ++++++++++++++++++++-----
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c  | 14 ++++++++------
 drivers/gpu/drm/amd/amdgpu/soc15.c      | 16 +++++++++++-----
 5 files changed, 50 insertions(+), 22 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +8 -6
index ef4272d..6b203c9 100644
@@ -307,12 +307,14 @@ static void gfx_v9_0_init_golden_registers(struct amdgpu_device *adev)
 {
 	switch (adev->asic_type) {
 	case CHIP_VEGA10:
-		soc15_program_register_sequence(adev,
-						 golden_settings_gc_9_0,
-						 ARRAY_SIZE(golden_settings_gc_9_0));
-		soc15_program_register_sequence(adev,
-						 golden_settings_gc_9_0_vg10,
-						 ARRAY_SIZE(golden_settings_gc_9_0_vg10));
+		if (!amdgpu_virt_support_skip_setting(adev)) {
+			soc15_program_register_sequence(adev,
+							 golden_settings_gc_9_0,
+							 ARRAY_SIZE(golden_settings_gc_9_0));
+			soc15_program_register_sequence(adev,
+							 golden_settings_gc_9_0_vg10,
+							 ARRAY_SIZE(golden_settings_gc_9_0_vg10));
+		}
 		break;
 	case CHIP_VEGA12:
 		soc15_program_register_sequence(adev,
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -0
index 727e26a..b41574e 100644
@@ -1087,6 +1087,9 @@ static void gmc_v9_0_init_golden_registers(struct amdgpu_device *adev)
 
 	switch (adev->asic_type) {
 	case CHIP_VEGA10:
+		if (amdgpu_virt_support_skip_setting(adev))
+			break;
+		/* fall through */
 	case CHIP_VEGA20:
 		soc15_program_register_sequence(adev,
 						golden_settings_mmhub_1_0_0,
drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c +20 -5
index 1741056..8054131 100644
@@ -111,6 +111,9 @@ static void mmhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
 		WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
 			     max(adev->gmc.fb_end, adev->gmc.agp_end) >> 18);
 
+	if (amdgpu_virt_support_skip_setting(adev))
+		return;
+
 	/* Set default page address. */
 	value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +
 		adev->vm_manager.vram_base_offset;
@@ -156,6 +159,9 @@ static void mmhub_v1_0_init_cache_regs(struct amdgpu_device *adev)
 {
 	uint32_t tmp;
 
+	if (amdgpu_virt_support_skip_setting(adev))
+		return;
+
 	/* Setup L2 cache */
 	tmp = RREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL);
 	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_CACHE, 1);
@@ -201,6 +207,9 @@ static void mmhub_v1_0_enable_system_domain(struct amdgpu_device *adev)
 
 static void mmhub_v1_0_disable_identity_aperture(struct amdgpu_device *adev)
 {
+	if (amdgpu_virt_support_skip_setting(adev))
+		return;
+
 	WREG32_SOC15(MMHUB, 0, mmVM_L2_CONTEXT1_IDENTITY_APERTURE_LOW_ADDR_LO32,
 		     0XFFFFFFFF);
 	WREG32_SOC15(MMHUB, 0, mmVM_L2_CONTEXT1_IDENTITY_APERTURE_LOW_ADDR_HI32,
@@ -337,11 +346,13 @@ void mmhub_v1_0_gart_disable(struct amdgpu_device *adev)
 				0);
 	WREG32_SOC15(MMHUB, 0, mmMC_VM_MX_L1_TLB_CNTL, tmp);
 
-	/* Setup L2 cache */
-	tmp = RREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL);
-	tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_CACHE, 0);
-	WREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL, tmp);
-	WREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL3, 0);
+	if (!amdgpu_virt_support_skip_setting(adev)) {
+		/* Setup L2 cache */
+		tmp = RREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL);
+		tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_CACHE, 0);
+		WREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL, tmp);
+		WREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL3, 0);
+	}
 }
 
 /**
@@ -353,6 +364,10 @@ void mmhub_v1_0_gart_disable(struct amdgpu_device *adev)
 void mmhub_v1_0_set_fault_enable_default(struct amdgpu_device *adev, bool value)
 {
 	u32 tmp;
+
+	if (amdgpu_virt_support_skip_setting(adev))
+		return;
+
 	tmp = RREG32_SOC15(MMHUB, 0, mmVM_L2_PROTECTION_FAULT_CNTL);
 	tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
 			RANGE_PROTECTION_FAULT_ENABLE_DEFAULT, value);
drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c +8 -6
index 8691b62..233b7f4 100644
@@ -212,12 +212,14 @@ static void sdma_v4_0_init_golden_registers(struct amdgpu_device *adev)
 {
 	switch (adev->asic_type) {
 	case CHIP_VEGA10:
-		soc15_program_register_sequence(adev,
-						 golden_settings_sdma_4,
-						 ARRAY_SIZE(golden_settings_sdma_4));
-		soc15_program_register_sequence(adev,
-						 golden_settings_sdma_vg10,
-						 ARRAY_SIZE(golden_settings_sdma_vg10));
+		if (!amdgpu_virt_support_skip_setting(adev)) {
+			soc15_program_register_sequence(adev,
+							 golden_settings_sdma_4,
+							 ARRAY_SIZE(golden_settings_sdma_4));
+			soc15_program_register_sequence(adev,
+							 golden_settings_sdma_vg10,
+							 ARRAY_SIZE(golden_settings_sdma_vg10));
+		}
 		break;
 	case CHIP_VEGA12:
 		soc15_program_register_sequence(adev,
drivers/gpu/drm/amd/amdgpu/soc15.c +11 -5
index 9150e93..4b7dcaa 100644
@@ -995,11 +995,17 @@ static void soc15_doorbell_range_init(struct amdgpu_device *adev)
 	int i;
 	struct amdgpu_ring *ring;
 
-	for (i = 0; i < adev->sdma.num_instances; i++) {
-		ring = &adev->sdma.instance[i].ring;
-		adev->nbio_funcs->sdma_doorbell_range(adev, i,
-			ring->use_doorbell, ring->doorbell_index,
-			adev->doorbell_index.sdma_doorbell_range);
+	/*  Two reasons to skip
+	*		1, Host driver already programmed them
+	*		2, To avoid registers program violations in SR-IOV
+	*/
+	if (!amdgpu_virt_support_skip_setting(adev)) {
+		for (i = 0; i < adev->sdma.num_instances; i++) {
+			ring = &adev->sdma.instance[i].ring;
+			adev->nbio_funcs->sdma_doorbell_range(adev, i,
+				ring->use_doorbell, ring->doorbell_index,
+				adev->doorbell_index.sdma_doorbell_range);
+		}
 	}
 
 	adev->nbio_funcs->ih_doorbell_range(adev, adev->irq.ih.use_doorbell,
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1557283541-7877-8-git-send-email-Trigger.Huang-5C7GfCeVMHo@public.gmane.org

Reviewed-by
#12 lore raw

Re: [PATCH 7/9] drm/amdgpu: Skip setting some regs under Vega10 VF

Quoted 3 lines · Trigger Huang

For Vega10 SR-IOV VF, skip setting some regs due to:
1, host will program thme

Typo: thme -> them

With that fixed:

Quoted 170 lines

2, avoid VF register programming violations

Change-Id Id43e7fca7775035be47696c67a74ad418403036b
---
 drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c   | 14 ++++++++------
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c   |  3 +++
 drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c | 25 ++++++++++++++++++++-----
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c  | 14 ++++++++------
 drivers/gpu/drm/amd/amdgpu/soc15.c      | 16 +++++++++++-----
 5 files changed, 50 insertions(+), 22 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +8 -6
index ef4272d..6b203c9 100644
@@ -307,12 +307,14 @@ static void gfx_v9_0_init_golden_registers(struct amdgpu_device *adev)
 {
        switch (adev->asic_type) {
        case CHIP_VEGA10:
-               soc15_program_register_sequence(adev,
-                                                golden_settings_gc_9_0,
-                                                ARRAY_SIZE(golden_settings_gc_9_0));
-               soc15_program_register_sequence(adev,
-                                                golden_settings_gc_9_0_vg10,
-                                                ARRAY_SIZE(golden_settings_gc_9_0_vg10));
+               if (!amdgpu_virt_support_skip_setting(adev)) {
+                       soc15_program_register_sequence(adev,
+                                                        golden_settings_gc_9_0,
+                                                        ARRAY_SIZE(golden_settings_gc_9_0));
+                       soc15_program_register_sequence(adev,
+                                                        golden_settings_gc_9_0_vg10,
+                                                        ARRAY_SIZE(golden_settings_gc_9_0_vg10));
+               }
                break;
        case CHIP_VEGA12:
                soc15_program_register_sequence(adev,
drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +3 -0
index 727e26a..b41574e 100644
@@ -1087,6 +1087,9 @@ static void gmc_v9_0_init_golden_registers(struct amdgpu_device *adev)
 
        switch (adev->asic_type) {
        case CHIP_VEGA10:
+               if (amdgpu_virt_support_skip_setting(adev))
+                       break;
+               /* fall through */
        case CHIP_VEGA20:
                soc15_program_register_sequence(adev,
                                                golden_settings_mmhub_1_0_0,
drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c +20 -5
index 1741056..8054131 100644
@@ -111,6 +111,9 @@ static void mmhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
                WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
                             max(adev->gmc.fb_end, adev->gmc.agp_end) >> 18);
 
+       if (amdgpu_virt_support_skip_setting(adev))
+               return;
+
        /* Set default page address. */
        value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +
                adev->vm_manager.vram_base_offset;
@@ -156,6 +159,9 @@ static void mmhub_v1_0_init_cache_regs(struct amdgpu_device *adev)
 {
        uint32_t tmp;
 
+       if (amdgpu_virt_support_skip_setting(adev))
+               return;
+
        /* Setup L2 cache */
        tmp = RREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL);
        tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_CACHE, 1);
@@ -201,6 +207,9 @@ static void mmhub_v1_0_enable_system_domain(struct amdgpu_device *adev)
 
 static void mmhub_v1_0_disable_identity_aperture(struct amdgpu_device *adev)
 {
+       if (amdgpu_virt_support_skip_setting(adev))
+               return;
+
        WREG32_SOC15(MMHUB, 0, mmVM_L2_CONTEXT1_IDENTITY_APERTURE_LOW_ADDR_LO32,
                     0XFFFFFFFF);
        WREG32_SOC15(MMHUB, 0, mmVM_L2_CONTEXT1_IDENTITY_APERTURE_LOW_ADDR_HI32,
@@ -337,11 +346,13 @@ void mmhub_v1_0_gart_disable(struct amdgpu_device *adev)
                                0);
        WREG32_SOC15(MMHUB, 0, mmMC_VM_MX_L1_TLB_CNTL, tmp);
 
-       /* Setup L2 cache */
-       tmp = RREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL);
-       tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_CACHE, 0);
-       WREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL, tmp);
-       WREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL3, 0);
+       if (!amdgpu_virt_support_skip_setting(adev)) {
+               /* Setup L2 cache */
+               tmp = RREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL);
+               tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_CACHE, 0);
+               WREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL, tmp);
+               WREG32_SOC15(MMHUB, 0, mmVM_L2_CNTL3, 0);
+       }
 }
 
 /**
@@ -353,6 +364,10 @@ void mmhub_v1_0_gart_disable(struct amdgpu_device *adev)
 void mmhub_v1_0_set_fault_enable_default(struct amdgpu_device *adev, bool value)
 {
        u32 tmp;
+
+       if (amdgpu_virt_support_skip_setting(adev))
+               return;
+
        tmp = RREG32_SOC15(MMHUB, 0, mmVM_L2_PROTECTION_FAULT_CNTL);
        tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
                        RANGE_PROTECTION_FAULT_ENABLE_DEFAULT, value);
drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c +8 -6
index 8691b62..233b7f4 100644
@@ -212,12 +212,14 @@ static void sdma_v4_0_init_golden_registers(struct amdgpu_device *adev)
 {
        switch (adev->asic_type) {
        case CHIP_VEGA10:
-               soc15_program_register_sequence(adev,
-                                                golden_settings_sdma_4,
-                                                ARRAY_SIZE(golden_settings_sdma_4));
-               soc15_program_register_sequence(adev,
-                                                golden_settings_sdma_vg10,
-                                                ARRAY_SIZE(golden_settings_sdma_vg10));
+               if (!amdgpu_virt_support_skip_setting(adev)) {
+                       soc15_program_register_sequence(adev,
+                                                        golden_settings_sdma_4,
+                                                        ARRAY_SIZE(golden_settings_sdma_4));
+                       soc15_program_register_sequence(adev,
+                                                        golden_settings_sdma_vg10,
+                                                        ARRAY_SIZE(golden_settings_sdma_vg10));
+               }
                break;
        case CHIP_VEGA12:
                soc15_program_register_sequence(adev,
drivers/gpu/drm/amd/amdgpu/soc15.c +11 -6
index 9150e93..4b7dcaa 100644
@@ -995,11 +995,17 @@ static void soc15_doorbell_range_init(struct amdgpu_device *adev)
        int i;
        struct amdgpu_ring *ring;
 
-       for (i = 0; i < adev->sdma.num_instances; i++) {
-               ring = &adev->sdma.instance[i].ring;
-               adev->nbio_funcs->sdma_doorbell_range(adev, i,
-                       ring->use_doorbell, ring->doorbell_index,
-                       adev->doorbell_index.sdma_doorbell_range);
+       /*  Two reasons to skip
+       *               1, Host driver already programmed them
+       *               2, To avoid registers program violations in SR-IOV
+       */
+       if (!amdgpu_virt_support_skip_setting(adev)) {
+               for (i = 0; i < adev->sdma.num_instances; i++) {
+                       ring = &adev->sdma.instance[i].ring;
+                       adev->nbio_funcs->sdma_doorbell_range(adev, i,
+                               ring->use_doorbell, ring->doorbell_index,
+                               adev->doorbell_index.sdma_doorbell_range);
+               }
        }
 
        adev->nbio_funcs->ih_doorbell_range(adev, adev->irq.ih.use_doorbell,
--

2.7.4

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#13 lore raw

PATCH 8/9 drm/amdgpu: add basic func for RLC program reg

In reply to a message not in the archive

New feature for RLC, some registers can be programmed by RLC interface under SR-IOV VF:

WREG32_SOC15_RLC_SHADOW:
1, for GRBM_GFX_CNTL, firstly the new register value should be be
programmed to SCRATCH_REG2
1, for GRBM_GFX_INDEX, firstly the new register value should be be
programmed to SCRATCH_REG3

WREG32_RLC:
for registers supported to be programmed by RLC interface, the following sequence should be used:
1, write the value to SCRATCH_REG0
2, write reg | 0x80000000 to SCRATCH_REG1 3, write 0x1 to RLC_SPARE_INT to notify RLC 4, polling SCRATCH_REG1 to check if finished

Change-Id I5017d2cc10d475a2f8f7204b8043b3e5f870348b
---
 drivers/gpu/drm/amd/amdgpu/soc15_common.h | 57 ++++++++++++++++++++++++++++++-
 1 file changed, 56 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/soc15_common.h +56 -1
index 49c2625..c634606 100644
@@ -89,6 +89,61 @@
 			(sram_sel << UVD_DPG_LMA_CTL__SRAM_SEL__SHIFT)); \
 	} while (0)
 
-#endif
 
+#define WREG32_RLC(reg, value) \
+	do {							\
+		if (amdgpu_virt_support_rlc_prg_reg(adev)) {    \
+			uint32_t i = 0;	\
+			uint32_t retries = 50000;	\
+			uint32_t r0 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0;	\
+			uint32_t r1 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1;	\
+			uint32_t spare_int = adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT;	\
+			WREG32(r0, value);	\
+			WREG32(r1, (reg | 0x80000000));	\
+			WREG32(spare_int, 0x1);	\
+			for (i = 0; i < retries; i++) {	\
+				u32 tmp = RREG32(r1);	\
+				if (!(tmp & 0x80000000))	\
+					break;	\
+				udelay(10);	\
+			}	\
+			if (i >= retries)	\
+				pr_err("timeout: rlcg program reg:0x%05x failed !\n", reg);	\
+		} else {	\
+			WREG32(reg, value); \
+		}	\
+	} while (0)
+
+#define WREG32_SOC15_RLC_SHADOW(ip, inst, reg, value) \
+	do {							\
+		uint32_t target_reg = adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg;\
+		if (amdgpu_virt_support_rlc_prg_reg(adev)) {    \
+			uint32_t r2 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG2;	\
+			uint32_t r3 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3;	\
+			uint32_t grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL;   \
+			uint32_t grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX;   \
+			if (target_reg == grbm_cntl) \
+				WREG32(r2, value);	\
+			else if (target_reg == grbm_idx) \
+				WREG32(r3, value);	\
+			WREG32(target_reg, value);	\
+		} else {	\
+			WREG32(target_reg, value); \
+		}	\
+	} while (0)
+
+#define WREG32_SOC15_RLC(ip, inst, reg, value) \
+	do {							\
+			uint32_t target_reg = adev->reg_offset[GC_HWIP][0][reg##_BASE_IDX] + reg;\
+			WREG32_RLC(target_reg, value); \
+	} while (0)
+
+#define WREG32_FIELD15_RLC(ip, idx, reg, field, val)   \
+    WREG32_RLC((adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg), \
+    (RREG32(adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg) \
+    & ~REG_FIELD_MASK(reg, field)) | (val) << REG_FIELD_SHIFT(reg, field))
+
+#define WREG32_SOC15_OFFSET_RLC(ip, inst, reg, offset, value) \
+    WREG32_RLC(((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg) + offset), value)
 
+#endif
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#14 lore raw

PATCH 9/9 drm/amdgpu: RLC to program regs for Vega10 SR-IOV

In reply to a message not in the archive

Under Vega10 SR-IOV, with new RLC's new feature, VF should call RLC to program some registers if supported

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c |  30 +++----
 drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c             | 100 +++++++++++-----------
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c          |  20 ++---
 drivers/gpu/drm/amd/amdgpu/soc15.c                |  12 ++-
 4 files changed, 85 insertions(+), 77 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c +15 -15
index 62067d0..9cd0fd7 100644
@@ -259,8 +259,8 @@ static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid,
 
 	lock_srbm(kgd, 0, 0, 0, vmid);
 
-	WREG32(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_CONFIG), sh_mem_config);
-	WREG32(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_BASES), sh_mem_bases);
+	WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_CONFIG), sh_mem_config);
+	WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmSH_MEM_BASES), sh_mem_bases);
 	/* APE1 no longer exists on GFX9 */
 
 	unlock_srbm(kgd);
@@ -413,7 +413,7 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
 		value = RREG32(SOC15_REG_OFFSET(GC, 0, mmRLC_CP_SCHEDULERS));
 		value = REG_SET_FIELD(value, RLC_CP_SCHEDULERS, scheduler1,
 			((mec << 5) | (pipe << 3) | queue_id | 0x80));
-		WREG32(SOC15_REG_OFFSET(GC, 0, mmRLC_CP_SCHEDULERS), value);
+		WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmRLC_CP_SCHEDULERS), value);
 	}
 
 	/* HQD registers extend from CP_MQD_BASE_ADDR to CP_HQD_EOP_WPTR_MEM. */
@@ -422,13 +422,13 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
 
 	for (reg = hqd_base;
 	     reg <= SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI); reg++)
-		WREG32(reg, mqd_hqd[reg - hqd_base]);
+		WREG32_RLC(reg, mqd_hqd[reg - hqd_base]);
 
 
 	/* Activate doorbell logic before triggering WPTR poll. */
 	data = REG_SET_FIELD(m->cp_hqd_pq_doorbell_control,
 			     CP_HQD_PQ_DOORBELL_CONTROL, DOORBELL_EN, 1);
-	WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_DOORBELL_CONTROL), data);
+	WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_DOORBELL_CONTROL), data);
 
 	if (wptr) {
 		/* Don't read wptr with get_user because the user
@@ -457,25 +457,25 @@ static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id,
 		guessed_wptr += m->cp_hqd_pq_wptr_lo & ~(queue_size - 1);
 		guessed_wptr += (uint64_t)m->cp_hqd_pq_wptr_hi << 32;
 
-		WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_LO),
+		WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_LO),
 		       lower_32_bits(guessed_wptr));
-		WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI),
+		WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_HI),
 		       upper_32_bits(guessed_wptr));
-		WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_POLL_ADDR),
+		WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_POLL_ADDR),
 		       lower_32_bits((uintptr_t)wptr));
-		WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_POLL_ADDR_HI),
+		WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_PQ_WPTR_POLL_ADDR_HI),
 		       upper_32_bits((uintptr_t)wptr));
 		WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_PQ_WPTR_POLL_CNTL1),
 		       get_queue_mask(adev, pipe_id, queue_id));
 	}
 
 	/* Start the EOP fetcher */
-	WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_EOP_RPTR),
+	WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_EOP_RPTR),
 	       REG_SET_FIELD(m->cp_hqd_eop_rptr,
 			     CP_HQD_EOP_RPTR, INIT_FETCHER, 1));
 
 	data = REG_SET_FIELD(m->cp_hqd_active, CP_HQD_ACTIVE, ACTIVE, 1);
-	WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE), data);
+	WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_ACTIVE), data);
 
 	release_queue(kgd);
 
@@ -677,7 +677,7 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
 	acquire_queue(kgd, pipe_id, queue_id);
 
 	if (m->cp_hqd_vmid == 0)
-		WREG32_FIELD15(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0);
+		WREG32_FIELD15_RLC(GC, 0, RLC_CP_SCHEDULERS, scheduler1, 0);
 
 	switch (reset_type) {
 	case KFD_PREEMPT_TYPE_WAVEFRONT_DRAIN:
@@ -691,7 +691,7 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,
 		break;
 	}
 
-	WREG32(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_DEQUEUE_REQUEST), type);
+	WREG32_RLC(SOC15_REG_OFFSET(GC, 0, mmCP_HQD_DEQUEUE_REQUEST), type);
 
 	end_jiffies = (utimeout * HZ / 1000) + jiffies;
 	while (true) {
@@ -925,7 +925,7 @@ static int kgd_wave_control_execute(struct kgd_dev *kgd,
 
 	mutex_lock(&adev->grbm_idx_mutex);
 
-	WREG32(SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX), gfx_index_val);
+	WREG32_SOC15_RLC_SHADOW(GC, 0, mmGRBM_GFX_INDEX, gfx_index_val);
 	WREG32(SOC15_REG_OFFSET(GC, 0, mmSQ_CMD), sq_cmd);
 
 	data = REG_SET_FIELD(data, GRBM_GFX_INDEX,
@@ -935,7 +935,7 @@ static int kgd_wave_control_execute(struct kgd_dev *kgd,
 	data = REG_SET_FIELD(data, GRBM_GFX_INDEX,
 		SE_BROADCAST_WRITES, 1);
 
-	WREG32(SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX), data);
+	WREG32_SOC15_RLC_SHADOW(GC, 0, mmGRBM_GFX_INDEX, data);
 	mutex_unlock(&adev->grbm_idx_mutex);
 
 	return 0;
drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +50 -50

237 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +10 -10
index 7bb5359..0dc8926 100644
@@ -71,12 +71,12 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
 	uint64_t value;
 
 	/* Program the AGP BAR */
-	WREG32_SOC15(GC, 0, mmMC_VM_AGP_BASE, 0);
-	WREG32_SOC15(GC, 0, mmMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
-	WREG32_SOC15(GC, 0, mmMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
+	WREG32_SOC15_RLC(GC, 0, mmMC_VM_AGP_BASE, 0);
+	WREG32_SOC15_RLC(GC, 0, mmMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
+	WREG32_SOC15_RLC(GC, 0, mmMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
 
 	/* Program the system aperture low logical page number. */
-	WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_LOW_ADDR,
+	WREG32_SOC15_RLC(GC, 0, mmMC_VM_SYSTEM_APERTURE_LOW_ADDR,
 		     min(adev->gmc.fb_start, adev->gmc.agp_start) >> 18);
 
 	if (adev->asic_type == CHIP_RAVEN && adev->rev_id >= 0x8)
@@ -86,11 +86,11 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
 		 * workaround that increase system aperture high address (add 1)
 		 * to get rid of the VM fault and hardware hang.
 		 */
-		WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
+		WREG32_SOC15_RLC(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
 			     max((adev->gmc.fb_end >> 18) + 0x1,
 				 adev->gmc.agp_end >> 18));
 	else
-		WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
+		WREG32_SOC15_RLC(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
 			     max(adev->gmc.fb_end, adev->gmc.agp_end) >> 18);
 
 	/* Set default page address. */
@@ -129,7 +129,7 @@ static void gfxhub_v1_0_init_tlb_regs(struct amdgpu_device *adev)
 			    MTYPE, MTYPE_UC);/* XXX for emulation. */
 	tmp = REG_SET_FIELD(tmp, MC_VM_MX_L1_TLB_CNTL, ATC_EN, 1);
 
-	WREG32_SOC15(GC, 0, mmMC_VM_MX_L1_TLB_CNTL, tmp);
+	WREG32_SOC15_RLC(GC, 0, mmMC_VM_MX_L1_TLB_CNTL, tmp);
 }
 
 static void gfxhub_v1_0_init_cache_regs(struct amdgpu_device *adev)
@@ -267,9 +267,9 @@ int gfxhub_v1_0_gart_enable(struct amdgpu_device *adev)
 		 * VF copy registers so vbios post doesn't program them, for
 		 * SRIOV driver need to program them
 		 */
-		WREG32_SOC15(GC, 0, mmMC_VM_FB_LOCATION_BASE,
+		WREG32_SOC15_RLC(GC, 0, mmMC_VM_FB_LOCATION_BASE,
 			     adev->gmc.vram_start >> 24);
-		WREG32_SOC15(GC, 0, mmMC_VM_FB_LOCATION_TOP,
+		WREG32_SOC15_RLC(GC, 0, mmMC_VM_FB_LOCATION_TOP,
 			     adev->gmc.vram_end >> 24);
 	}
 
@@ -303,7 +303,7 @@ void gfxhub_v1_0_gart_disable(struct amdgpu_device *adev)
 				MC_VM_MX_L1_TLB_CNTL,
 				ENABLE_ADVANCED_DRIVER_MODEL,
 				0);
-	WREG32_SOC15(GC, 0, mmMC_VM_MX_L1_TLB_CNTL, tmp);
+	WREG32_SOC15_RLC(GC, 0, mmMC_VM_MX_L1_TLB_CNTL, tmp);
 
 	/* Setup L2 cache */
 	WREG32_FIELD15(GC, 0, VM_L2_CNTL, ENABLE_L2_CACHE, 0);
drivers/gpu/drm/amd/amdgpu/soc15.c +10 -2
index 4b7dcaa..43b74d0 100644
@@ -230,7 +230,7 @@ void soc15_grbm_select(struct amdgpu_device *adev,
 	grbm_gfx_cntl = REG_SET_FIELD(grbm_gfx_cntl, GRBM_GFX_CNTL, VMID, vmid);
 	grbm_gfx_cntl = REG_SET_FIELD(grbm_gfx_cntl, GRBM_GFX_CNTL, QUEUEID, queue);
 
-	WREG32(SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_CNTL), grbm_gfx_cntl);
+	WREG32_SOC15_RLC_SHADOW(GC, 0, mmGRBM_GFX_CNTL, grbm_gfx_cntl);
 }
 
 static void soc15_vga_set_state(struct amdgpu_device *adev, bool state)
@@ -385,7 +385,15 @@ void soc15_program_register_sequence(struct amdgpu_device *adev,
 			tmp &= ~(entry->and_mask);
 			tmp |= entry->or_mask;
 		}
-		WREG32(reg, tmp);
+
+		if (reg == SOC15_REG_OFFSET(GC, 0, mmPA_SC_BINNER_EVENT_CNTL_3) ||
+			reg == SOC15_REG_OFFSET(GC, 0, mmPA_SC_ENHANCE) ||
+			reg == SOC15_REG_OFFSET(GC, 0, mmPA_SC_ENHANCE_1) ||
+			reg == SOC15_REG_OFFSET(GC, 0, mmSH_MEM_CONFIG))
+			WREG32_RLC(reg, tmp);
+		else
+			WREG32(reg, tmp);
+
 	}
 
 }
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Reviewed-by
#15 lore raw
In reply to a message not in the archive

Minor comments on 5 and 7. Rest are:

Alex

Quoted 65 lines · Trigger Huang

To support new Vega10 SR-IOV L1 security, KMD need some modifications
1: Due to the new features supported in FW(PSP, RLC, etc),
for register access during initialization, we have more modes:
1), request PSP to program
2), request RLC to program
3), request SR-IOV host driver to program and skip
programming them in amdgpu
4), Legacy MMIO access
We will try to read the firmware version to see which mode is support

2: If PSP FW support to program some registers, such as IH,
we need:
1), initialize PSP before IH
2), send the specific command to PSP

3: Support VMR ring. VMR ring, compared with physical platform
TMR ring, the program sequence are nearly the same, but we will use another register set, mmMP0_SMN_C2PMSG_101/102/103 to communicate with PSP

4: Skip programming some registers in guest KMD
As some registers are processed by new L1 security, amdgpu on VF will fail to program them, and don’t worry, these registers will be programmed on the SR-IOV host driver side.

5: Call RLC to program some registers in instead of MMIO
With new L1 policy, some registers can’t be programmed in SR-IOV VF amdgpu with MMIO. Fortunately, new RLC firmware will support to program them with specific program sequence, which are described in the patch commit message

Trigger Huang (9):
drm/amdgpu: init vega10 SR-IOV reg access mode drm/amdgpu: initialize PSP before IH under SR-IOV drm/amdgpu: Add new PSP cmd GFX_CMD_ID_PROG_REG drm/amdgpu: implement PSP cmd GFX_CMD_ID_PROG_REG drm/amdgpu: call psp to progrm ih cntl in SR-IOV drm/amdgpu: Support PSP VMR ring for Vega10 VF drm/amdgpu: Skip setting some regs under Vega10 VF drm/amdgpu: add basic func for RLC program reg drm/amdgpu: RLC to program regs for Vega10 SR-IOV

 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c |  30 ++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c        |   4 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c           |  28 +++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h           |  11 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c          |  43 +++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h          |  12 ++
 drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c             | 114 ++++++++++---------
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c          |  20 ++--
 drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c             |   3 +
 drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c           |  25 ++++-
 drivers/gpu/drm/amd/amdgpu/mxgpu_ai.c             |  19 ++++
 drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h           |   8 ++
 drivers/gpu/drm/amd/amdgpu/psp_v3_1.c             | 131 ++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c            |  14 ++-
 drivers/gpu/drm/amd/amdgpu/soc15.c                |  52 ++++++---
 drivers/gpu/drm/amd/amdgpu/soc15_common.h         |  57 +++++++++-
 drivers/gpu/drm/amd/amdgpu/vega10_ih.c            |  91 +++++++++++++--
 17 files changed, 514 insertions(+), 148 deletions(-)
Quoted 7 lines

--
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx