Skip to contentSkip to conversation

Various SI fixes, fix Radeon HD 7870 XT (v2)

PATCH Applied Timur Kristóf 2 participants 6 messages Apr 18 – Apr 22

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

This series fixes amdgpu to work on the Radeon HD 7870 XT which has never worked with the Linux open source drivers before.

There are also various other fixes related to GART and UVD.

Changes in v2:
Consider CGTS_USER_TCC_DISABLE when reading disabled TCCs. Dropped VCE related patches, will send those in a different series.

Timur Kristóf (4):
drm/amdgpu/gmc: Fix AMDGPU_GART_PLACEMENT_LOW to not overlap with VRAM drm/amdgpu/uvd3.1: Don't validate the firmware when already validated Documentation/gpu: Add TCC, update TCP in amdgpu glossary drm/amdgpu/gfx6: Support harvested SI chips with disabled TCCs (v2)

 Documentation/gpu/amdgpu/amdgpu-glossary.rst |  9 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c      |  5 +-
 drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c        | 66 ++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c        | 10 +++
 4 files changed, 88 insertions(+), 2 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/4 drm/amdgpu/gmc: Fix AMDGPU_GART_PLACEMENT_LOW to not overlap with VRAM

When the GART placement is set to AMDGPU_GART_PLACEMENT_LOW:
Make sure that GART does not overlap with VRAM when VRAM is configured to be in the low address space.

Solve this according to the following logic:
- When GART fits before VRAM, use zero address for GART
- Otherwise, put GART after the end of VRAM, aligned to 4 GiB

Previously, I had assumed this was not possible so it was OK to not handle it, but now we got a report from a user who has a board that is configured this way.

Fixes 917f91d8d8e8 ("drm/amdgpu/gmc: add a way to force a particular placement for GART")
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +4 -1
index 290b9f9043036..f69a931e04fe9 100644
@@ -314,7 +314,10 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
 		mc->gart_start = max_mc_address - mc->gart_size + 1;
 		break;
 	case AMDGPU_GART_PLACEMENT_LOW:
-		mc->gart_start = 0;
+		if (size_bf >= mc->gart_size)
+			mc->gart_start = 0;
+		else
+			mc->gart_start = ALIGN(mc->fb_end, four_gb);
 		break;
 	case AMDGPU_GART_PLACEMENT_BEST_FIT:
 	default:
Signature · 1 line
2.53.0
#2 lore raw

PATCH 2/4 drm/amdgpu/uvd3.1: Don't validate the firmware when already validated

In reply to Timur Kristóf

UVD 3.1 firmware validation seems to always fail after attempting it when it had already been validated. (This works similarly with the VCE 1.0 as well.)

Don't attempt repeating the validation when it's already done.

This caused issues in situations when the system isn't able to suspend the GPU properly and so the GPU isn't actually powered down. Then amdgpu would fail when calling the IP block resume function.

Fixes bb7978111dd3 ("drm/amdgpu: fix SI UVD firmware validate resume fail")
---
 drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c +10 -0
index fea576a7f397f..efb3fde919ee3 100644
@@ -242,6 +242,10 @@ static void uvd_v3_1_mc_resume(struct amdgpu_device *adev)
 	uint64_t addr;
 	uint32_t size;
 
+	/* When the keyselect is already set, don't perturb it. */
+	if (RREG32(mmUVD_FW_START))
+		return;
+
 	/* program the VCPU memory controller bits 0-27 */
 	addr = (adev->uvd.inst->gpu_addr + AMDGPU_UVD_FIRMWARE_OFFSET) >> 3;
 	size = AMDGPU_UVD_FIRMWARE_SIZE(adev) >> 3;
@@ -284,6 +288,12 @@ static int uvd_v3_1_fw_validate(struct amdgpu_device *adev)
 	int i;
 	uint32_t keysel = adev->uvd.keyselect;
 
+	if (RREG32(mmUVD_FW_START) & UVD_FW_STATUS__PASS_MASK) {
+		dev_dbg(adev->dev, "UVD keyselect already set: 0x%x (on CPU: 0x%x)\n",
+			RREG32(mmUVD_FW_START), adev->uvd.keyselect);
+		return 0;
+	}
+
 	WREG32(mmUVD_FW_START, keysel);
 
 	for (i = 0; i < 10; ++i) {
Signature · 1 line
2.53.0
#3 lore raw

PATCH 3/4 Documentation/gpu: Add TCC, update TCP in amdgpu glossary

In reply to Timur Kristóf

These are the L2 and L1 cache on some AMD GPU architectures. Add them to the glossary, keeping existing alphabetical order.

---
 Documentation/gpu/amdgpu/amdgpu-glossary.rst | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
Documentation/gpu/amdgpu/amdgpu-glossary.rst +8 -1
index 033167025fcca..d553dd599c966 100644
@@ -233,8 +233,15 @@ we have a dedicated glossary for Display Core at
     TC
       Texture Cache
 
+    TCC
+      Texture Cache per Channel - L2 cache attached to the memory channels.
+      May be used when shader cores are accessing memory.
+      Despite "Texture" in the name, this is used by any kind of memory access.
+      TCCs may be mapped to TCPs, depending on the architecture.
+
     TCP (AMDGPU)
-      Texture Cache per Pipe. Even though the name "Texture" is part of this
+      Texture Cache per Pipe - L1 cache attached to each CU.
+      Even though the name "Texture" is part of this
       acronym, the TCP represents the path to memory shaders; i.e., it is not
       related to texture. The name is a leftover from older designs where shader
       stages had different cache designs; it refers to the L1 cache in older
Signature · 1 line
2.53.0
#4 lore raw

PATCH 4/4 drm/amdgpu/gfx6: Support harvested SI chips with disabled TCCs (v2)

In reply to Timur Kristóf

This commit fixes amdgpu to work on the Radeon HD 7870 XT which has never worked with the Linux open source drivers before.

Some boards have "harvested" chips, meaning that some parts of the chip are disabled and fused, and it's sold for cheaper and under a different marketing name.
On a harvested chip, any of the following can be disabled:
- CUs (Compute Units)
- RBs (Render Backend, aka. ROP)
- Memory channels (ie. the chip has a lower bandwidth)
- TCCs (ie. less L2 cache)

Handle chips with harvested TCCs by patching the registers that configure how TCCs are mapped.

If some TCCs are disabled, we need to make sure that the disabled TCCs are not used, and the remaining TCCs are used optimally.

TCP_CHAN_STEER_LO/HI control which TCC is used by TCP channels. TCP_ADDR_CONFIG.NUM_TCC_BANKS controls how many channels are used.

Note that the TCC configuration is highly relevant to performance. Suboptimal configuration (eg. CHAN_STEER=0) can significantly reduce gaming performance.

For optimal performance:
- Rely on the CHAN_STEER from the golden registers table, only skip disabled TCCs but keep the mapping order.
- Limit NUM_TCC_BANKS to number of active TCCs to avoid thrashing, which performs better than using the same TCC twice.

v2:
- Also consider CGTS_USER_TCC_DISABLE for disabled TCCs.

---
 drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c | 66 +++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)
drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c +66 -0
index 73223d97a87f5..ac90d8e9d86a8 100644
@@ -1571,6 +1571,71 @@ static void gfx_v6_0_setup_spi(struct amdgpu_device *adev)
 	mutex_unlock(&adev->grbm_idx_mutex);
 }
 
+/**
+ * gfx_v6_0_setup_tcc() - setup which TCCs are used
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Verify whether the current GPU has any TCCs disabled,
+ * which can happen when the GPU is harvested and some
+ * memory channels are disabled, reducing the memory bus width.
+ * For example, on the Radeon HD 7870 XT (Tahiti LE).
+ *
+ * If some TCCs are disabled, we need to make sure that
+ * the disabled TCCs are not used, and the remaining TCCs
+ * are used optimally.
+ *
+ * TCP_CHAN_STEER_LO/HI control which TCC is used by TCP channels.
+ * TCP_ADDR_CONFIG.NUM_TCC_BANKS controls how many channels are used.
+ *
+ * For optimal performance:
+ * - Rely on the CHAN_STEER from the golden registers table,
+ *   only skip disabled TCCs but keep the mapping order.
+ * - Limit NUM_TCC_BANKS to number of active TCCs to avoid thrashing,
+ *   which performs better than using the same TCC twice.
+ */
+static void gfx_v6_0_setup_tcc(struct amdgpu_device *adev)
+{
+	u32 i, tcc, tcp_addr_config, num_active_tcc = 0;
+	u64 chan_steer, patched_chan_steer = 0;
+	const u32 num_max_tcc = adev->gfx.config.max_texture_channel_caches;
+	const u32 dis_tcc_mask =
+		amdgpu_gfx_create_bitmask(num_max_tcc) &
+		(REG_GET_FIELD(RREG32(mmCGTS_TCC_DISABLE),
+			       CGTS_TCC_DISABLE, TCC_DISABLE) |
+		 REG_GET_FIELD(RREG32(mmCGTS_USER_TCC_DISABLE),
+			       CGTS_USER_TCC_DISABLE, TCC_DISABLE));
+
+	/* When no TCC is disabled, the golden registers table already has optimal TCC setup */
+	if (!dis_tcc_mask)
+		return;
+
+	/* Each 4-bit nibble contains the index of a TCC used by all TCPs */
+	chan_steer = RREG32(mmTCP_CHAN_STEER_LO) | ((u64)RREG32(mmTCP_CHAN_STEER_HI) << 32ull);
+
+	/* Patch the TCP to TCC mapping to skip disabled TCCs */
+	for (i = 0; i < num_max_tcc; ++i) {
+		tcc = (chan_steer >> (u64)(4 * i)) & 0xf;
+
+		if (!((1 << tcc) & dis_tcc_mask)) {
+			/* Copy enabled TCC indices to the patched register value. */
+			patched_chan_steer |= (u64)tcc << (u64)(4 * num_active_tcc);
+			++num_active_tcc;
+		}
+	}
+
+	WARN_ON(num_active_tcc != num_max_tcc - hweight32(dis_tcc_mask));
+
+	/* Patch number of TCCs used by TCPs */
+	tcp_addr_config = REG_SET_FIELD(RREG32(mmTCP_ADDR_CONFIG),
+					TCP_ADDR_CONFIG, NUM_TCC_BANKS,
+					num_active_tcc - 1);
+
+	WREG32(mmTCP_ADDR_CONFIG, tcp_addr_config);
+	WREG32(mmTCP_CHAN_STEER_HI, upper_32_bits(patched_chan_steer));
+	WREG32(mmTCP_CHAN_STEER_LO, lower_32_bits(patched_chan_steer));
+}
+
 static void gfx_v6_0_config_init(struct amdgpu_device *adev)
 {
 	adev->gfx.config.double_offchip_lds_buf = 0;
@@ -1729,6 +1794,7 @@ static void gfx_v6_0_constants_init(struct amdgpu_device *adev)
 	gfx_v6_0_tiling_mode_table_init(adev);
 
 	gfx_v6_0_setup_rb(adev);
+	gfx_v6_0_setup_tcc(adev);
 
 	gfx_v6_0_setup_spi(adev);
 
Signature · 1 line
2.53.0
Applied
#5 lore raw

Re: [PATCH 4/4] drm/amdgpu/gfx6: Support harvested SI chips with disabled TCCs (v2)

Applied the series. Thanks!

Quoted 132 lines · Timur Kristóf

This commit fixes amdgpu to work on the Radeon HD 7870 XT which has never worked with the Linux open source drivers before.

Some boards have "harvested" chips, meaning that some parts of the chip are disabled and fused, and it's sold for cheaper and under a different marketing name.
On a harvested chip, any of the following can be disabled:
- CUs (Compute Units)
- RBs (Render Backend, aka. ROP)
- Memory channels (ie. the chip has a lower bandwidth)
- TCCs (ie. less L2 cache)

Handle chips with harvested TCCs by patching the registers that configure how TCCs are mapped.

If some TCCs are disabled, we need to make sure that the disabled TCCs are not used, and the remaining TCCs are used optimally.

TCP_CHAN_STEER_LO/HI control which TCC is used by TCP channels. TCP_ADDR_CONFIG.NUM_TCC_BANKS controls how many channels are used.

Note that the TCC configuration is highly relevant to performance. Suboptimal configuration (eg. CHAN_STEER=0) can significantly reduce gaming performance.

For optimal performance:
- Rely on the CHAN_STEER from the golden registers table, only skip disabled TCCs but keep the mapping order.
- Limit NUM_TCC_BANKS to number of active TCCs to avoid thrashing, which performs better than using the same TCC twice.

v2:
- Also consider CGTS_USER_TCC_DISABLE for disabled TCCs.

---
 drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c | 66 +++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)
drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c +66 -1
index 73223d97a87f5..ac90d8e9d86a8 100644
@@ -1571,6 +1571,71 @@ static void gfx_v6_0_setup_spi(struct amdgpu_device *adev)
        mutex_unlock(&adev->grbm_idx_mutex);
 }
 
+/**
+ * gfx_v6_0_setup_tcc() - setup which TCCs are used
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Verify whether the current GPU has any TCCs disabled,
+ * which can happen when the GPU is harvested and some
+ * memory channels are disabled, reducing the memory bus width.
+ * For example, on the Radeon HD 7870 XT (Tahiti LE).
+ *
+ * If some TCCs are disabled, we need to make sure that
+ * the disabled TCCs are not used, and the remaining TCCs
+ * are used optimally.
+ *
+ * TCP_CHAN_STEER_LO/HI control which TCC is used by TCP channels.
+ * TCP_ADDR_CONFIG.NUM_TCC_BANKS controls how many channels are used.
+ *
+ * For optimal performance:
+ * - Rely on the CHAN_STEER from the golden registers table,
+ *   only skip disabled TCCs but keep the mapping order.
+ * - Limit NUM_TCC_BANKS to number of active TCCs to avoid thrashing,
+ *   which performs better than using the same TCC twice.
+ */
+static void gfx_v6_0_setup_tcc(struct amdgpu_device *adev)
+{
+       u32 i, tcc, tcp_addr_config, num_active_tcc = 0;
+       u64 chan_steer, patched_chan_steer = 0;
+       const u32 num_max_tcc = adev->gfx.config.max_texture_channel_caches;
+       const u32 dis_tcc_mask =
+               amdgpu_gfx_create_bitmask(num_max_tcc) &
+               (REG_GET_FIELD(RREG32(mmCGTS_TCC_DISABLE),
+                              CGTS_TCC_DISABLE, TCC_DISABLE) |
+                REG_GET_FIELD(RREG32(mmCGTS_USER_TCC_DISABLE),
+                              CGTS_USER_TCC_DISABLE, TCC_DISABLE));
+
+       /* When no TCC is disabled, the golden registers table already has optimal TCC setup */
+       if (!dis_tcc_mask)
+               return;
+
+       /* Each 4-bit nibble contains the index of a TCC used by all TCPs */
+       chan_steer = RREG32(mmTCP_CHAN_STEER_LO) | ((u64)RREG32(mmTCP_CHAN_STEER_HI) << 32ull);
+
+       /* Patch the TCP to TCC mapping to skip disabled TCCs */
+       for (i = 0; i < num_max_tcc; ++i) {
+               tcc = (chan_steer >> (u64)(4 * i)) & 0xf;
+
+               if (!((1 << tcc) & dis_tcc_mask)) {
+                       /* Copy enabled TCC indices to the patched register value. */
+                       patched_chan_steer |= (u64)tcc << (u64)(4 * num_active_tcc);
+                       ++num_active_tcc;
+               }
+       }
+
+       WARN_ON(num_active_tcc != num_max_tcc - hweight32(dis_tcc_mask));
+
+       /* Patch number of TCCs used by TCPs */
+       tcp_addr_config = REG_SET_FIELD(RREG32(mmTCP_ADDR_CONFIG),
+                                       TCP_ADDR_CONFIG, NUM_TCC_BANKS,
+                                       num_active_tcc - 1);
+
+       WREG32(mmTCP_ADDR_CONFIG, tcp_addr_config);
+       WREG32(mmTCP_CHAN_STEER_HI, upper_32_bits(patched_chan_steer));
+       WREG32(mmTCP_CHAN_STEER_LO, lower_32_bits(patched_chan_steer));
+}
+
 static void gfx_v6_0_config_init(struct amdgpu_device *adev)
 {
        adev->gfx.config.double_offchip_lds_buf = 0;
@@ -1729,6 +1794,7 @@ static void gfx_v6_0_constants_init(struct amdgpu_device *adev)
        gfx_v6_0_tiling_mode_table_init(adev);
 
        gfx_v6_0_setup_rb(adev);
+       gfx_v6_0_setup_tcc(adev);
 
        gfx_v6_0_setup_spi(adev);
 
--

2.53.0