Skip to contentSkip to conversation

Update AUX invalidation sequence

PATCH Superseded Andi Shyti 4 participants 30 messages Jul 20, 2023 – Jul 21, 2023

Subsystems (3)

v7 of 9 Superseded Latest is v9 →

lore
Share & tools
lore

Andi Shyti

author
#0 lore raw

v7 cover Update AUX invalidation sequence

Hi,

as there are new hardware directives, we need a little adaptation for the AUX invalidation sequence.

In this version we support all the engines affected by this change.

The stable backport has some challenges because the original patch that this series fixes has had more changes in between.

This patch is slowly exploding with code refactorings and features added and fixed.

Thanks a lot Nirmoy, Andrzej and Matt for your review and for the fruitful discussions!

Thanks,
Andi

Changelog:
=========
v6 -> v7
- Fix correct sequence applied to the correct engine. A little confusion promptly cought by Nirmoy when applying to the VD engine the sequence belonging to the compute engines. Thanks a lot, Nirmoy!

v5 -> v6
- Fixed ccs flush in the engines VE and BCS. They are sent as a separate command instead of added in the pipe control.
- Separated the CCS flusing in the pipe control patch with the quiescing of the memory. They were meant to be on separate patch already in the previous verision, but apparently I squashed them by mistake.

v4 -> v5
- The AUX CCS is added as a device property instead of checking against FLAT CCS. This adds the new HAS_AUX_CCS check (Patch 2, new).
- little and trivial refactoring here and there.
- extended the flags{0,1}/bit_group_{0,1} renaming to other functions.
- Created an intel_emit_pipe_control_cs() wrapper for submitting the pipe control.
- Quiesce memory for all the engines, not just RCS (Patch 6, new).
- The PIPE_CONTROL_CCS_FLUSH is added to all the engines.
- Remove redundant EMIT_FLUSH_CCS mode flag.
- Remove unnecessary NOOPs from the command streamer for invalidating the CCS table.
- Use INVALID_MMIO_REG and gen12_get_aux_inv_reg() instad of __MMIO(0) and reg.reg.
- Remove useless wrapper and just use gen12_get_aux_inv_reg().

v3 -> v4
- A trivial patch 3 is added to rename the flags with bit_group_{0,1} to align with the datasheet naming.
- Patch 4 fixes a confusion I made where the CCS flag was applied to the wrong bit group.

v2 -> v3
- added r-b from Nirmoy in patch 1 and 4.
- added patch 3 which enables the ccs_flush in the control pipe for mtl+ compute and render engines.
- added redundant checks in patch 2 for enabling the EMIT_FLUSH flag.

v1 -> v2
- add a clean up preliminary patch for the existing registers
- add support for more engines
- add the Fixes tag

Andi Shyti (7):
drm/i915/gt: Cleanup aux invalidation registers
drm/i915: Add the has_aux_ccs device property
drm/i915/gt: Rename flags with bit_group_X according to the datasheet drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control drm/i915/gt: Refactor intel_emit_pipe_control_cs() in a single
function
drm/i915/gt: Ensure memory quiesced before invalidation for all
engines
drm/i915/gt: Support aux invalidation on all engines

Jonathan Cavitt (2):
drm/i915/gt: Ensure memory quiesced before invalidation drm/i915/gt: Poll aux invalidation register bit on invalidation

 drivers/gpu/drm/i915/gt/gen8_engine_cs.c     | 188 ++++++++++++-------
 drivers/gpu/drm/i915/gt/gen8_engine_cs.h     |  21 ++-
 drivers/gpu/drm/i915/gt/intel_gpu_commands.h |   2 +
 drivers/gpu/drm/i915/gt/intel_gt_regs.h      |  16 +-
 drivers/gpu/drm/i915/gt/intel_lrc.c          |  17 +-
 drivers/gpu/drm/i915/i915_drv.h              |   1 +
 drivers/gpu/drm/i915/i915_pci.c              |   5 +-
 drivers/gpu/drm/i915/intel_device_info.h     |   1 +
 8 files changed, 151 insertions(+), 100 deletions(-)
Signature · 1 line
2.40.1
author
#1 lore raw

PATCH v7 1/9 drm/i915/gt: Cleanup aux invalidation registers

Fix the 'NV' definition postfix that is supposed to be INV.

Take the chance to also order properly the registers based on their address and call the GEN12_GFX_CCS_AUX_INV address as GEN12_CCS_AUX_INV like all the other similar registers.

Remove also VD1, VD3 and VE1 registers that don't exist and add BCS0 and CCS0.

---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c |  8 ++++----
 drivers/gpu/drm/i915/gt/intel_gt_regs.h  | 16 ++++++++--------
 drivers/gpu/drm/i915/gt/intel_lrc.c      |  6 +++---
 3 files changed, 15 insertions(+), 15 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +4 -4
index 23857cc08eca1..563efee055602 100644
@@ -287,8 +287,8 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
 		if (!HAS_FLAT_CCS(rq->engine->i915)) {
 			/* hsdes: 1809175790 */
-			cs = gen12_emit_aux_table_inv(rq->engine->gt,
-						      cs, GEN12_GFX_CCS_AUX_NV);
+			cs = gen12_emit_aux_table_inv(rq->engine->gt, cs,
+						      GEN12_CCS_AUX_INV);
 		}
 
 		*cs++ = preparser_disable(false);
@@ -348,10 +348,10 @@ int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 	if (aux_inv) { /* hsdes: 1809175790 */
 		if (rq->engine->class == VIDEO_DECODE_CLASS)
 			cs = gen12_emit_aux_table_inv(rq->engine->gt,
-						      cs, GEN12_VD0_AUX_NV);
+						      cs, GEN12_VD0_AUX_INV);
 		else
 			cs = gen12_emit_aux_table_inv(rq->engine->gt,
-						      cs, GEN12_VE0_AUX_NV);
+						      cs, GEN12_VE0_AUX_INV);
 	}
 
 	if (mode & EMIT_INVALIDATE)
drivers/gpu/drm/i915/gt/intel_gt_regs.h +8 -8
index 718cb2c80f79e..2cdfb2f713d02 100644
@@ -332,9 +332,11 @@
 #define GEN8_PRIVATE_PAT_HI			_MMIO(0x40e0 + 4)
 #define GEN10_PAT_INDEX(index)			_MMIO(0x40e0 + (index) * 4)
 #define BSD_HWS_PGA_GEN7			_MMIO(0x4180)
-#define GEN12_GFX_CCS_AUX_NV			_MMIO(0x4208)
-#define GEN12_VD0_AUX_NV			_MMIO(0x4218)
-#define GEN12_VD1_AUX_NV			_MMIO(0x4228)
+
+#define GEN12_CCS_AUX_INV			_MMIO(0x4208)
+#define GEN12_VD0_AUX_INV			_MMIO(0x4218)
+#define GEN12_VE0_AUX_INV			_MMIO(0x4238)
+#define GEN12_BCS0_AUX_INV			_MMIO(0x4248)
 
 #define GEN8_RTCR				_MMIO(0x4260)
 #define GEN8_M1TCR				_MMIO(0x4264)
@@ -342,14 +344,12 @@
 #define GEN8_BTCR				_MMIO(0x426c)
 #define GEN8_VTCR				_MMIO(0x4270)
 
-#define GEN12_VD2_AUX_NV			_MMIO(0x4298)
-#define GEN12_VD3_AUX_NV			_MMIO(0x42a8)
-#define GEN12_VE0_AUX_NV			_MMIO(0x4238)
-
 #define BLT_HWS_PGA_GEN7			_MMIO(0x4280)
 
-#define GEN12_VE1_AUX_NV			_MMIO(0x42b8)
+#define GEN12_VD2_AUX_INV			_MMIO(0x4298)
+#define GEN12_CCS0_AUX_INV			_MMIO(0x42c8)
 #define   AUX_INV				REG_BIT(0)
+
 #define VEBOX_HWS_PGA_GEN7			_MMIO(0x4380)
 
 #define GEN12_AUX_ERR_DBG			_MMIO(0x43f4)
drivers/gpu/drm/i915/gt/intel_lrc.c +3 -3
index 1b710102390bf..235f3fab60a98 100644
@@ -1374,7 +1374,7 @@ gen12_emit_indirect_ctx_rcs(const struct intel_context *ce, u32 *cs)
 	/* hsdes: 1809175790 */
 	if (!HAS_FLAT_CCS(ce->engine->i915))
 		cs = gen12_emit_aux_table_inv(ce->engine->gt,
-					      cs, GEN12_GFX_CCS_AUX_NV);
+					      cs, GEN12_CCS_AUX_INV);
 
 	/* Wa_16014892111 */
 	if (IS_MTL_GRAPHICS_STEP(ce->engine->i915, M, STEP_A0, STEP_B0) ||
@@ -1403,10 +1403,10 @@ gen12_emit_indirect_ctx_xcs(const struct intel_context *ce, u32 *cs)
 	if (!HAS_FLAT_CCS(ce->engine->i915)) {
 		if (ce->engine->class == VIDEO_DECODE_CLASS)
 			cs = gen12_emit_aux_table_inv(ce->engine->gt,
-						      cs, GEN12_VD0_AUX_NV);
+						      cs, GEN12_VD0_AUX_INV);
 		else if (ce->engine->class == VIDEO_ENHANCEMENT_CLASS)
 			cs = gen12_emit_aux_table_inv(ce->engine->gt,
-						      cs, GEN12_VE0_AUX_NV);
+						      cs, GEN12_VE0_AUX_INV);
 	}
 
 	return cs;
Signature · 1 line
2.40.1
author
#2 lore raw

PATCH v7 2/9 drm/i915: Add the has_aux_ccs device property

In reply to Andi Shyti

We always assumed that a device might either have AUX or FLAT CCS, but this is an approximation that is not always true as it requires some further per device checks.

Add the "has_aux_ccs" flag in the intel_device_info structure in order to have a per device flag indicating of the AUX CCS.

Cc Matt Roper <[email protected]>
Cc Jonathan Cavitt <[email protected]>
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 4 ++--
 drivers/gpu/drm/i915/i915_drv.h          | 1 +
 drivers/gpu/drm/i915/i915_pci.c          | 5 ++++-
 drivers/gpu/drm/i915/intel_device_info.h | 1 +
 4 files changed, 8 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +2 -2
index 563efee055602..0d4d5e0407a2d 100644
@@ -267,7 +267,7 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 		else if (engine->class == COMPUTE_CLASS)
 			flags &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
 
-		if (!HAS_FLAT_CCS(rq->engine->i915))
+		if (HAS_AUX_CCS(rq->engine->i915))
 			count = 8 + 4;
 		else
 			count = 8;
@@ -307,7 +307,7 @@ int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 	if (mode & EMIT_INVALIDATE) {
 		cmd += 2;
 
-		if (!HAS_FLAT_CCS(rq->engine->i915) &&
+		if (HAS_AUX_CCS(rq->engine->i915) &&
 		    (rq->engine->class == VIDEO_DECODE_CLASS ||
 		     rq->engine->class == VIDEO_ENHANCEMENT_CLASS)) {
 			aux_inv = rq->engine->mask &
drivers/gpu/drm/i915/i915_drv.h +1 -0
index 682ef2b5c7d59..e9cc048b5727a 100644
@@ -848,6 +848,7 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
  * stored in lmem to support the 3D and media compression formats.
  */
 #define HAS_FLAT_CCS(i915)   (INTEL_INFO(i915)->has_flat_ccs)
+#define HAS_AUX_CCS(i915)    (INTEL_INFO(i915)->has_aux_ccs)
 
 #define HAS_GT_UC(i915)	(INTEL_INFO(i915)->has_gt_uc)
 
drivers/gpu/drm/i915/i915_pci.c +4 -1
index fcacdc21643cf..c9ff1d11a9fce 100644
@@ -643,7 +643,8 @@ static const struct intel_device_info jsl_info = {
 	TGL_CACHELEVEL, \
 	.has_global_mocs = 1, \
 	.has_pxp = 1, \
-	.max_pat_index = 3
+	.max_pat_index = 3, \
+	.has_aux_ccs = 1
 
 static const struct intel_device_info tgl_info = {
 	GEN12_FEATURES,
@@ -775,6 +776,7 @@ static const struct intel_device_info dg2_info = {
 
 static const struct intel_device_info ats_m_info = {
 	DG2_FEATURES,
+	.has_aux_ccs = 1,
 	.require_force_probe = 1,
 	.tuning_thread_rr_after_dep = 1,
 };
@@ -827,6 +829,7 @@ static const struct intel_device_info mtl_info = {
 	.__runtime.media.ip.ver = 13,
 	PLATFORM(INTEL_METEORLAKE),
 	.extra_gt_list = xelpmp_extra_gt,
+	.has_aux_ccs = 1,
 	.has_flat_ccs = 0,
 	.has_gmd_id = 1,
 	.has_guc_deprivilege = 1,
drivers/gpu/drm/i915/intel_device_info.h +1 -0
index dbfe6443457b5..93485507506cc 100644
@@ -151,6 +151,7 @@ enum intel_ppgtt_type {
 	func(has_reset_engine); \
 	func(has_3d_pipeline); \
 	func(has_4tile); \
+	func(has_aux_ccs); \
 	func(has_flat_ccs); \
 	func(has_global_mocs); \
 	func(has_gmd_id); \
Signature · 1 line
2.40.1
#3 lore raw

Re: [v7,2/9] drm/i915: Add the has_aux_ccs device property

Hi Andy,

Quoted 2 lines · On Thursday, 20 July 2023 23:07:30 CEST Andi Shyti wrote:

We always assumed that a device might either have AUX or FLAT CCS, but this is an approximation that is not always true

If there exists a device that can have CCSs that fall into either none or both of those categories then I think we should have that device or two listed here as an example, regardless of deducible from the change or not. Or if there are no such devices so far, but we are going to introduce some, then I think we should provide that information here.

drivers/gpu/drm/

Quoted 18 lines

as it
requires some further per device checks.

Add the "has_aux_ccs" flag in the intel_device_info structure in order to have a per device flag indicating of the AUX CCS.

Cc Matt Roper <[email protected]>
Cc Jonathan Cavitt <[email protected]>
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 4 ++--
 drivers/gpu/drm/i915/i915_drv.h          | 1 +
 drivers/gpu/drm/i915/i915_pci.c          | 5 ++++-
 drivers/gpu/drm/i915/intel_device_info.h | 1 +
 4 files changed, 8 insertions(+), 3 deletions(-)
drivers/gpu/drm/ renamed from drivers/gpu/drm/i915/gt/gen8_engine_cs.c +0 -0

i915/gt/gen8_engine_cs.c

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_rcs(struct i915_request *rq, u32

Quoted 4 lines

index 563efee055602..0d4d5e0407a2d 100644

drivers/gpu/drm/i915/gt/gen8_engine_cs.c +0 -0
@@ -267,7 +267,7 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 

mode)

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_xcs(struct i915_request *rq, u32

Quoted 9 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +1 -1
 		else if (engine->class == COMPUTE_CLASS)
 			flags &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
 
-		if (!HAS_FLAT_CCS(rq->engine->i915))
+		if (HAS_AUX_CCS(rq->engine->i915))
 			count = 8 + 4;
 		else
 			count = 8;
@@ -307,7 +307,7 @@ int gen12_emit_flush_xcs(struct i915_request *rq, u32 

mode)

drivers/gpu/drm/i915/

Quoted 9 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +1 -1
 	if (mode & EMIT_INVALIDATE) {
 		cmd += 2;
 
-		if (!HAS_FLAT_CCS(rq->engine->i915) &&
+		if (HAS_AUX_CCS(rq->engine->i915) &&
 		    (rq->engine->class == VIDEO_DECODE_CLASS ||
 		     rq->engine->class == VIDEO_ENHANCEMENT_CLASS)) {
 			aux_inv = rq->engine->mask &
drivers/gpu/drm/i915/ renamed from drivers/gpu/drm/i915/i915_drv.h +0 -0

i915_drv.h

drivers/gpu/drm/i915/

Quoted 12 lines

index 682ef2b5c7d59..e9cc048b5727a 100644

drivers/gpu/drm/i915/i915_drv.h +1 -0
@@ -848,6 +848,7 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
  * stored in lmem to support the 3D and media compression formats.
  */
 #define HAS_FLAT_CCS(i915)   (INTEL_INFO(i915)->has_flat_ccs)
+#define HAS_AUX_CCS(i915)    (INTEL_INFO(i915)->has_aux_ccs)
 
 #define HAS_GT_UC(i915)	(INTEL_INFO(i915)->has_gt_uc)
 
drivers/gpu/drm/i915/ renamed from drivers/gpu/drm/i915/i915_pci.c +0 -0

i915_pci.c

drivers/gpu/drm/i915/i915_pci.c static const struct intel_device_info jsl_info = { · line 647

Quoted 10 lines

index fcacdc21643cf..c9ff1d11a9fce 100644

drivers/gpu/drm/i915/i915_pci.c +2 -1
@@ -643,7 +643,8 @@ static const struct intel_device_info jsl_info = {
 	TGL_CACHELEVEL, \
 	.has_global_mocs = 1, \
 	.has_pxp = 1, \
-	.max_pat_index = 3
+	.max_pat_index = 3, \
+	.has_aux_ccs = 1

NIT: Can we please have the last element also followed by comma, like in other
places (e.g. see below)? That will simplify future patches.

Other than that, LGTM.

Thanks,
Janusz

drivers/gpu/drm/

Quoted 20 lines
drivers/gpu/drm/i915/i915_pci.c +2 -0
 static const struct intel_device_info tgl_info = {
 	GEN12_FEATURES,
@@ -775,6 +776,7 @@ static const struct intel_device_info dg2_info = {
 
 static const struct intel_device_info ats_m_info = {
 	DG2_FEATURES,
+	.has_aux_ccs = 1,
 	.require_force_probe = 1,
 	.tuning_thread_rr_after_dep = 1,
 };
@@ -827,6 +829,7 @@ static const struct intel_device_info mtl_info = {
 	.__runtime.media.ip.ver = 13,
 	PLATFORM(INTEL_METEORLAKE),
 	.extra_gt_list = xelpmp_extra_gt,
+	.has_aux_ccs = 1,
 	.has_flat_ccs = 0,
 	.has_gmd_id = 1,
 	.has_guc_deprivilege = 1,
drivers/gpu/drm/ renamed from drivers/gpu/drm/i915/intel_device_info.h +0 -0

i915/intel_device_info.h

drivers/gpu/drm/i915/intel_device_info.h enum intel_ppgtt_type { · line 157

Quoted 12 lines

index dbfe6443457b5..93485507506cc 100644

drivers/gpu/drm/i915/intel_device_info.h +1 -0
@@ -151,6 +151,7 @@ enum intel_ppgtt_type {
 	func(has_reset_engine); \
 	func(has_3d_pipeline); \
 	func(has_4tile); \
+	func(has_aux_ccs); \
 	func(has_flat_ccs); \
 	func(has_global_mocs); \
 	func(has_gmd_id); \

---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych.

Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.

author
#4 lore raw

Re: [v7,2/9] drm/i915: Add the has_aux_ccs device property

Hi Janusz,

Quoted 11 lines · Janusz

Hi Andy,

Quoted 2 lines · On Thursday, 20 July 2023 23:07:30 CEST Andi Shyti wrote:

We always assumed that a device might either have AUX or FLAT CCS, but this is an approximation that is not always true

If there exists a device that can have CCSs that fall into either none or both of those categories then I think we should have that device or two listed here as an example, regardless of deducible from the change or not. Or if there are no such devices so far, but we are going to introduce some, then I think we should provide that information here.

true! I will improve the commit log.

[...]

Quoted 13 lines

drivers/gpu/drm/i915/i915_pci.c static const struct intel_device_info jsl_info = { · line 647

Quoted 9 lines
drivers/gpu/drm/i915/i915_pci.c +2 -1
@@ -643,7 +643,8 @@ static const struct intel_device_info jsl_info = {
 	TGL_CACHELEVEL, \
 	.has_global_mocs = 1, \
 	.has_pxp = 1, \
-	.max_pat_index = 3
+	.max_pat_index = 3, \
+	.has_aux_ccs = 1

NIT: Can we please have the last element also followed by comma, like in other
places (e.g. see below)? That will simplify future patches.

Quoted 1 line

Other than that, LGTM.

As Andrzej and Matt suggested I will take another approach, i.e. adding a helper function that tells whether the aux invalidation is necessary or not.

Thanks,
Andi

#5 lore raw

Re: [Intel-gfx] [PATCH v7 2/9] drm/i915: Add the has_aux_ccs device property

In reply to Andi Shyti
Quoted 6 lines · Andi Shyti

We always assumed that a device might either have AUX or FLAT CCS, but this is an approximation that is not always true as it requires some further per device checks.

Add the "has_aux_ccs" flag in the intel_device_info structure in order to have a per device flag indicating of the AUX CCS.

As Matt mentioned in v6, aux_ccs is present also in older platforms. This is about presence/necessity (?) of aux_ccs table invalidation. Maybe has_aux_ccs_inv, dunno?

Moreover you define flag per device, but this is rather per engine, theoretically could be also:
MTL:
.aux_ccs_inv_mask = BIT(RCS0) | BIT(BCS0) | ...
Others:
.aux_ccs_inv_mask = BIT(RCS0) | ...

looks overkill,
maybe helper function would be simpler, up to you.

Regards
Andrzej

Quoted 88 lines
Cc Matt Roper <[email protected]>
Cc Jonathan Cavitt <[email protected]>
Cc <[email protected]> # v5.8+
---
  drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 4 ++--
  drivers/gpu/drm/i915/i915_drv.h          | 1 +
  drivers/gpu/drm/i915/i915_pci.c          | 5 ++++-
  drivers/gpu/drm/i915/intel_device_info.h | 1 +
  4 files changed, 8 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +2 -2
index 563efee055602..0d4d5e0407a2d 100644
@@ -267,7 +267,7 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
  		else if (engine->class == COMPUTE_CLASS)
  			flags &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
 
-		if (!HAS_FLAT_CCS(rq->engine->i915))
+		if (HAS_AUX_CCS(rq->engine->i915))
  			count = 8 + 4;
  		else
  			count = 8;
@@ -307,7 +307,7 @@ int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
  	if (mode & EMIT_INVALIDATE) {
  		cmd += 2;
 
-		if (!HAS_FLAT_CCS(rq->engine->i915) &&
+		if (HAS_AUX_CCS(rq->engine->i915) &&
  		    (rq->engine->class == VIDEO_DECODE_CLASS ||
  		     rq->engine->class == VIDEO_ENHANCEMENT_CLASS)) {
  			aux_inv = rq->engine->mask &
drivers/gpu/drm/i915/i915_drv.h +1 -0
index 682ef2b5c7d59..e9cc048b5727a 100644
@@ -848,6 +848,7 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915,
   * stored in lmem to support the 3D and media compression formats.
   */
  #define HAS_FLAT_CCS(i915)   (INTEL_INFO(i915)->has_flat_ccs)
+#define HAS_AUX_CCS(i915)    (INTEL_INFO(i915)->has_aux_ccs)
 
  #define HAS_GT_UC(i915)	(INTEL_INFO(i915)->has_gt_uc)
 
drivers/gpu/drm/i915/i915_pci.c +4 -1
index fcacdc21643cf..c9ff1d11a9fce 100644
@@ -643,7 +643,8 @@ static const struct intel_device_info jsl_info = {
  	TGL_CACHELEVEL, \
  	.has_global_mocs = 1, \
  	.has_pxp = 1, \
-	.max_pat_index = 3
+	.max_pat_index = 3, \
+	.has_aux_ccs = 1
 
  static const struct intel_device_info tgl_info = {
  	GEN12_FEATURES,
@@ -775,6 +776,7 @@ static const struct intel_device_info dg2_info = {
 
  static const struct intel_device_info ats_m_info = {
  	DG2_FEATURES,
+	.has_aux_ccs = 1,
  	.require_force_probe = 1,
  	.tuning_thread_rr_after_dep = 1,
  };
@@ -827,6 +829,7 @@ static const struct intel_device_info mtl_info = {
  	.__runtime.media.ip.ver = 13,
  	PLATFORM(INTEL_METEORLAKE),
  	.extra_gt_list = xelpmp_extra_gt,
+	.has_aux_ccs = 1,
  	.has_flat_ccs = 0,
  	.has_gmd_id = 1,
  	.has_guc_deprivilege = 1,
drivers/gpu/drm/i915/intel_device_info.h +1 -0
index dbfe6443457b5..93485507506cc 100644
@@ -151,6 +151,7 @@ enum intel_ppgtt_type {
  	func(has_reset_engine); \
  	func(has_3d_pipeline); \
  	func(has_4tile); \
+	func(has_aux_ccs); \
  	func(has_flat_ccs); \
  	func(has_global_mocs); \
  	func(has_gmd_id); \
author
#6 lore raw

Re: [Intel-gfx] [PATCH v7 2/9] drm/i915: Add the has_aux_ccs device property

Hi Andrzej,

Quoted 11 lines · Andrzej Hajda
Quoted 6 lines · Andi Shyti

We always assumed that a device might either have AUX or FLAT CCS, but this is an approximation that is not always true as it requires some further per device checks.

Add the "has_aux_ccs" flag in the intel_device_info structure in order to have a per device flag indicating of the AUX CCS.

As Matt mentioned in v6, aux_ccs is present also in older platforms. This is about presence/necessity (?) of aux_ccs table invalidation. Maybe has_aux_ccs_inv, dunno?

yes, true! It's aux_ccs_inv.

Quoted 6 lines

Moreover you define flag per device, but this is rather per engine, theoretically could be also:
MTL:
.aux_ccs_inv_mask = BIT(RCS0) | BIT(BCS0) | ...
Others:
.aux_ccs_inv_mask = BIT(RCS0) | ...

there is already some engine discrimination that is mandatory later in the series. Doing it here it's a bit replicated.

Quoted 2 lines

looks overkill,
maybe helper function would be simpler, up to you.

Yes, a helper function that checks on the platform and returns true or false... it looks hardcoded to me, while the info structure is hardcoded by definition and looks easier to maintain by just toggling on/off a single flag in that structure. That's why I decided to place it there.

But, because there are already two people suggesting it, then I will try it in v8.

Thanks,
Andi

author
#7 lore raw

PATCH v7 3/9 drm/i915/gt: Ensure memory quiesced before invalidation

In reply to Andi Shyti

From: Jonathan Cavitt <[email protected]>

All memory traffic must be quiesced before requesting an aux invalidation on platforms that use Aux CCS.

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +5 -1
index 0d4d5e0407a2d..5fbc3f630f32b 100644
@@ -202,7 +202,11 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 {
 	struct intel_engine_cs *engine = rq->engine;
 
-	if (mode & EMIT_FLUSH) {
+	/*
+	 * On Aux CCS platforms the invalidation of the Aux
+	 * table requires quiescing memory traffic beforehand
+	 */
+	if (mode & EMIT_FLUSH || HAS_AUX_CCS(engine->i915)) {
 		u32 flags = 0;
 		int err;
 		u32 *cs;
Signature · 1 line
2.40.1
author
#8 lore raw

PATCH v7 4/9 drm/i915/gt: Rename flags with bit_group_X according to the datasheet

In reply to Andi Shyti

In preparation of the next patch align with the datasheet (BSPEC 47112) with the naming of the pipe control set of flag values. The variable "flags" in gen12_emit_flush_rcs() is applied as a set of flags called Bit Group 1.

Define also the Bit Group 0 as bit_group_0 where currently only PIPE_CONTROL0_HDC_PIPELINE_FLUSH bit is set.

---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 34 +++++++++++++-----------
 drivers/gpu/drm/i915/gt/gen8_engine_cs.h | 18 ++++++++-----
 2 files changed, 29 insertions(+), 23 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +18 -16
index 5fbc3f630f32b..7566c89d9def3 100644
@@ -207,7 +207,8 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 	 * table requires quiescing memory traffic beforehand
 	 */
 	if (mode & EMIT_FLUSH || HAS_AUX_CCS(engine->i915)) {
-		u32 flags = 0;
+		u32 bit_group_0 = 0;
+		u32 bit_group_1 = 0;
 		int err;
 		u32 *cs;
 
@@ -215,32 +216,33 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 		if (err)
 			return err;
 
-		flags |= PIPE_CONTROL_TILE_CACHE_FLUSH;
-		flags |= PIPE_CONTROL_FLUSH_L3;
-		flags |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
-		flags |= PIPE_CONTROL_DEPTH_CACHE_FLUSH;
+		bit_group_0 |= PIPE_CONTROL0_HDC_PIPELINE_FLUSH;
+
+		bit_group_1 |= PIPE_CONTROL_TILE_CACHE_FLUSH;
+		bit_group_1 |= PIPE_CONTROL_FLUSH_L3;
+		bit_group_1 |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
+		bit_group_1 |= PIPE_CONTROL_DEPTH_CACHE_FLUSH;
 		/* Wa_1409600907:tgl,adl-p */
-		flags |= PIPE_CONTROL_DEPTH_STALL;
-		flags |= PIPE_CONTROL_DC_FLUSH_ENABLE;
-		flags |= PIPE_CONTROL_FLUSH_ENABLE;
+		bit_group_1 |= PIPE_CONTROL_DEPTH_STALL;
+		bit_group_1 |= PIPE_CONTROL_DC_FLUSH_ENABLE;
+		bit_group_1 |= PIPE_CONTROL_FLUSH_ENABLE;
 
-		flags |= PIPE_CONTROL_STORE_DATA_INDEX;
-		flags |= PIPE_CONTROL_QW_WRITE;
+		bit_group_1 |= PIPE_CONTROL_STORE_DATA_INDEX;
+		bit_group_1 |= PIPE_CONTROL_QW_WRITE;
 
-		flags |= PIPE_CONTROL_CS_STALL;
+		bit_group_1 |= PIPE_CONTROL_CS_STALL;
 
 		if (!HAS_3D_PIPELINE(engine->i915))
-			flags &= ~PIPE_CONTROL_3D_ARCH_FLAGS;
+			bit_group_1 &= ~PIPE_CONTROL_3D_ARCH_FLAGS;
 		else if (engine->class == COMPUTE_CLASS)
-			flags &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
+			bit_group_1 &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
 
 		cs = intel_ring_begin(rq, 6);
 		if (IS_ERR(cs))
 			return PTR_ERR(cs);
 
-		cs = gen12_emit_pipe_control(cs,
-					     PIPE_CONTROL0_HDC_PIPELINE_FLUSH,
-					     flags, LRC_PPHWSP_SCRATCH_ADDR);
+		cs = gen12_emit_pipe_control(cs, bit_group_0, bit_group_1,
+					     LRC_PPHWSP_SCRATCH_ADDR);
 		intel_ring_advance(rq, cs);
 	}
 
drivers/gpu/drm/i915/gt/gen8_engine_cs.h +11 -7
index 655e5c00ddc27..a44eda096557c 100644
@@ -49,25 +49,29 @@ u32 *gen12_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs);
 u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv_reg);
 
 static inline u32 *
-__gen8_emit_pipe_control(u32 *batch, u32 flags0, u32 flags1, u32 offset)
+__gen8_emit_pipe_control(u32 *batch, u32 bit_group_0,
+			 u32 bit_group_1, u32 offset)
 {
 	memset(batch, 0, 6 * sizeof(u32));
 
-	batch[0] = GFX_OP_PIPE_CONTROL(6) | flags0;
-	batch[1] = flags1;
+	batch[0] = GFX_OP_PIPE_CONTROL(6) | bit_group_0;
+	batch[1] = bit_group_1;
 	batch[2] = offset;
 
 	return batch + 6;
 }
 
-static inline u32 *gen8_emit_pipe_control(u32 *batch, u32 flags, u32 offset)
+static inline u32 *gen8_emit_pipe_control(u32 *batch,
+					  u32 bit_group_1, u32 offset)
 {
-	return __gen8_emit_pipe_control(batch, 0, flags, offset);
+	return __gen8_emit_pipe_control(batch, 0, bit_group_1, offset);
 }
 
-static inline u32 *gen12_emit_pipe_control(u32 *batch, u32 flags0, u32 flags1, u32 offset)
+static inline u32 *gen12_emit_pipe_control(u32 *batch, u32 bit_group_0,
+					   u32 bit_group_1, u32 offset)
 {
-	return __gen8_emit_pipe_control(batch, flags0, flags1, offset);
+	return __gen8_emit_pipe_control(batch, bit_group_0,
+					bit_group_1, offset);
 }
 
 static inline u32 *
Signature · 1 line
2.40.1
author
#9 lore raw

PATCH v7 5/9 drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control

In reply to Andi Shyti

Enable the CCS_FLUSH bit 13 in the control pipe for render and compute engines in platforms starting from Meteor Lake (BSPEC 43904 and 47112).

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")
Cc Jonathan Cavitt <[email protected]>
Cc Nirmoy Das <[email protected]>
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c     | 7 +++++++
 drivers/gpu/drm/i915/gt/intel_gpu_commands.h | 1 +
 2 files changed, 8 insertions(+)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +7 -0
index 7566c89d9def3..9d050b9a19194 100644
@@ -218,6 +218,13 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
 		bit_group_0 |= PIPE_CONTROL0_HDC_PIPELINE_FLUSH;
 
+		/*
+		 * When required, in MTL+ platforms we need to
+		 * set the CCS_FLUSH bit in the pipe control
+		 */
+		if (GRAPHICS_VER_FULL(rq->i915) >= IP_VER(12, 70))
+			bit_group_0 |= PIPE_CONTROL_CCS_FLUSH;
+
 		bit_group_1 |= PIPE_CONTROL_TILE_CACHE_FLUSH;
 		bit_group_1 |= PIPE_CONTROL_FLUSH_L3;
 		bit_group_1 |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
drivers/gpu/drm/i915/gt/intel_gpu_commands.h +1 -0
index 5d143e2a8db03..5df7cce23197c 100644
@@ -299,6 +299,7 @@
 #define   PIPE_CONTROL_QW_WRITE				(1<<14)
 #define   PIPE_CONTROL_POST_SYNC_OP_MASK                (3<<14)
 #define   PIPE_CONTROL_DEPTH_STALL			(1<<13)
+#define   PIPE_CONTROL_CCS_FLUSH			(1<<13) /* MTL+ */
 #define   PIPE_CONTROL_WRITE_FLUSH			(1<<12)
 #define   PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH	(1<<12) /* gen6+ */
 #define   PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE	(1<<11) /* MBZ on ILK */
Signature · 1 line
2.40.1
#10 lore raw

Re: [Intel-gfx] [PATCH v7 5/9] drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control

Quoted 19 lines · Andi Shyti

Enable the CCS_FLUSH bit 13 in the control pipe for render and compute engines in platforms starting from Meteor Lake (BSPEC 43904 and 47112).

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")
Cc Jonathan Cavitt <[email protected]>
Cc Nirmoy Das <[email protected]>
Cc <[email protected]> # v5.8+
---
  drivers/gpu/drm/i915/gt/gen8_engine_cs.c     | 7 +++++++
  drivers/gpu/drm/i915/gt/intel_gpu_commands.h | 1 +
  2 files changed, 8 insertions(+)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +0 -0
index 7566c89d9def3..9d050b9a19194 100644

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode) · line 226

Quoted 9 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +6 -0
@@ -218,6 +218,13 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
  		bit_group_0 |= PIPE_CONTROL0_HDC_PIPELINE_FLUSH;
 
+		/*
+		 * When required, in MTL+ platforms we need to
+		 * set the CCS_FLUSH bit in the pipe control
+		 */
+		if (GRAPHICS_VER_FULL(rq->i915) >= IP_VER(12, 70))
+			bit_group_0 |= PIPE_CONTROL_CCS_FLUSH;

BSpec 43904 mentions also other platforms. Why only MTL+?

Regards
Andrzej

Quoted 16 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +1 -0
+
  		bit_group_1 |= PIPE_CONTROL_TILE_CACHE_FLUSH;
  		bit_group_1 |= PIPE_CONTROL_FLUSH_L3;
  		bit_group_1 |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
drivers/gpu/drm/i915/gt/intel_gpu_commands.h +1 -0
index 5d143e2a8db03..5df7cce23197c 100644
@@ -299,6 +299,7 @@
  #define   PIPE_CONTROL_QW_WRITE				(1<<14)
  #define   PIPE_CONTROL_POST_SYNC_OP_MASK                (3<<14)
  #define   PIPE_CONTROL_DEPTH_STALL			(1<<13)
+#define   PIPE_CONTROL_CCS_FLUSH			(1<<13) /* MTL+ */
  #define   PIPE_CONTROL_WRITE_FLUSH			(1<<12)
  #define   PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH	(1<<12) /* gen6+ */
  #define   PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE	(1<<11) /* MBZ on ILK */
author
#11 lore raw

Re: [Intel-gfx] [PATCH v7 5/9] drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control

Hi Nirmoy,

Quoted 15 lines · Andrzej Hajda
Quoted 14 lines · Andi Shyti

Enable the CCS_FLUSH bit 13 in the control pipe for render and compute engines in platforms starting from Meteor Lake (BSPEC 43904 and 47112).

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")
Cc Jonathan Cavitt <[email protected]>
Cc Nirmoy Das <[email protected]>
Cc <[email protected]> # v5.8+
---
  drivers/gpu/drm/i915/gt/gen8_engine_cs.c     | 7 +++++++
  drivers/gpu/drm/i915/gt/intel_gpu_commands.h | 1 +
  2 files changed, 8 insertions(+)
Quoted 14 lines

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode) · line 224

Quoted 12 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +6 -0
index 7566c89d9def3..9d050b9a19194 100644
@@ -218,6 +218,13 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
  		bit_group_0 |= PIPE_CONTROL0_HDC_PIPELINE_FLUSH;
+		/*
+		 * When required, in MTL+ platforms we need to
+		 * set the CCS_FLUSH bit in the pipe control
+		 */
+		if (GRAPHICS_VER_FULL(rq->i915) >= IP_VER(12, 70))
+			bit_group_0 |= PIPE_CONTROL_CCS_FLUSH;

BSpec 43904 mentions also other platforms. Why only MTL+?

This is the process of quiescing the engine and that is done in the pipe control sequence.

In the pipe control sequence each engine has its own sequence, even though render and compute overlap on some bits, while the others overlap on other bits.

Besides that MTL+ need this extra bit to be set in the pipe control and that is bit 13 defined as PIPE_CONTROL_CCS_FLUSH.

Thanks,
Andi

#12 lore raw

Re: [Intel-gfx] [PATCH v7 5/9] drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control

In reply to Andi Shyti
Quoted 19 lines · Andi Shyti

Enable the CCS_FLUSH bit 13 in the control pipe for render and compute engines in platforms starting from Meteor Lake (BSPEC 43904 and 47112).

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")
Cc Jonathan Cavitt <[email protected]>
Cc Nirmoy Das <[email protected]>
Cc <[email protected]> # v5.8+
---
  drivers/gpu/drm/i915/gt/gen8_engine_cs.c     | 7 +++++++
  drivers/gpu/drm/i915/gt/intel_gpu_commands.h | 1 +
  2 files changed, 8 insertions(+)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +0 -0
index 7566c89d9def3..9d050b9a19194 100644

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode) · line 227

Quoted 10 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +7 -0
@@ -218,6 +218,13 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
  		bit_group_0 |= PIPE_CONTROL0_HDC_PIPELINE_FLUSH;
 
+		/*
+		 * When required, in MTL+ platforms we need to
+		 * set the CCS_FLUSH bit in the pipe control
+		 */
+		if (GRAPHICS_VER_FULL(rq->i915) >= IP_VER(12, 70))
+			bit_group_0 |= PIPE_CONTROL_CCS_FLUSH;
+

Btw, not for this patch, but related: rcs and ccs have slightly different set of flushes according to bspec but this functions is the same for both. Is it sth we should address, or just safe simplification.

Regards
Andrzej

Quoted 15 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +0 -0
  		bit_group_1 |= PIPE_CONTROL_TILE_CACHE_FLUSH;
  		bit_group_1 |= PIPE_CONTROL_FLUSH_L3;
  		bit_group_1 |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
drivers/gpu/drm/i915/gt/intel_gpu_commands.h +1 -0
index 5d143e2a8db03..5df7cce23197c 100644
@@ -299,6 +299,7 @@
  #define   PIPE_CONTROL_QW_WRITE				(1<<14)
  #define   PIPE_CONTROL_POST_SYNC_OP_MASK                (3<<14)
  #define   PIPE_CONTROL_DEPTH_STALL			(1<<13)
+#define   PIPE_CONTROL_CCS_FLUSH			(1<<13) /* MTL+ */
  #define   PIPE_CONTROL_WRITE_FLUSH			(1<<12)
  #define   PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH	(1<<12) /* gen6+ */
  #define   PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE	(1<<11) /* MBZ on ILK */
author
#13 lore raw

Re: [Intel-gfx] [PATCH v7 5/9] drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control

Hi Andrzej,

Quoted 10 lines
Quoted 10 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +4 -0
index 7566c89d9def3..9d050b9a19194 100644
@@ -218,6 +218,13 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
  		bit_group_0 |= PIPE_CONTROL0_HDC_PIPELINE_FLUSH;
+		/*
+		 * When required, in MTL+ platforms we need to
+		 * set the CCS_FLUSH bit in the pipe control
+		 */
Quoted 8 lines

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode) · line 227

Quoted 3 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +3 -0
+		if (GRAPHICS_VER_FULL(rq->i915) >= IP_VER(12, 70))
+			bit_group_0 |= PIPE_CONTROL_CCS_FLUSH;
+

Btw, not for this patch, but related: rcs and ccs have slightly different set of flushes according to bspec but this functions is the same for both. Is it sth we should address, or just safe simplification.

I guess this is not only used for ccs aux invalidation. I think the BSPEC is specifying the minimum set of bits that need to be set in the pipe control. So that I left it as it is and just added this bit for MTL+.

Thanks,
Andi

#14 lore raw

Re: [v7,5/9] drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control

In reply to Andi Shyti

Hi Andi,

Quoted 5 lines · On Thursday, 20 July 2023 23:07:33 CEST Andi Shyti wrote:

Enable the CCS_FLUSH bit 13 in the control pipe for render and compute engines in platforms starting from Meteor Lake (BSPEC 43904 and 47112).

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")

I'm not sure why you think that your change fixes that commit. Can you please explain?

Thanks,
Janusz

Quoted 33 lines
Cc Jonathan Cavitt <[email protected]>
Cc Nirmoy Das <[email protected]>
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c     | 7 +++++++
 drivers/gpu/drm/i915/gt/intel_gpu_commands.h | 1 +
 2 files changed, 8 insertions(+)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +7 -0
index 7566c89d9def3..9d050b9a19194 100644
@@ -218,6 +218,13 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
 		bit_group_0 |= PIPE_CONTROL0_HDC_PIPELINE_FLUSH;
 
+		/*
+		 * When required, in MTL+ platforms we need to
+		 * set the CCS_FLUSH bit in the pipe control
+		 */
+		if (GRAPHICS_VER_FULL(rq->i915) >= IP_VER(12, 70))
+			bit_group_0 |= PIPE_CONTROL_CCS_FLUSH;
+
 		bit_group_1 |= PIPE_CONTROL_TILE_CACHE_FLUSH;
 		bit_group_1 |= PIPE_CONTROL_FLUSH_L3;
 		bit_group_1 |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH;
drivers/gpu/drm/i915/gt/intel_gpu_commands.h +0 -0
index 5d143e2a8db03..5df7cce23197c 100644

drivers/gpu/drm/i915/gt/intel_gpu_commands.h line 305

Quoted 7 lines
drivers/gpu/drm/i915/gt/intel_gpu_commands.h +1 -0
@@ -299,6 +299,7 @@
 #define   PIPE_CONTROL_QW_WRITE				(1<<14)
 #define   PIPE_CONTROL_POST_SYNC_OP_MASK                (3<<14)
 #define   PIPE_CONTROL_DEPTH_STALL			(1<<13)
+#define   PIPE_CONTROL_CCS_FLUSH			(1<<13) /* MTL+ */
 #define   PIPE_CONTROL_WRITE_FLUSH			(1<<12)
 #define   PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH	(1<<12) /* gen6+ */
 #define   PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE	(1<<11) /* MBZ on ILK */

---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych.

Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.

author
#15 lore raw

Re: [v7,5/9] drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control

Hi Janusz,

Quoted 8 lines
Quoted 5 lines

Enable the CCS_FLUSH bit 13 in the control pipe for render and compute engines in platforms starting from Meteor Lake (BSPEC 43904 and 47112).

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")

I'm not sure why you think that your change fixes that commit. Can you please explain?

Hardware folks have provided a new sequence for performing the quiescing of the engines... that's how it is :)

Thanks,
Andi

author
#16 lore raw

PATCH v7 6/9 drm/i915/gt: Refactor intel_emit_pipe_control_cs() in a single function

In reply to Andi Shyti

Just a trivial refactoring for reducing the number of code duplicate. This will come at handy in the next commits.

---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 44 +++++++++++++-----------
 1 file changed, 23 insertions(+), 21 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +23 -21
index 9d050b9a19194..202d6ff8b5264 100644
@@ -177,23 +177,31 @@ u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv
 	return cs;
 }
 
+static u32 *intel_emit_pipe_control_cs(struct i915_request *rq, u32 bit_group_0,
+				       u32 bit_group_1, u32 offset)
+{
+	u32 *cs;
+
+	cs = intel_ring_begin(rq, 6);
+	if (IS_ERR(cs))
+		return cs;
+
+	cs = gen12_emit_pipe_control(cs, bit_group_0, bit_group_1,
+				     LRC_PPHWSP_SCRATCH_ADDR);
+	intel_ring_advance(rq, cs);
+
+	return cs;
+}
+
 static int mtl_dummy_pipe_control(struct i915_request *rq)
 {
 	/* Wa_14016712196 */
 	if (IS_MTL_GRAPHICS_STEP(rq->engine->i915, M, STEP_A0, STEP_B0) ||
-	    IS_MTL_GRAPHICS_STEP(rq->engine->i915, P, STEP_A0, STEP_B0)) {
-		u32 *cs;
-
-		/* dummy PIPE_CONTROL + depth flush */
-		cs = intel_ring_begin(rq, 6);
-		if (IS_ERR(cs))
-			return PTR_ERR(cs);
-		cs = gen12_emit_pipe_control(cs,
-					     0,
-					     PIPE_CONTROL_DEPTH_CACHE_FLUSH,
-					     LRC_PPHWSP_SCRATCH_ADDR);
-		intel_ring_advance(rq, cs);
-	}
+	    IS_MTL_GRAPHICS_STEP(rq->engine->i915, P, STEP_A0, STEP_B0))
+		intel_emit_pipe_control_cs(rq,
+					   0,
+					   PIPE_CONTROL_DEPTH_CACHE_FLUSH,
+					   LRC_PPHWSP_SCRATCH_ADDR);
 
 	return 0;
 }
@@ -210,7 +218,6 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 		u32 bit_group_0 = 0;
 		u32 bit_group_1 = 0;
 		int err;
-		u32 *cs;
 
 		err = mtl_dummy_pipe_control(rq);
 		if (err)
@@ -244,13 +251,8 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 		else if (engine->class == COMPUTE_CLASS)
 			bit_group_1 &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
 
-		cs = intel_ring_begin(rq, 6);
-		if (IS_ERR(cs))
-			return PTR_ERR(cs);
-
-		cs = gen12_emit_pipe_control(cs, bit_group_0, bit_group_1,
-					     LRC_PPHWSP_SCRATCH_ADDR);
-		intel_ring_advance(rq, cs);
+		intel_emit_pipe_control_cs(rq, bit_group_0, bit_group_1,
+					   LRC_PPHWSP_SCRATCH_ADDR);
 	}
 
 	if (mode & EMIT_INVALIDATE) {
Signature · 1 line
2.40.1
#17 lore raw

Re: [Intel-gfx] [PATCH v7 6/9] drm/i915/gt: Refactor intel_emit_pipe_control_cs() in a single function

Quoted 14 lines · Andi Shyti

Just a trivial refactoring for reducing the number of code duplicate. This will come at handy in the next commits.

---
  drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 44 +++++++++++++-----------
  1 file changed, 23 insertions(+), 21 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +0 -0
index 9d050b9a19194..202d6ff8b5264 100644

drivers/gpu/drm/i915/gt/gen8_engine_cs.c u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv · line 181

Quoted 5 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +2 -0
@@ -177,23 +177,31 @@ u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv
  	return cs;
  }
 
+static u32 *intel_emit_pipe_control_cs(struct i915_request *rq, u32 bit_group_0,
+				       u32 bit_group_1, u32 offset)

s/intel/gen12/

but this and few other issues were raised already by Matt in v6.

Regards
Andrzej

Quoted 63 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +21 -21
+{
+	u32 *cs;
+
+	cs = intel_ring_begin(rq, 6);
+	if (IS_ERR(cs))
+		return cs;
+
+	cs = gen12_emit_pipe_control(cs, bit_group_0, bit_group_1,
+				     LRC_PPHWSP_SCRATCH_ADDR);
+	intel_ring_advance(rq, cs);
+
+	return cs;
+}
+
  static int mtl_dummy_pipe_control(struct i915_request *rq)
  {
  	/* Wa_14016712196 */
  	if (IS_MTL_GRAPHICS_STEP(rq->engine->i915, M, STEP_A0, STEP_B0) ||
-	    IS_MTL_GRAPHICS_STEP(rq->engine->i915, P, STEP_A0, STEP_B0)) {
-		u32 *cs;
-
-		/* dummy PIPE_CONTROL + depth flush */
-		cs = intel_ring_begin(rq, 6);
-		if (IS_ERR(cs))
-			return PTR_ERR(cs);
-		cs = gen12_emit_pipe_control(cs,
-					     0,
-					     PIPE_CONTROL_DEPTH_CACHE_FLUSH,
-					     LRC_PPHWSP_SCRATCH_ADDR);
-		intel_ring_advance(rq, cs);
-	}
+	    IS_MTL_GRAPHICS_STEP(rq->engine->i915, P, STEP_A0, STEP_B0))
+		intel_emit_pipe_control_cs(rq,
+					   0,
+					   PIPE_CONTROL_DEPTH_CACHE_FLUSH,
+					   LRC_PPHWSP_SCRATCH_ADDR);
 
  	return 0;
  }
@@ -210,7 +218,6 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
  		u32 bit_group_0 = 0;
  		u32 bit_group_1 = 0;
  		int err;
-		u32 *cs;
 
  		err = mtl_dummy_pipe_control(rq);
  		if (err)
@@ -244,13 +251,8 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
  		else if (engine->class == COMPUTE_CLASS)
  			bit_group_1 &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
 
-		cs = intel_ring_begin(rq, 6);
-		if (IS_ERR(cs))
-			return PTR_ERR(cs);
-
-		cs = gen12_emit_pipe_control(cs, bit_group_0, bit_group_1,
-					     LRC_PPHWSP_SCRATCH_ADDR);
-		intel_ring_advance(rq, cs);
+		intel_emit_pipe_control_cs(rq, bit_group_0, bit_group_1,
+					   LRC_PPHWSP_SCRATCH_ADDR);
  	}
 
  	if (mode & EMIT_INVALIDATE) {
author
#18 lore raw

Re: [Intel-gfx] [PATCH v7 6/9] drm/i915/gt: Refactor intel_emit_pipe_control_cs() in a single function

Quoted 23 lines · Andrzej Hajda
Quoted 14 lines · Andi Shyti

Just a trivial refactoring for reducing the number of code duplicate. This will come at handy in the next commits.

---
  drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 44 +++++++++++++-----------
  1 file changed, 23 insertions(+), 21 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +0 -0
index 9d050b9a19194..202d6ff8b5264 100644

drivers/gpu/drm/i915/gt/gen8_engine_cs.c u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv · line 180

Quoted 4 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +2 -0
@@ -177,23 +177,31 @@ u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv
  	return cs;
  }
+static u32 *intel_emit_pipe_control_cs(struct i915_request *rq, u32 bit_group_0,
+				       u32 bit_group_1, u32 offset)

s/intel/gen12/

Quoted 1 line

but this and few other issues were raised already by Matt in v6.

Thanks!
Andi

Reviewed-by Janusz Krzysztofik on v7 6/9
#19 lore raw

Re: [v7, 6/9] drm/i915/gt: Refactor intel_emit_pipe_control_cs() in a single function

In reply to Andi Shyti

Hi Andi,

Quoted 31 lines · On Thursday, 20 July 2023 23:07:34 CEST Andi Shyti wrote:

Just a trivial refactoring for reducing the number of code duplicate. This will come at handy in the next commits.

---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 44 +++++++++++++-----------
 1 file changed, 23 insertions(+), 21 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +14 -0
index 9d050b9a19194..202d6ff8b5264 100644
@@ -177,23 +177,31 @@ u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv
 	return cs;
 }
 
+static u32 *intel_emit_pipe_control_cs(struct i915_request *rq, u32 bit_group_0,
+				       u32 bit_group_1, u32 offset)
+{
+	u32 *cs;
+
+	cs = intel_ring_begin(rq, 6);
+	if (IS_ERR(cs))
+		return cs;
+
+	cs = gen12_emit_pipe_control(cs, bit_group_0, bit_group_1,
+				     LRC_PPHWSP_SCRATCH_ADDR);
+	intel_ring_advance(rq, cs);
+
+	return cs;

drivers/gpu/drm/i915/gt/gen8_engine_cs.c u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv · line 187

Quoted 10 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +2 -4
+}
+
 static int mtl_dummy_pipe_control(struct i915_request *rq)
 {
 	/* Wa_14016712196 */
 	if (IS_MTL_GRAPHICS_STEP(rq->engine->i915, M, STEP_A0, STEP_B0) ||
-	    IS_MTL_GRAPHICS_STEP(rq->engine->i915, P, STEP_A0, STEP_B0)) {
-		u32 *cs;
-
-		/* dummy PIPE_CONTROL + depth flush */

NIT: Maybe you could keep that comment, above intel_emit_pipe_control_cs(...).

Anyway, LGTM.

Quoted 32 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +5 -12
-		cs = intel_ring_begin(rq, 6);
-		if (IS_ERR(cs))
-			return PTR_ERR(cs);
-		cs = gen12_emit_pipe_control(cs,
-					     0,
-					     PIPE_CONTROL_DEPTH_CACHE_FLUSH,
-					     LRC_PPHWSP_SCRATCH_ADDR);
-		intel_ring_advance(rq, cs);
-	}
+	    IS_MTL_GRAPHICS_STEP(rq->engine->i915, P, STEP_A0, STEP_B0))
+		intel_emit_pipe_control_cs(rq,
+					   0,
+					   PIPE_CONTROL_DEPTH_CACHE_FLUSH,
+					   LRC_PPHWSP_SCRATCH_ADDR);
 
 	return 0;
 }
@@ -210,7 +218,6 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 		u32 bit_group_0 = 0;
 		u32 bit_group_1 = 0;
 		int err;
-		u32 *cs;
 
 		err = mtl_dummy_pipe_control(rq);
 		if (err)
@@ -244,13 +251,8 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 		else if (engine->class == COMPUTE_CLASS)
 			bit_group_1 &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
 
-		cs = intel_ring_begin(rq, 6);
-		if (IS_ERR(cs))

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode) · line 258

Quoted 10 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +2 -5
-			return PTR_ERR(cs);
-
-		cs = gen12_emit_pipe_control(cs, bit_group_0, bit_group_1,
-					     LRC_PPHWSP_SCRATCH_ADDR);
-		intel_ring_advance(rq, cs);
+		intel_emit_pipe_control_cs(rq, bit_group_0, bit_group_1,
+					   LRC_PPHWSP_SCRATCH_ADDR);
 	}
 
 	if (mode & EMIT_INVALIDATE) {

---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych.

Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.

author
#20 lore raw

PATCH v7 7/9 drm/i915/gt: Ensure memory quiesced before invalidation for all engines

In reply to Andi Shyti

Commit af9e423a8aae ("drm/i915/gt: Ensure memory quiesced before invalidation") has made sure that the memory is quiesced before invalidating the AUX CCS table. Do it for all the other engines and not just RCS.

Cc Jonathan Cavitt <[email protected]>
Cc Matt Roper <[email protected]>
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 36 ++++++++++++++++--------
 1 file changed, 25 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +25 -11
index 202d6ff8b5264..b6dd22eb2d9b2 100644
@@ -316,26 +316,40 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 {
 	intel_engine_mask_t aux_inv = 0;
-	u32 cmd, *cs;
+	u32 cmd_flush = 0;
+	u32 cmd = 4;
+	u32 *cs;
 
-	cmd = 4;
-	if (mode & EMIT_INVALIDATE) {
+	if (mode & EMIT_INVALIDATE)
 		cmd += 2;
 
-		if (HAS_AUX_CCS(rq->engine->i915) &&
-		    (rq->engine->class == VIDEO_DECODE_CLASS ||
-		     rq->engine->class == VIDEO_ENHANCEMENT_CLASS)) {
-			aux_inv = rq->engine->mask &
-				~GENMASK(_BCS(I915_MAX_BCS - 1), BCS0);
-			if (aux_inv)
-				cmd += 4;
-		}
+	if (HAS_AUX_CCS(rq->engine->i915))
+		aux_inv = rq->engine->mask &
+			  ~GENMASK(_BCS(I915_MAX_BCS - 1), BCS0);
+
+	/*
+	 * On Aux CCS platforms the invalidation of the Aux
+	 * table requires quiescing memory traffic beforehand
+	 */
+	if (aux_inv) {
+		cmd += 4; /* for the AUX invalidation */
+		cmd += 2; /* for the engine quiescing */
+
+		cmd_flush = MI_FLUSH_DW;
+
+		if (rq->engine->class == COPY_ENGINE_CLASS)
+			cmd_flush |= MI_FLUSH_DW_CCS;
 	}
 
 	cs = intel_ring_begin(rq, cmd);
 	if (IS_ERR(cs))
 		return PTR_ERR(cs);
 
+	if (cmd_flush) {
+		*cs++ = cmd_flush;
+		*cs++ = 0;
+	}
+
 	if (mode & EMIT_INVALIDATE)
 		*cs++ = preparser_disable(true);
 
Signature · 1 line
2.40.1
#21 lore raw

Re: [v7, 7/9] drm/i915/gt: Ensure memory quiesced before invalidation for all engines

Hi Andi,

Quoted 1 line · On Thursday, 20 July 2023 23:07:35 CEST Andi Shyti wrote:

Commit af9e423a8aae

You can't use this commit ID, it is invalid (the patch you are referring to belongs to your series, then is not available in any official repository, hence no stable commit ID yet).

Quoted 4 lines

("drm/i915/gt: Ensure memory quiesced before invalidation") has made sure that the memory is quiesced before invalidating the AUX CCS table. Do it for all the other engines and not just RCS.

Why do we need that now for other engines, while that former patch seemed to suggest we didn't?

Thanks,
Janusz

Quoted 56 lines
Cc Jonathan Cavitt <[email protected]>
Cc Matt Roper <[email protected]>
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 36 ++++++++++++++++--------
 1 file changed, 25 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +20 -11
index 202d6ff8b5264..b6dd22eb2d9b2 100644
@@ -316,26 +316,40 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 {
 	intel_engine_mask_t aux_inv = 0;
-	u32 cmd, *cs;
+	u32 cmd_flush = 0;
+	u32 cmd = 4;
+	u32 *cs;
 
-	cmd = 4;
-	if (mode & EMIT_INVALIDATE) {
+	if (mode & EMIT_INVALIDATE)
 		cmd += 2;
 
-		if (HAS_AUX_CCS(rq->engine->i915) &&
-		    (rq->engine->class == VIDEO_DECODE_CLASS ||
-		     rq->engine->class == VIDEO_ENHANCEMENT_CLASS)) {
-			aux_inv = rq->engine->mask &
-				~GENMASK(_BCS(I915_MAX_BCS - 1), BCS0);
-			if (aux_inv)
-				cmd += 4;
-		}
+	if (HAS_AUX_CCS(rq->engine->i915))
+		aux_inv = rq->engine->mask &
+			  ~GENMASK(_BCS(I915_MAX_BCS - 1), BCS0);
+
+	/*
+	 * On Aux CCS platforms the invalidation of the Aux
+	 * table requires quiescing memory traffic beforehand
+	 */
+	if (aux_inv) {
+		cmd += 4; /* for the AUX invalidation */
+		cmd += 2; /* for the engine quiescing */
+
+		cmd_flush = MI_FLUSH_DW;
+
+		if (rq->engine->class == COPY_ENGINE_CLASS)
+			cmd_flush |= MI_FLUSH_DW_CCS;
 	}
 
 	cs = intel_ring_begin(rq, cmd);
 	if (IS_ERR(cs))

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode) · line 355

Quoted 10 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +5 -0
 		return PTR_ERR(cs);
 
+	if (cmd_flush) {
+		*cs++ = cmd_flush;
+		*cs++ = 0;
+	}
+
 	if (mode & EMIT_INVALIDATE)
 		*cs++ = preparser_disable(true);
 

---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych.

Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.

author
#22 lore raw

Re: [v7, 7/9] drm/i915/gt: Ensure memory quiesced before invalidation for all engines

Hi Janusz,

Quoted 8 lines · Janusz

Hi Andi,

Quoted 1 line · On Thursday, 20 July 2023 23:07:35 CEST Andi Shyti wrote:

Commit af9e423a8aae

You can't use this commit ID, it is invalid (the patch you are referring to belongs to your series, then is not available in any official repository, hence no stable commit ID yet).

yes, I need to update it, I know... this has changed at every revision, but I was lazy and decided to do it at the end after the whole review process was done. I didn't think that anyone would go and check it :-D

It's good to know that there is a thorough review here! Thanks!

Quoted 7 lines
Quoted 4 lines

("drm/i915/gt: Ensure memory quiesced before invalidation") has made sure that the memory is quiesced before invalidating the AUX CCS table. Do it for all the other engines and not just RCS.

Why do we need that now for other engines, while that former patch seemed to suggest we didn't?

This whole work started from a a couple of patches from Jonathan and slowly exploded in this series of 9 patches. I tried to preserve his work and just add things around them.

What Jonathan originally did was to add aux invalidation support only for RCS and Compute engines, but then hardware guys updated the docs asking to do it for basically all the engines.

Thank you,
Andi

author
#23 lore raw

PATCH v7 8/9 drm/i915/gt: Poll aux invalidation register bit on invalidation

In reply to Andi Shyti

From: Jonathan Cavitt <[email protected]>

For platforms that use Aux CCS, wait for aux invalidation to complete by checking the aux invalidation register bit is cleared.

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c     | 17 ++++++++++++-----
 drivers/gpu/drm/i915/gt/intel_gpu_commands.h |  1 +
 2 files changed, 13 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +12 -5
index b6dd22eb2d9b2..3ded597f002a2 100644
@@ -172,7 +172,15 @@ u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv
 	*cs++ = MI_LOAD_REGISTER_IMM(1) | MI_LRI_MMIO_REMAP_EN;
 	*cs++ = i915_mmio_reg_offset(inv_reg) + gsi_offset;
 	*cs++ = AUX_INV;
-	*cs++ = MI_NOOP;
+
+	*cs++ = MI_SEMAPHORE_WAIT_TOKEN |
+		MI_SEMAPHORE_REGISTER_POLL |
+		MI_SEMAPHORE_POLL |
+		MI_SEMAPHORE_SAD_EQ_SDD;
+	*cs++ = 0;
+	*cs++ = i915_mmio_reg_offset(inv_reg) + gsi_offset;
+	*cs++ = 0;
+	*cs++ = 0;
 
 	return cs;
 }
@@ -282,10 +290,9 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 		else if (engine->class == COMPUTE_CLASS)
 			flags &= ~PIPE_CONTROL_3D_ENGINE_FLAGS;
 
+		count = 8;
 		if (HAS_AUX_CCS(rq->engine->i915))
-			count = 8 + 4;
-		else
-			count = 8;
+			count += 8;
 
 		cs = intel_ring_begin(rq, count);
 		if (IS_ERR(cs))
@@ -332,7 +339,7 @@ int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 	 * table requires quiescing memory traffic beforehand
 	 */
 	if (aux_inv) {
-		cmd += 4; /* for the AUX invalidation */
+		cmd += 8; /* for the AUX invalidation */
 		cmd += 2; /* for the engine quiescing */
 
 		cmd_flush = MI_FLUSH_DW;
drivers/gpu/drm/i915/gt/intel_gpu_commands.h +1 -0
index 5df7cce23197c..2bd8d98d21102 100644
@@ -121,6 +121,7 @@
 #define   MI_SEMAPHORE_TARGET(engine)	((engine)<<15)
 #define MI_SEMAPHORE_WAIT	MI_INSTR(0x1c, 2) /* GEN8+ */
 #define MI_SEMAPHORE_WAIT_TOKEN	MI_INSTR(0x1c, 3) /* GEN12+ */
+#define   MI_SEMAPHORE_REGISTER_POLL	(1 << 16)
 #define   MI_SEMAPHORE_POLL		(1 << 15)
 #define   MI_SEMAPHORE_SAD_GT_SDD	(0 << 12)
 #define   MI_SEMAPHORE_SAD_GTE_SDD	(1 << 12)
Signature · 1 line
2.40.1
author
#24 lore raw

PATCH v7 9/9 drm/i915/gt: Support aux invalidation on all engines

In reply to Andi Shyti

Perform some refactoring with the purpose of keeping in one single place all the operations around the aux table invalidation.

With this refactoring add more engines where the invalidation should be performed.

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")
Cc Jonathan Cavitt <[email protected]>
Cc Matt Roper <[email protected]>
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 58 +++++++++++++++---------
 drivers/gpu/drm/i915/gt/gen8_engine_cs.h |  3 +-
 drivers/gpu/drm/i915/gt/intel_lrc.c      | 17 +------
 3 files changed, 41 insertions(+), 37 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +37 -21
index 3ded597f002a2..30fb4e0af6134 100644
@@ -165,9 +165,36 @@ static u32 preparser_disable(bool state)
 	return MI_ARB_CHECK | 1 << 8 | state;
 }
 
-u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv_reg)
+static i915_reg_t gen12_get_aux_inv_reg(struct intel_engine_cs *engine)
 {
-	u32 gsi_offset = gt->uncore->gsi_offset;
+	if (!HAS_AUX_CCS(engine->i915))
+		return INVALID_MMIO_REG;
+
+	switch (engine->id) {
+	case RCS0:
+		return GEN12_CCS_AUX_INV;
+	case BCS0:
+		return GEN12_BCS0_AUX_INV;
+	case VCS0:
+		return GEN12_VD0_AUX_INV;
+	case VCS2:
+		return GEN12_VD2_AUX_INV;
+	case VECS0:
+		return GEN12_VE0_AUX_INV;
+	case CCS0:
+		return GEN12_CCS0_AUX_INV;
+	default:
+		return INVALID_MMIO_REG;
+	}
+}
+
+u32 *gen12_emit_aux_table_inv(struct intel_engine_cs *engine, u32 *cs)
+{
+	i915_reg_t inv_reg = gen12_get_aux_inv_reg(engine);
+	u32 gsi_offset = engine->gt->uncore->gsi_offset;
+
+	if (i915_mmio_reg_valid(inv_reg))
+		return cs;
 
 	*cs++ = MI_LOAD_REGISTER_IMM(1) | MI_LRI_MMIO_REMAP_EN;
 	*cs++ = i915_mmio_reg_offset(inv_reg) + gsi_offset;
@@ -201,6 +228,11 @@ static u32 *intel_emit_pipe_control_cs(struct i915_request *rq, u32 bit_group_0,
 	return cs;
 }
 
+static bool gen12_engine_has_aux_inv(struct intel_engine_cs *engine)
+{
+	return i915_mmio_reg_valid(gen12_get_aux_inv_reg(engine));
+}
+
 static int mtl_dummy_pipe_control(struct i915_request *rq)
 {
 	/* Wa_14016712196 */
@@ -307,11 +339,7 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
 		cs = gen8_emit_pipe_control(cs, flags, LRC_PPHWSP_SCRATCH_ADDR);
 
-		if (!HAS_FLAT_CCS(rq->engine->i915)) {
-			/* hsdes: 1809175790 */
-			cs = gen12_emit_aux_table_inv(rq->engine->gt, cs,
-						      GEN12_CCS_AUX_INV);
-		}
+		cs = gen12_emit_aux_table_inv(engine, cs);
 
 		*cs++ = preparser_disable(false);
 		intel_ring_advance(rq, cs);
@@ -322,7 +350,6 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
 int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 {
-	intel_engine_mask_t aux_inv = 0;
 	u32 cmd_flush = 0;
 	u32 cmd = 4;
 	u32 *cs;
@@ -330,15 +357,11 @@ int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 	if (mode & EMIT_INVALIDATE)
 		cmd += 2;
 
-	if (HAS_AUX_CCS(rq->engine->i915))
-		aux_inv = rq->engine->mask &
-			  ~GENMASK(_BCS(I915_MAX_BCS - 1), BCS0);
-
 	/*
 	 * On Aux CCS platforms the invalidation of the Aux
 	 * table requires quiescing memory traffic beforehand
 	 */
-	if (aux_inv) {
+	if (gen12_engine_has_aux_inv(rq->engine)) {
 		cmd += 8; /* for the AUX invalidation */
 		cmd += 2; /* for the engine quiescing */
 
@@ -381,14 +404,7 @@ int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 	*cs++ = 0; /* upper addr */
 	*cs++ = 0; /* value */
 
-	if (aux_inv) { /* hsdes: 1809175790 */
-		if (rq->engine->class == VIDEO_DECODE_CLASS)
-			cs = gen12_emit_aux_table_inv(rq->engine->gt,
-						      cs, GEN12_VD0_AUX_INV);
-		else
-			cs = gen12_emit_aux_table_inv(rq->engine->gt,
-						      cs, GEN12_VE0_AUX_INV);
-	}
+	cs = gen12_emit_aux_table_inv(rq->engine, cs);
 
 	if (mode & EMIT_INVALIDATE)
 		*cs++ = preparser_disable(false);
drivers/gpu/drm/i915/gt/gen8_engine_cs.h +2 -1
index a44eda096557c..867ba697aceb8 100644
@@ -13,6 +13,7 @@
 #include "intel_gt_regs.h"
 #include "intel_gpu_commands.h"
 
+struct intel_engine_cs;
 struct intel_gt;
 struct i915_request;
 
@@ -46,7 +47,7 @@ u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs);
 u32 *gen11_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs);
 u32 *gen12_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs);
 
-u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv_reg);
+u32 *gen12_emit_aux_table_inv(struct intel_engine_cs *engine, u32 *cs);
 
 static inline u32 *
 __gen8_emit_pipe_control(u32 *batch, u32 bit_group_0,
drivers/gpu/drm/i915/gt/intel_lrc.c +2 -15
index 235f3fab60a98..119deb9f938c7 100644
@@ -1371,10 +1371,7 @@ gen12_emit_indirect_ctx_rcs(const struct intel_context *ce, u32 *cs)
 	    IS_DG2_G11(ce->engine->i915))
 		cs = gen8_emit_pipe_control(cs, PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE, 0);
 
-	/* hsdes: 1809175790 */
-	if (!HAS_FLAT_CCS(ce->engine->i915))
-		cs = gen12_emit_aux_table_inv(ce->engine->gt,
-					      cs, GEN12_CCS_AUX_INV);
+	cs = gen12_emit_aux_table_inv(ce->engine, cs);
 
 	/* Wa_16014892111 */
 	if (IS_MTL_GRAPHICS_STEP(ce->engine->i915, M, STEP_A0, STEP_B0) ||
@@ -1399,17 +1396,7 @@ gen12_emit_indirect_ctx_xcs(const struct intel_context *ce, u32 *cs)
 						    PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE,
 						    0);
 
-	/* hsdes: 1809175790 */
-	if (!HAS_FLAT_CCS(ce->engine->i915)) {
-		if (ce->engine->class == VIDEO_DECODE_CLASS)
-			cs = gen12_emit_aux_table_inv(ce->engine->gt,
-						      cs, GEN12_VD0_AUX_INV);
-		else if (ce->engine->class == VIDEO_ENHANCEMENT_CLASS)
-			cs = gen12_emit_aux_table_inv(ce->engine->gt,
-						      cs, GEN12_VE0_AUX_INV);
-	}
-
-	return cs;
+	return gen12_emit_aux_table_inv(ce->engine, cs);
 }
 
 static void
Signature · 1 line
2.40.1
#25 lore raw

Re: [v7,9/9] drm/i915/gt: Support aux invalidation on all engines

Hi Andi,

Quoted 48 lines · On Thursday, 20 July 2023 23:07:37 CEST Andi Shyti wrote:

Perform some refactoring with the purpose of keeping in one single place all the operations around the aux table invalidation.

With this refactoring add more engines where the invalidation should be performed.

Fixes 972282c4cf24 ("drm/i915/gen12: Add aux table invalidate for all engines")
Cc Jonathan Cavitt <[email protected]>
Cc Matt Roper <[email protected]>
Cc <[email protected]> # v5.8+
---
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c | 58 +++++++++++++++---------
 drivers/gpu/drm/i915/gt/gen8_engine_cs.h |  3 +-
 drivers/gpu/drm/i915/gt/intel_lrc.c      | 17 +------
 3 files changed, 41 insertions(+), 37 deletions(-)
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +19 -2
index 3ded597f002a2..30fb4e0af6134 100644
@@ -165,9 +165,36 @@ static u32 preparser_disable(bool state)
 	return MI_ARB_CHECK | 1 << 8 | state;
 }
 
-u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv_reg)
+static i915_reg_t gen12_get_aux_inv_reg(struct intel_engine_cs *engine)
 {
-	u32 gsi_offset = gt->uncore->gsi_offset;
+	if (!HAS_AUX_CCS(engine->i915))
+		return INVALID_MMIO_REG;
+
+	switch (engine->id) {
+	case RCS0:
+		return GEN12_CCS_AUX_INV;
+	case BCS0:
+		return GEN12_BCS0_AUX_INV;
+	case VCS0:
+		return GEN12_VD0_AUX_INV;
+	case VCS2:
+		return GEN12_VD2_AUX_INV;
+	case VECS0:
+		return GEN12_VE0_AUX_INV;
+	case CCS0:
+		return GEN12_CCS0_AUX_INV;
+	default:
+		return INVALID_MMIO_REG;

drivers/gpu/drm/i915/gt/gen8_engine_cs.c static u32 preparser_disable(bool state) · line 197

Quoted 10 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +10 -0
+	}
+}
+
+u32 *gen12_emit_aux_table_inv(struct intel_engine_cs *engine, u32 *cs)
+{
+	i915_reg_t inv_reg = gen12_get_aux_inv_reg(engine);
+	u32 gsi_offset = engine->gt->uncore->gsi_offset;
+
+	if (i915_mmio_reg_valid(inv_reg))
+		return cs;

Is that correct? Now the original body of gen12_emit_aux_table_inv() will be executed only if either (!HAS_AUX_CCS(engine->i915) or the engine is not one of (RCS0, BCS0, VCS0, VCS2 or CCS0), ...

Quoted 16 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +5 -0
 	*cs++ = MI_LOAD_REGISTER_IMM(1) | MI_LRI_MMIO_REMAP_EN;
 	*cs++ = i915_mmio_reg_offset(inv_reg) + gsi_offset;
@@ -201,6 +228,11 @@ static u32 *intel_emit_pipe_control_cs(struct i915_request *rq, u32 bit_group_0,
 	return cs;
 }
 
+static bool gen12_engine_has_aux_inv(struct intel_engine_cs *engine)
+{
+	return i915_mmio_reg_valid(gen12_get_aux_inv_reg(engine));
+}
+
 static int mtl_dummy_pipe_control(struct i915_request *rq)
 {
 	/* Wa_14016712196 */

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode) · line 310

Quoted 4 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +0 -1
@@ -307,11 +339,7 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
 		cs = gen8_emit_pipe_control(cs, flags, LRC_PPHWSP_SCRATCH_ADDR);
 
-		if (!HAS_FLAT_CCS(rq->engine->i915)) {

... while before it was executed only if (!HAS_FLAT_CCS(rq->engine->i915)), which, according to commit description of PATCH 2/9, rather had the opposite meaning. Am I missing something?

Thanks,
Janusz

Quoted 96 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +3 -18
-			/* hsdes: 1809175790 */
-			cs = gen12_emit_aux_table_inv(rq->engine->gt, cs,
-						      GEN12_CCS_AUX_INV);
-		}
+		cs = gen12_emit_aux_table_inv(engine, cs);
 
 		*cs++ = preparser_disable(false);
 		intel_ring_advance(rq, cs);
@@ -322,7 +350,6 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
 int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 {
-	intel_engine_mask_t aux_inv = 0;
 	u32 cmd_flush = 0;
 	u32 cmd = 4;
 	u32 *cs;
@@ -330,15 +357,11 @@ int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 	if (mode & EMIT_INVALIDATE)
 		cmd += 2;
 
-	if (HAS_AUX_CCS(rq->engine->i915))
-		aux_inv = rq->engine->mask &
-			  ~GENMASK(_BCS(I915_MAX_BCS - 1), BCS0);
-
 	/*
 	 * On Aux CCS platforms the invalidation of the Aux
 	 * table requires quiescing memory traffic beforehand
 	 */
-	if (aux_inv) {
+	if (gen12_engine_has_aux_inv(rq->engine)) {
 		cmd += 8; /* for the AUX invalidation */
 		cmd += 2; /* for the engine quiescing */
 
@@ -381,14 +404,7 @@ int gen12_emit_flush_xcs(struct i915_request *rq, u32 mode)
 	*cs++ = 0; /* upper addr */
 	*cs++ = 0; /* value */
 
-	if (aux_inv) { /* hsdes: 1809175790 */
-		if (rq->engine->class == VIDEO_DECODE_CLASS)
-			cs = gen12_emit_aux_table_inv(rq->engine->gt,
-						      cs, GEN12_VD0_AUX_INV);
-		else
-			cs = gen12_emit_aux_table_inv(rq->engine->gt,
-						      cs, GEN12_VE0_AUX_INV);
-	}
+	cs = gen12_emit_aux_table_inv(rq->engine, cs);
 
 	if (mode & EMIT_INVALIDATE)
 		*cs++ = preparser_disable(false);
drivers/gpu/drm/i915/gt/gen8_engine_cs.h +2 -1
index a44eda096557c..867ba697aceb8 100644
@@ -13,6 +13,7 @@
 #include "intel_gt_regs.h"
 #include "intel_gpu_commands.h"
 
+struct intel_engine_cs;
 struct intel_gt;
 struct i915_request;
 
@@ -46,7 +47,7 @@ u32 *gen8_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs);
 u32 *gen11_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs);
 u32 *gen12_emit_fini_breadcrumb_rcs(struct i915_request *rq, u32 *cs);
 
-u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv_reg);
+u32 *gen12_emit_aux_table_inv(struct intel_engine_cs *engine, u32 *cs);
 
 static inline u32 *
 __gen8_emit_pipe_control(u32 *batch, u32 bit_group_0,
drivers/gpu/drm/i915/gt/intel_lrc.c +1 -9
index 235f3fab60a98..119deb9f938c7 100644
@@ -1371,10 +1371,7 @@ gen12_emit_indirect_ctx_rcs(const struct intel_context *ce, u32 *cs)
 	    IS_DG2_G11(ce->engine->i915))
 		cs = gen8_emit_pipe_control(cs, PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE, 0);
 
-	/* hsdes: 1809175790 */
-	if (!HAS_FLAT_CCS(ce->engine->i915))
-		cs = gen12_emit_aux_table_inv(ce->engine->gt,
-					      cs, GEN12_CCS_AUX_INV);
+	cs = gen12_emit_aux_table_inv(ce->engine, cs);
 
 	/* Wa_16014892111 */
 	if (IS_MTL_GRAPHICS_STEP(ce->engine->i915, M, STEP_A0, STEP_B0) ||
@@ -1399,17 +1396,7 @@ gen12_emit_indirect_ctx_xcs(const struct intel_context *ce, u32 *cs)
 						    PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE,
 						    0);
 
-	/* hsdes: 1809175790 */
-	if (!HAS_FLAT_CCS(ce->engine->i915)) {
-		if (ce->engine->class == VIDEO_DECODE_CLASS)
-			cs = gen12_emit_aux_table_inv(ce->engine->gt,
-						      cs, GEN12_VD0_AUX_INV);

drivers/gpu/drm/i915/gt/intel_lrc.c gen12_emit_indirect_ctx_xcs(const struct intel_context *ce, u32 *cs) · line 1402

Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_lrc.c +1 -6
-		else if (ce->engine->class == VIDEO_ENHANCEMENT_CLASS)
-			cs = gen12_emit_aux_table_inv(ce->engine->gt,
-						      cs, GEN12_VE0_AUX_INV);
-	}
-
-	return cs;
+	return gen12_emit_aux_table_inv(ce->engine, cs);
 }
 
 static void

---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych.

Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.

author
#26 lore raw

Re: [v7,9/9] drm/i915/gt: Support aux invalidation on all engines

Hi Janusz,

Quoted 62 lines
Quoted 30 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +19 -2
index 3ded597f002a2..30fb4e0af6134 100644
@@ -165,9 +165,36 @@ static u32 preparser_disable(bool state)
 	return MI_ARB_CHECK | 1 << 8 | state;
 }
 
-u32 *gen12_emit_aux_table_inv(struct intel_gt *gt, u32 *cs, const i915_reg_t inv_reg)
+static i915_reg_t gen12_get_aux_inv_reg(struct intel_engine_cs *engine)
 {
-	u32 gsi_offset = gt->uncore->gsi_offset;
+	if (!HAS_AUX_CCS(engine->i915))
+		return INVALID_MMIO_REG;
+
+	switch (engine->id) {
+	case RCS0:
+		return GEN12_CCS_AUX_INV;
+	case BCS0:
+		return GEN12_BCS0_AUX_INV;
+	case VCS0:
+		return GEN12_VD0_AUX_INV;
+	case VCS2:
+		return GEN12_VD2_AUX_INV;
+	case VECS0:
+		return GEN12_VE0_AUX_INV;
+	case CCS0:
+		return GEN12_CCS0_AUX_INV;
+	default:
+		return INVALID_MMIO_REG;

drivers/gpu/drm/i915/gt/gen8_engine_cs.c static u32 preparser_disable(bool state) · line 197

Quoted 10 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +10 -0
+	}
+}
+
+u32 *gen12_emit_aux_table_inv(struct intel_engine_cs *engine, u32 *cs)
+{
+	i915_reg_t inv_reg = gen12_get_aux_inv_reg(engine);
+	u32 gsi_offset = engine->gt->uncore->gsi_offset;
+
+	if (i915_mmio_reg_valid(inv_reg))
+		return cs;

Is that correct? Now the original body of gen12_emit_aux_table_inv() will be executed only if either (!HAS_AUX_CCS(engine->i915) or the engine is not one of (RCS0, BCS0, VCS0, VCS2 or CCS0), ...

Quoted 17 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +5 -0
 	*cs++ = MI_LOAD_REGISTER_IMM(1) | MI_LRI_MMIO_REMAP_EN;
 	*cs++ = i915_mmio_reg_offset(inv_reg) + gsi_offset;
@@ -201,6 +228,11 @@ static u32 *intel_emit_pipe_control_cs(struct i915_request *rq, u32 bit_group_0,
 	return cs;
 }
 
+static bool gen12_engine_has_aux_inv(struct intel_engine_cs *engine)
+{
+	return i915_mmio_reg_valid(gen12_get_aux_inv_reg(engine));
+}
+
 static int mtl_dummy_pipe_control(struct i915_request *rq)
 {
 	/* Wa_14016712196 */
@@ -307,11 +339,7 @@ int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode)
 
Quoted 7 lines

drivers/gpu/drm/i915/gt/gen8_engine_cs.c int gen12_emit_flush_rcs(struct i915_request *rq, u32 mode) · line 310

Quoted 3 lines
drivers/gpu/drm/i915/gt/gen8_engine_cs.c +0 -1
 		cs = gen8_emit_pipe_control(cs, flags, LRC_PPHWSP_SCRATCH_ADDR);
 
-		if (!HAS_FLAT_CCS(rq->engine->i915)) {

... while before it was executed only if (!HAS_FLAT_CCS(rq->engine->i915)), which, according to commit description of PATCH 2/9, rather had the opposite meaning. Am I missing something?

flat_ccs and aux_ccs are not mutually exclusive, so far the can both miss like in PVC. So that the !HAS_FLAT_CCS() is an approximation and that's why we need a better evaluation.

Aux invalidation is needed only on platforms from TGL and beyond excluding PVC. The above engines are the only engines where AUX invalidation happens, but there are no cases when we reach the default condition, as the emit_flush_rcs is already called within that set of engines. The default is there just for completeness.

Does this answer?

Thanks,
Andi

#27 lore raw

[Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Update AUX invalidation sequence (rev8)

In reply to Andi Shyti
== Series Details ==

== Series Details ==

Series: Update AUX invalidation sequence (rev8)
URL : https://patchwork.freedesktop.org/series/119798/
State : warning

== Summary ==

Error: dim checkpatch failed
696c6b239a71 drm/i915/gt: Cleanup aux invalidation registers ba3873d2802a drm/i915: Add the has_aux_ccs device property 8cc2211308ad drm/i915/gt: Ensure memory quiesced before invalidation d1b12d76fbff drm/i915/gt: Rename flags with bit_group_X according to the datasheet 54b2cec88008 drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control -:42: CHECK:SPACING: spaces preferred around that '<<' (ctx:VxV)
#42: FILE: drivers/gpu/drm/i915/gt/intel_gpu_commands.h:302:
+#define PIPE_CONTROL_CCS_FLUSH (1<<13) /* MTL+ */

                                 			  ^

total: 0 errors, 0 warnings, 1 checks, 20 lines checked
7778886ca085 drm/i915/gt: Refactor intel_emit_pipe_control_cs() in a single function 920a67fa76d2 drm/i915/gt: Ensure memory quiesced before invalidation for all engines -:7: WARNING:UNKNOWN_COMMIT_ID: Unknown commit id 'af9e423a8aae', maybe rebased or not pulled? #7:
Commit af9e423a8aae ("drm/i915/gt: Ensure memory quiesced before

total: 0 errors, 1 warnings, 0 checks, 51 lines checked c9cabd4ec887 drm/i915/gt: Poll aux invalidation register bit on invalidation 854ed4a400db drm/i915/gt: Support aux invalidation on all engines

#29 lore raw

[Intel-gfx] ✗ Fi.CI.BAT: failure for Update AUX invalidation sequence (rev8)

In reply to Andi Shyti
== Series Details ==

== Series Details ==

Series: Update AUX invalidation sequence (rev8)
URL : https://patchwork.freedesktop.org/series/119798/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_13400 -> Patchwork_119798v8
====================================================

Summary
-------

**FAILURE**

Serious unknown changes coming with Patchwork_119798v8 absolutely need to be verified manually.

If you think the reported changes have nothing to do with the changes introduced in Patchwork_119798v8, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/index.html

Participating hosts (43 -> 42)
------------------------------

Missing (1): fi-snb-2520m

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in Patchwork_119798v8:

### IGT changes ###

#### Possible regressions ####

* igt@i915_module_load@load:
- bat-rpls-1: [PASS][1] -> [ABORT][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-rpls-1/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-rpls-1/igt@[email protected]
- bat-adlp-6: [PASS][3] -> [ABORT][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-adlp-6/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-adlp-6/igt@[email protected]
- bat-adls-5: [PASS][5] -> [ABORT][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-adls-5/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-adls-5/igt@[email protected]
- bat-dg1-5: [PASS][7] -> [ABORT][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-dg1-5/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-dg1-5/igt@[email protected]
- bat-dg1-7: [PASS][9] -> [ABORT][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-dg1-7/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-dg1-7/igt@[email protected]
- bat-adlp-9: [PASS][11] -> [ABORT][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-adlp-9/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-adlp-9/igt@[email protected]
- bat-dg2-11: [PASS][13] -> [ABORT][14]
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-dg2-11/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-dg2-11/igt@[email protected]
- bat-adln-1: [PASS][15] -> [ABORT][16]
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-adln-1/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-adln-1/igt@[email protected]
- bat-adlm-1: [PASS][17] -> [ABORT][18]
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-adlm-1/igt@[email protected]
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-adlm-1/igt@[email protected]
- fi-tgl-1115g4: [PASS][19] -> [ABORT][20]
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/fi-tgl-1115g4/igt@[email protected]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/fi-tgl-1115g4/igt@[email protected]
- bat-atsm-1: [PASS][21] -> [ABORT][22]
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-atsm-1/igt@[email protected]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-atsm-1/igt@[email protected]
- bat-dg2-9: [PASS][23] -> [ABORT][24]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-dg2-9/igt@[email protected]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-dg2-9/igt@[email protected]
- bat-rpls-2: [PASS][25] -> [ABORT][26]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-rpls-2/igt@[email protected]
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-rpls-2/igt@[email protected]
- bat-dg2-8: [PASS][27] -> [ABORT][28]
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-dg2-8/igt@[email protected]
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-dg2-8/igt@[email protected]

#### Warnings ####

* igt@i915_module_load@load:
- bat-adlp-11: [DMESG-WARN][29] ([i915#4423]) -> [ABORT][30]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-adlp-11/igt@[email protected]
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-adlp-11/igt@[email protected]

#### Suppressed ####

The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.

* igt@i915_module_load@load:
- {bat-dg2-13}: [DMESG-WARN][31] ([Intel XE#486] / [i915#8879]) -> [ABORT][32]
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-dg2-13/igt@[email protected]
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-dg2-13/igt@[email protected]
- {bat-dg2-14}: [DMESG-WARN][33] ([Intel XE#486] / [i915#8879]) -> [ABORT][34]
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-dg2-14/igt@[email protected]
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-dg2-14/igt@[email protected]

Known issues
------------

Here are the changes found in Patchwork_119798v8 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@i915_module_load@load:
- fi-rkl-11600: [PASS][35] -> [ABORT][36] ([i915#8695])
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/fi-rkl-11600/igt@[email protected]
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/fi-rkl-11600/igt@[email protected]
- bat-rplp-1: [PASS][37] -> [ABORT][38] ([i915#8695])
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-rplp-1/igt@[email protected]
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-rplp-1/igt@[email protected]
- bat-mtlp-6: [PASS][39] -> [ABORT][40] ([i915#8141])
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-mtlp-6/igt@[email protected]
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-mtlp-6/igt@[email protected]

* igt@i915_pm_rpm@basic-pci-d3-state:
- fi-cfl-8109u: [PASS][41] -> [FAIL][42] ([i915#7940])
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/fi-cfl-8109u/igt@[email protected]
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/fi-cfl-8109u/igt@[email protected]

* igt@i915_pm_rpm@basic-rte:
- fi-cfl-guc: [PASS][43] -> [FAIL][44] ([i915#7940]) +1 similar issue
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/fi-cfl-guc/igt@[email protected]
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/fi-cfl-guc/igt@[email protected]
- fi-kbl-7567u: [PASS][45] -> [FAIL][46] ([i915#7940])
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/fi-kbl-7567u/igt@[email protected]
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/fi-kbl-7567u/igt@[email protected]

* igt@i915_pm_rpm@module-reload:
- fi-skl-guc: [PASS][47] -> [FAIL][48] ([i915#7940])
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/fi-skl-guc/igt@[email protected]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/fi-skl-guc/igt@[email protected]

* igt@i915_selftest@live@gt_heartbeat:
- fi-apl-guc: [PASS][49] -> [DMESG-FAIL][50] ([i915#5334])
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/fi-apl-guc/igt@i915_selftest@live@gt_heartbeat.html
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/fi-apl-guc/igt@i915_selftest@live@gt_heartbeat.html

#### Possible fixes ####

* igt@i915_pm_rpm@basic-pci-d3-state:
- fi-skl-guc: [FAIL][51] ([i915#7940]) -> [PASS][52]
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/fi-skl-guc/igt@[email protected]
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/fi-skl-guc/igt@[email protected]

#### Warnings ####

* igt@i915_module_load@load:
- bat-mtlp-8: [DMESG-WARN][53] ([i915#8937]) -> [ABORT][54] ([i915#8141])
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/bat-mtlp-8/igt@[email protected]
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/bat-mtlp-8/igt@[email protected]

* igt@i915_pm_rpm@basic-rte:
- fi-kbl-guc: [FAIL][55] ([i915#7940]) -> [FAIL][56] ([i915#8843])
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13400/fi-kbl-guc/igt@[email protected]
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/fi-kbl-guc/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[Intel XE#486]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/486
[i915#4423]: https://gitlab.freedesktop.org/drm/intel/issues/4423
[i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
[i915#7940]: https://gitlab.freedesktop.org/drm/intel/issues/7940
[i915#8141]: https://gitlab.freedesktop.org/drm/intel/issues/8141
[i915#8695]: https://gitlab.freedesktop.org/drm/intel/issues/8695
[i915#8843]: https://gitlab.freedesktop.org/drm/intel/issues/8843
[i915#8879]: https://gitlab.freedesktop.org/drm/intel/issues/8879
[i915#8937]: https://gitlab.freedesktop.org/drm/intel/issues/8937

Build changes
-------------

* Linux: CI_DRM_13400 -> Patchwork_119798v8

CI-20190529: 20190529
CI_DRM_13400: cc69df372f21eb3073c062132ee9eb3649605931 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_7396: 8e84faf33c2cf3482c7dff814d256089bc03db5d @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_119798v8: cc69df372f21eb3073c062132ee9eb3649605931 @ git://anongit.freedesktop.org/gfx-ci/linux

### Linux commits

d320bc32be19 drm/i915/gt: Support aux invalidation on all engines a2dd41fc306d drm/i915/gt: Poll aux invalidation register bit on invalidation ae803659a0b1 drm/i915/gt: Ensure memory quiesced before invalidation for all engines e1bac37fcd84 drm/i915/gt: Refactor intel_emit_pipe_control_cs() in a single function 1dad70d437d3 drm/i915/gt: Enable the CCS_FLUSH bit in the pipe control cc406a4025cb drm/i915/gt: Rename flags with bit_group_X according to the datasheet 24ce9bf28182 drm/i915/gt: Ensure memory quiesced before invalidation 6c45dd8e1fc6 drm/i915: Add the has_aux_ccs device property 1744edf1a445 drm/i915/gt: Cleanup aux invalidation registers

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_119798v8/index.html