Skip to contentSkip to conversation

drm/i915/guc: Provide register list to be saved/restored during engine reset

PATCH In review Michel Thierry 4 participants 8 messages Apr 19, 2017 – Apr 21, 2017 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

PATCH v6 13/20 drm/i915/guc: Provide register list to be saved/restored during engine reset

From: Arun Siluvery <[email protected]>

GuC expects a list of registers from the driver which are saved/restored during engine reset. The type of value to be saved is controlled by flags. We provide a minimal set of registers that we want GuC to save and restore. This is not an issue in case of engine reset as driver initializes most of them following an engine reset, but in case of media reset (aka watchdog reset) which is completely internal to GuC (including resubmission of hung workload), it is necessary to provide this list, otherwise GuC won't be able to schedule further workloads after a reset. This is the minimal set of registers identified for things to work as expected but if we see any new issues, this register list can be expanded.

In order to not loose any existing workarounds, we have to let GuC know the registers and its values. These will be reapplied after the reset. Note that we can't just read the current value because most of these registers are masked (so we have a workaround for a workaround for a workaround).

v2: REGSET_MASKED is too difficult for GuC, use REGSET_SAVE_DEFAULT_VALUE and current value from RING_MODE reg instead; no need to preserve head/tail either, be extra paranoid and save whitelisted registers (Daniele).

v3: Workarounds added only once during _init_workarounds also have to been restored, or we risk loosing them after internal GuC reset (Daniele).

---
 drivers/gpu/drm/i915/i915_drv.h            |  3 ++
 drivers/gpu/drm/i915/i915_guc_submission.c | 68 +++++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/intel_engine_cs.c     | 65 +++++++++++++++++++---------
 3 files changed, 114 insertions(+), 22 deletions(-)
drivers/gpu/drm/i915/i915_drv.h +3 -0
index fa3988c5033b..1ba1ac016973 100644
@@ -1913,7 +1913,10 @@ struct i915_wa_reg {
 
 struct i915_workarounds {
 	struct i915_wa_reg reg[I915_MAX_WA_REGS];
+	/* list of registers (and their values) that GuC will have to restore */
+	struct i915_wa_reg guc_reg[GUC_REGSET_MAX_REGISTERS];
 	u32 count;
+	u32 guc_count;
 	u32 hw_whitelist_count[I915_NUM_ENGINES];
 };
 
drivers/gpu/drm/i915/i915_guc_submission.c +66 -2

93 lines not on this page · raw message

drivers/gpu/drm/i915/intel_engine_cs.c +45 -20

128 lines not on this page · raw message

Signature · 6 lines
2.11.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer
#2 lore raw
Quoted 31 lines · Michel Thierry

From: Arun Siluvery <[email protected]>

GuC expects a list of registers from the driver which are saved/restored during engine reset. The type of value to be saved is controlled by flags. We provide a minimal set of registers that we want GuC to save and restore. This is not an issue in case of engine reset as driver initializes most of them following an engine reset, but in case of media reset (aka watchdog reset) which is completely internal to GuC (including resubmission of hung workload), it is necessary to provide this list, otherwise GuC won't be able to schedule further workloads after a reset. This is the minimal set of registers identified for things to work as expected but if we see any new issues, this register list can be expanded.

In order to not loose any existing workarounds, we have to let GuC know the registers and its values. These will be reapplied after the reset. Note that we can't just read the current value because most of these registers are masked (so we have a workaround for a workaround for a workaround).

v2: REGSET_MASKED is too difficult for GuC, use REGSET_SAVE_DEFAULT_VALUE and current value from RING_MODE reg instead; no need to preserve head/tail either, be extra paranoid and save whitelisted registers (Daniele).

v3: Workarounds added only once during _init_workarounds also have to been restored, or we risk loosing them after internal GuC reset (Daniele).

<SNIP>

drivers/gpu/drm/i915/intel_engine_cs.c static int gen9_init_workarounds(struct intel_engine_cs *engine) · line 759

Quoted 7 lines
(quoted diff) +0 -0
@@ -732,15 +755,16 @@ static int gen9_init_workarounds(struct intel_engine_cs *engine)
 >  	int ret;


  /* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk */

drivers/gpu/drm/i915/intel_engine_cs.c +2 -1
-	I915_WRITE(GEN9_CSFE_CHICKEN1_RCS, _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
+	I915_GUC_REG_WRITE(GEN9_CSFE_CHICKEN1_RCS,
+			   _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));

To make grepping easier, how about?

	I915_WRITE(GUC_REG(GEN9_CSFE_CHICKEN1_RCS),
		   ...);

Regards, Joonas

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw
Quoted 45 lines · Joonas Lahtinen
Quoted 31 lines · Michel Thierry

From: Arun Siluvery <[email protected]>

GuC expects a list of registers from the driver which are saved/restored during engine reset. The type of value to be saved is controlled by flags. We provide a minimal set of registers that we want GuC to save and restore. This is not an issue in case of engine reset as driver initializes most of them following an engine reset, but in case of media reset (aka watchdog reset) which is completely internal to GuC (including resubmission of hung workload), it is necessary to provide this list, otherwise GuC won't be able to schedule further workloads after a reset. This is the minimal set of registers identified for things to work as expected but if we see any new issues, this register list can be expanded.

In order to not loose any existing workarounds, we have to let GuC know the registers and its values. These will be reapplied after the reset. Note that we can't just read the current value because most of these registers are masked (so we have a workaround for a workaround for a workaround).

v2: REGSET_MASKED is too difficult for GuC, use REGSET_SAVE_DEFAULT_VALUE and current value from RING_MODE reg instead; no need to preserve head/tail either, be extra paranoid and save whitelisted registers (Daniele).

v3: Workarounds added only once during _init_workarounds also have to been restored, or we risk loosing them after internal GuC reset (Daniele).

<SNIP>

drivers/gpu/drm/i915/intel_engine_cs.c static int gen9_init_workarounds(struct intel_engine_cs *engine) · line 759

Quoted 7 lines
drivers/gpu/drm/i915/intel_engine_cs.c +2 -1
@@ -732,15 +755,16 @@ static int gen9_init_workarounds(struct intel_engine_cs *engine)
 >  	int ret;
 
 	/* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk */
-	I915_WRITE(GEN9_CSFE_CHICKEN1_RCS, _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
+	I915_GUC_REG_WRITE(GEN9_CSFE_CHICKEN1_RCS,
+			   _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));

To make grepping easier, how about?

Quoted 5 lines
	I915_WRITE(GUC_REG(GEN9_CSFE_CHICKEN1_RCS),
		   ...);

Regards, Joonas

GUC_REG makes it sound like it is somehow related to GuC, while it isn't, we just want GuC to restore its value. What about GUC_REG_RESTORE?

Thanks,
Daniele
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw
Quoted 57 lines · Daniele Ceraolo Spurio
Quoted 54 lines · Joonas Lahtinen
Quoted 37 lines · Michel Thierry

From: Arun Siluvery <[email protected]>

GuC expects a list of registers from the driver which are saved/restored during engine reset. The type of value to be saved is controlled by flags. We provide a minimal set of registers that we want GuC to save and
restore. This is not an issue in case of engine reset as driver initializes
most of them following an engine reset, but in case of media reset (aka watchdog reset) which is completely internal to GuC (including resubmission
of hung workload), it is necessary to provide this list, otherwise GuC won't
be able to schedule further workloads after a reset. This is the minimal set of registers identified for things to work as expected but if we see any new issues, this register list can be expanded.

In order to not loose any existing workarounds, we have to let GuC know the registers and its values. These will be reapplied after the reset. Note that we can't just read the current value because most of these registers are masked (so we have a workaround for a workaround for a workaround).

v2: REGSET_MASKED is too difficult for GuC, use REGSET_SAVE_DEFAULT_VALUE
and current value from RING_MODE reg instead; no need to preserve head/tail either, be extra paranoid and save whitelisted registers (Daniele).

v3: Workarounds added only once during _init_workarounds also have to been restored, or we risk loosing them after internal GuC reset (Daniele).

<SNIP>

Review static int gen9_init_workarounds(struct · line 762

Quoted 10 lines
(quoted diff) +2 -1
@@ -732,15 +755,16 @@ static int gen9_init_workarounds(struct
 intel_engine_cs *engine)
 >      int ret;
 
     /* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk */
-    I915_WRITE(GEN9_CSFE_CHICKEN1_RCS,
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
+    I915_GUC_REG_WRITE(GEN9_CSFE_CHICKEN1_RCS,
+
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));

To make grepping easier, how about?

Quoted 9 lines
Quoted 5 lines
    I915_WRITE(GUC_REG(GEN9_CSFE_CHICKEN1_RCS),
           ...);

Regards, Joonas

GUC_REG makes it sound like it is somehow related to GuC, while it isn't, we just want GuC to restore its value. What about GUC_REG_RESTORE?

Honestly, I dont care about names, pick one and I add it.
Just a reminder, we not only need the reg offset, we want to save the value too.

-Michel
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#5 lore raw
Quoted 69 lines · Michel Thierry
Quoted 66 lines · Daniele Ceraolo Spurio
Quoted 62 lines · Joonas Lahtinen
Quoted 40 lines · Michel Thierry

From: Arun Siluvery <[email protected]>

GuC expects a list of registers from the driver which are saved/restored
during engine reset. The type of value to be saved is controlled by flags. We provide a minimal set of registers that we want GuC to save and
restore. This is not an issue in case of engine reset as driver initializes
most of them following an engine reset, but in case of media reset (aka watchdog reset) which is completely internal to GuC (including resubmission
of hung workload), it is necessary to provide this list, otherwise GuC won't
be able to schedule further workloads after a reset. This is the minimal
set of registers identified for things to work as expected but if we see
any new issues, this register list can be expanded.

In order to not loose any existing workarounds, we have to let GuC know the registers and its values. These will be reapplied after the reset. Note that we can't just read the current value because most of these registers are masked (so we have a workaround for a workaround for a workaround).

v2: REGSET_MASKED is too difficult for GuC, use REGSET_SAVE_DEFAULT_VALUE
and current value from RING_MODE reg instead; no need to preserve head/tail either, be extra paranoid and save whitelisted registers (Daniele).

v3: Workarounds added only once during _init_workarounds also have to been restored, or we risk loosing them after internal GuC reset (Daniele).

<SNIP>

Review static int gen9_init_workarounds(struct · line 762

Quoted 10 lines
(quoted diff) +2 -1
@@ -732,15 +755,16 @@ static int gen9_init_workarounds(struct
 intel_engine_cs *engine)
 >      int ret;
 
     /* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk */
-    I915_WRITE(GEN9_CSFE_CHICKEN1_RCS,
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
+    I915_GUC_REG_WRITE(GEN9_CSFE_CHICKEN1_RCS,
+
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));

To make grepping easier, how about?

    I915_WRITE(GUC_REG(GEN9_CSFE_CHICKEN1_RCS),
           ...);

Regards, Joonas

Quoted 8 lines
Quoted 3 lines

GUC_REG makes it sound like it is somehow related to GuC, while it isn't, we just want GuC to restore its value. What about GUC_REG_RESTORE?

Honestly, I dont care about names, pick one and I add it. Just a reminder, we not only need the reg offset, we want to save the value too.

I915_WRITE_GUC_RESTORE(reg, value) ?

That would be inline to the others we have, e.g. I915_WRITE_FW, I915_WRITE_CTL, I915_WRITE_HEAD/TAIL.

-Michel
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#6 lore raw
Quoted 84 lines · Michel Thierry
Quoted 71 lines · Michel Thierry
Quoted 68 lines · Daniele Ceraolo Spurio
Quoted 64 lines · Joonas Lahtinen
Quoted 42 lines · Michel Thierry

From: Arun Siluvery <[email protected]>

GuC expects a list of registers from the driver which are saved/restored
during engine reset. The type of value to be saved is controlled by flags. We provide a minimal set of registers that we want GuC to save and
restore. This is not an issue in case of engine reset as driver initializes
most of them following an engine reset, but in case of media reset (aka
watchdog reset) which is completely internal to GuC (including resubmission
of hung workload), it is necessary to provide this list, otherwise GuC won't
be able to schedule further workloads after a reset. This is the minimal
set of registers identified for things to work as expected but if we see
any new issues, this register list can be expanded.

In order to not loose any existing workarounds, we have to let GuC know
the registers and its values. These will be reapplied after the reset. Note that we can't just read the current value because most of these registers are masked (so we have a workaround for a workaround for a workaround).

v2: REGSET_MASKED is too difficult for GuC, use REGSET_SAVE_DEFAULT_VALUE
and current value from RING_MODE reg instead; no need to preserve head/tail either, be extra paranoid and save whitelisted registers (Daniele).

v3: Workarounds added only once during _init_workarounds also have to been restored, or we risk loosing them after internal GuC reset (Daniele).

<SNIP>

Review static int gen9_init_workarounds(struct · line 762

Quoted 10 lines
(quoted diff) +2 -1
@@ -732,15 +755,16 @@ static int gen9_init_workarounds(struct
 intel_engine_cs *engine)
 >      int ret;
 
     /* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk */
-    I915_WRITE(GEN9_CSFE_CHICKEN1_RCS,
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
+    I915_GUC_REG_WRITE(GEN9_CSFE_CHICKEN1_RCS,
+
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));

To make grepping easier, how about?

    I915_WRITE(GUC_REG(GEN9_CSFE_CHICKEN1_RCS),
           ...);

Regards, Joonas

Quoted 9 lines
Quoted 4 lines

GUC_REG makes it sound like it is somehow related to GuC, while it isn't, we just want GuC to restore its value. What about GUC_REG_RESTORE?

Honestly, I dont care about names, pick one and I add it. Just a reminder, we not only need the reg offset, we want to save the value too.

Quoted 6 lines

I915_WRITE_GUC_RESTORE(reg, value) ?

That would be inline to the others we have, e.g. I915_WRITE_FW, I915_WRITE_CTL, I915_WRITE_HEAD/TAIL.

-Michel

LGTM.

Daniele
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#7 lore raw
Quoted 87 lines · Daniele Ceraolo Spurio
Quoted 84 lines · Michel Thierry
Quoted 80 lines · Michel Thierry
Quoted 65 lines · Daniele Ceraolo Spurio
Quoted 62 lines · Joonas Lahtinen
Quoted 42 lines · Michel Thierry

From: Arun Siluvery <[email protected]>

GuC expects a list of registers from the driver which are saved/restored
during engine reset. The type of value to be saved is controlled by flags. We provide a minimal set of registers that we want GuC to save and
restore. This is not an issue in case of engine reset as driver initializes
most of them following an engine reset, but in case of media reset (aka
watchdog reset) which is completely internal to GuC (including resubmission
of hung workload), it is necessary to provide this list, otherwise GuC won't
be able to schedule further workloads after a reset. This is the minimal
set of registers identified for things to work as expected but if we see
any new issues, this register list can be expanded.

In order to not loose any existing workarounds, we have to let GuC know
the registers and its values. These will be reapplied after the reset. Note that we can't just read the current value because most of these registers are masked (so we have a workaround for a workaround for a workaround).

v2: REGSET_MASKED is too difficult for GuC, use REGSET_SAVE_DEFAULT_VALUE
and current value from RING_MODE reg instead; no need to preserve head/tail either, be extra paranoid and save whitelisted registers (Daniele).

v3: Workarounds added only once during _init_workarounds also have to been restored, or we risk loosing them after internal GuC reset (Daniele).

<SNIP>

Review static int gen9_init_workarounds(struct · line 762

Quoted 10 lines
(quoted diff) +2 -1
@@ -732,15 +755,16 @@ static int gen9_init_workarounds(struct
 intel_engine_cs *engine)
 >     int ret;
 
    /* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk */
-    I915_WRITE(GEN9_CSFE_CHICKEN1_RCS,
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
+    I915_GUC_REG_WRITE(GEN9_CSFE_CHICKEN1_RCS,
+
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));

To make grepping easier, how about?

   I915_WRITE(GUC_REG(GEN9_CSFE_CHICKEN1_RCS),
          ...);
Quoted 7 lines
Quoted 2 lines

Regards, Joonas

GUC_REG makes it sound like it is somehow related to GuC, while it isn't, we just want GuC to restore its value. What about GUC_REG_RESTORE?

Honestly, I dont care about names, pick one and I add it. Just a reminder, we not only need the reg offset, we want to save the value too.

Quoted 4 lines
Quoted 4 lines

I915_WRITE_GUC_RESTORE(reg, value) ?

That would be inline to the others we have, e.g. I915_WRITE_FW, I915_WRITE_CTL, I915_WRITE_HEAD/TAIL.

I915_WRITE_FW is not the same class as I915_WRITE_CTL/HEAD/TAIL, and I can say from experience the I915_*_CTL/HEAD/TAIL were a mistake (special casing one particular access to the ring mmio, but we often deviate from that pattern).

Looking at the above I see you are falling for the same trap as the ring shorthand... So are you sure the convenience will not be lost later? And in particular avoid using I915_WRITE_*() naming style as I would rather that was earmarked for the different mmio accessors.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#8 lore raw
Quoted 98 lines · Chris Wilson
Quoted 87 lines · Daniele Ceraolo Spurio
Quoted 84 lines · Michel Thierry
Quoted 80 lines · Michel Thierry
Quoted 65 lines · Daniele Ceraolo Spurio
Quoted 62 lines · Joonas Lahtinen
Quoted 42 lines · Michel Thierry

From: Arun Siluvery <[email protected]>

GuC expects a list of registers from the driver which are saved/restored
during engine reset. The type of value to be saved is controlled by flags. We provide a minimal set of registers that we want GuC to save and
restore. This is not an issue in case of engine reset as driver initializes
most of them following an engine reset, but in case of media reset (aka
watchdog reset) which is completely internal to GuC (including resubmission
of hung workload), it is necessary to provide this list, otherwise GuC won't
be able to schedule further workloads after a reset. This is the minimal
set of registers identified for things to work as expected but if we see
any new issues, this register list can be expanded.

In order to not loose any existing workarounds, we have to let GuC know
the registers and its values. These will be reapplied after the reset. Note that we can't just read the current value because most of these registers are masked (so we have a workaround for a workaround for a workaround).

v2: REGSET_MASKED is too difficult for GuC, use REGSET_SAVE_DEFAULT_VALUE
and current value from RING_MODE reg instead; no need to preserve head/tail either, be extra paranoid and save whitelisted registers (Daniele).

v3: Workarounds added only once during _init_workarounds also have to been restored, or we risk loosing them after internal GuC reset (Daniele).

<SNIP>

Review static int gen9_init_workarounds(struct · line 762

Quoted 10 lines
(quoted diff) +2 -1
@@ -732,15 +755,16 @@ static int gen9_init_workarounds(struct
 intel_engine_cs *engine)
 >     int ret;
 
    /* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk */
-    I915_WRITE(GEN9_CSFE_CHICKEN1_RCS,
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
+    I915_GUC_REG_WRITE(GEN9_CSFE_CHICKEN1_RCS,
+
 _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));

To make grepping easier, how about?

   I915_WRITE(GUC_REG(GEN9_CSFE_CHICKEN1_RCS),
          ...);
Quoted 7 lines
Quoted 2 lines

Regards, Joonas

GUC_REG makes it sound like it is somehow related to GuC, while it isn't, we just want GuC to restore its value. What about GUC_REG_RESTORE?

Honestly, I dont care about names, pick one and I add it. Just a reminder, we not only need the reg offset, we want to save the value too.

Quoted 4 lines
Quoted 4 lines

I915_WRITE_GUC_RESTORE(reg, value) ?

That would be inline to the others we have, e.g. I915_WRITE_FW, I915_WRITE_CTL, I915_WRITE_HEAD/TAIL.

I915_WRITE_FW is not the same class as I915_WRITE_CTL/HEAD/TAIL, and I can say from experience the I915_*_CTL/HEAD/TAIL were a mistake (special casing one particular access to the ring mmio, but we often deviate from that pattern).

Quoted 4 lines

Looking at the above I see you are falling for the same trap as the ring shorthand... So are you sure the convenience will not be lost later? And in particular avoid using I915_WRITE_*() naming style as I would rather that was earmarked for the different mmio accessors.

Ok, then can follow the pattern of the other workarounds & whitelist reg code?

E.g. WA_REG_GUC_RESTORE or WA_MMIO_REG_GUC_RESTORE (to make it clearer that these are not registers in the ctx image).

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx