Skip to contentSkip to conversation

drm/etnaviv: Ignore MC bit when checking for runtime suspend

PATCH In review Guido Günther 3 participants 10 messages Mar 2, 2020 – Jun 26, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover drm/etnaviv: Ignore MC bit when checking for runtime suspend

At least GC7000 fails to enter runtime suspend for long periods of time since the MC becomes busy again even when the FE is idle. The rest of the series makes detecting similar issues easier to debug in the future by checking all known bits in debugfs and also warning in the EBUSY case.

Tested on GC7000 with a reduced runtime delay of 50ms. Patches are against next-20200226.

Thanks to Lucas Stach for pointing me in the right direction.

Guido Günther (5):
drm/etnaviv: Fix typo in comment
drm/etnaviv: Update idle bits
drm/etnaviv: Consider all kwnown idle bits in debugfs drm/etnaviv: Ignore MC when checking runtime suspend idleness drm/etnaviv: Warn when GPU doesn't idle fast enough

 drivers/gpu/drm/etnaviv/etnaviv_gpu.c  | 26 ++++++++++++++++++++++----
 drivers/gpu/drm/etnaviv/state_hi.xml.h |  7 +++++++
 2 files changed, 29 insertions(+), 4 deletions(-)
Signature · 6 lines
2.23.0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#3 lore raw

PATCH 3/5 drm/etnaviv: Consider all kwnown idle bits in debugfs

We were missing out on some bits the vendor kernel driver knows about.

---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/gpu/drm/etnaviv/etnaviv_gpu.c +14 -0
index 873d9103164d..187de610b325 100644
@@ -930,6 +930,20 @@ int etnaviv_gpu_debugfs(struct etnaviv_gpu *gpu, struct seq_file *m)
 		seq_puts(m, "\t FP is not idle\n");
 	if ((idle & VIVS_HI_IDLE_STATE_TS) == 0)
 		seq_puts(m, "\t TS is not idle\n");
+	if ((idle & VIVS_HI_IDLE_STATE_BL) == 0)
+		seq_puts(m, "\t BL is not idle\n");
+	if ((idle & VIVS_HI_IDLE_STATE_ASYNCFE) == 0)
+		seq_puts(m, "\t ASYNCFE is not idle\n");
+	if ((idle & VIVS_HI_IDLE_STATE_MC) == 0)
+		seq_puts(m, "\t MC is not idle\n");
+	if ((idle & VIVS_HI_IDLE_STATE_PPA) == 0)
+		seq_puts(m, "\t PPA is not idle\n");
+	if ((idle & VIVS_HI_IDLE_STATE_WD) == 0)
+		seq_puts(m, "\t WD is not idle\n");
+	if ((idle & VIVS_HI_IDLE_STATE_NN) == 0)
+		seq_puts(m, "\t NN is not idle\n");
+	if ((idle & VIVS_HI_IDLE_STATE_TP) == 0)
+		seq_puts(m, "\t TP is not idle\n");
 	if (idle & VIVS_HI_IDLE_STATE_AXI_LP)
 		seq_puts(m, "\t AXI low power mode\n");
 
Signature · 1 line
2.23.0
#4 lore raw

PATCH 4/5 drm/etnaviv: Ignore MC when checking runtime suspend idleness

In reply to Guido Günther

Without that runtime suspend is often blocked due to etnaviv_gpu_rpm_suspend() returning -EBUSY since the FE seems to trigger the MC in its idle loop.

Ignoring the MC bit makes the GPU suspend as expected. This was tested on GC7000.

---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/gpu/drm/etnaviv/etnaviv_gpu.c +3 -2
index 187de610b325..da24e433f82a 100644
@@ -1820,8 +1820,9 @@ static int etnaviv_gpu_rpm_suspend(struct device *dev)
 	if (atomic_read(&gpu->sched.hw_rq_count))
 		return -EBUSY;
 
-	/* Check whether the hardware (except FE) is idle */
-	mask = gpu->idle_mask & ~VIVS_HI_IDLE_STATE_FE;
+	/* Check whether the hardware (except FE and MC) is idle */
+	mask = gpu->idle_mask & ~(VIVS_HI_IDLE_STATE_FE |
+				  VIVS_HI_IDLE_STATE_MC);
 	idle = gpu_read(gpu, VIVS_HI_IDLE_STATE) & mask;
 	if (idle != mask)
 		return -EBUSY;
Signature · 1 line
2.23.0
#1 lore raw

PATCH 1/5 drm/etnaviv: Fix typo in comment

In reply to Guido Günther

Use 'is' instead of 'it' so it becomes a valid sentence and spell 'resetting' correctly.

---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/etnaviv/etnaviv_gpu.c +1 -1
index 80b99acea1c4..873d9103164d 100644
@@ -506,7 +506,7 @@ static int etnaviv_hw_reset(struct etnaviv_gpu *gpu)
 		/* read idle register. */
 		idle = gpu_read(gpu, VIVS_HI_IDLE_STATE);
 
-		/* try reseting again if FE it not idle */
+		/* try resetting again if FE is not idle */
 		if ((idle & VIVS_HI_IDLE_STATE_FE) == 0) {
 			dev_dbg(gpu->dev, "FE is not idle\n");
 			continue;
Signature · 1 line
2.23.0
#2 lore raw

PATCH 2/5 drm/etnaviv: Update idle bits

In reply to Guido Günther

Update the state HI and common header from rnndb commit commit 19280a95a (rnndb: Update idle bits)

---
 drivers/gpu/drm/etnaviv/state_hi.xml.h | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/gpu/drm/etnaviv/state_hi.xml.h +7 -0
index 41d8da2b6f4f..8fe4598395f1 100644
@@ -81,6 +81,13 @@ DEALINGS IN THE SOFTWARE.
 #define VIVS_HI_IDLE_STATE_IM					0x00000200
 #define VIVS_HI_IDLE_STATE_FP					0x00000400
 #define VIVS_HI_IDLE_STATE_TS					0x00000800
+#define VIVS_HI_IDLE_STATE_BL					0x00001000
+#define VIVS_HI_IDLE_STATE_ASYNCFE				0x00002000
+#define VIVS_HI_IDLE_STATE_MC					0x00004000
+#define VIVS_HI_IDLE_STATE_PPA					0x00008000
+#define VIVS_HI_IDLE_STATE_WD					0x00010000
+#define VIVS_HI_IDLE_STATE_NN					0x00020000
+#define VIVS_HI_IDLE_STATE_TP					0x00040000
 #define VIVS_HI_IDLE_STATE_AXI_LP				0x80000000
 
 #define VIVS_HI_AXI_CONFIG					0x00000008
Signature · 6 lines
2.23.0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#5 lore raw

PATCH 5/5 drm/etnaviv: Warn when GPU doesn't idle fast enough

In reply to Guido Günther

If the GPU isn't idle after signalling pm_runtime_mark_last_busy() plus waiting for the autosuspend delay there's likely something wrong with the way we check idleness so warn about that.

---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/etnaviv/etnaviv_gpu.c +4 -1
index da24e433f82a..4fd16b8f8a7a 100644
@@ -1824,8 +1824,11 @@ static int etnaviv_gpu_rpm_suspend(struct device *dev)
 	mask = gpu->idle_mask & ~(VIVS_HI_IDLE_STATE_FE |
 				  VIVS_HI_IDLE_STATE_MC);
 	idle = gpu_read(gpu, VIVS_HI_IDLE_STATE) & mask;
-	if (idle != mask)
+	if (idle != mask) {
+		dev_warn_ratelimited(dev, "GPU not yet idle, mask: 0x%08x\n",
+				     idle);
 		return -EBUSY;
+	}
 
 	return etnaviv_gpu_hw_suspend(gpu);
 }
Signature · 6 lines
2.23.0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
maintainer
#6 lore raw
In reply to Guido Günther
Quoted 4 lines · Guido Günther

At least GC7000 fails to enter runtime suspend for long periods of time since the MC becomes busy again even when the FE is idle. The rest of the series makes detecting similar issues easier to debug in the future by checking all known bits in debugfs and also warning in the EBUSY case.

Thanks, series applied to etnaviv/next.

Quoted 2 lines

Tested on GC7000 with a reduced runtime delay of 50ms. Patches are against next-20200226.

I've already wondered if 200ms is too long, 50ms sounds more reasonable. Do you have any numbers on the power draw on the i.MX8M with idle GPU, vs. being fully power gated?

Regards,
Lucas

Quoted 9 lines

Thanks to Lucas Stach for pointing me in the right direction.

Guido Günther (5):
drm/etnaviv: Fix typo in comment
drm/etnaviv: Update idle bits
drm/etnaviv: Consider all kwnown idle bits in debugfs drm/etnaviv: Ignore MC when checking runtime suspend idleness drm/etnaviv: Warn when GPU doesn't idle fast enough

Quoted 4 lines
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c  | 26 ++++++++++++++++++++++----
 drivers/gpu/drm/etnaviv/state_hi.xml.h |  7 +++++++
 2 files changed, 29 insertions(+), 4 deletions(-)

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#7 lore raw

Hi Lucas,

Quoted 7 lines · Lucas Stach
Quoted 4 lines · Guido Günther

At least GC7000 fails to enter runtime suspend for long periods of time since the MC becomes busy again even when the FE is idle. The rest of the series makes detecting similar issues easier to debug in the future by checking all known bits in debugfs and also warning in the EBUSY case.

Thanks, series applied to etnaviv/next.

Thanks!

Quoted 7 lines
Quoted 3 lines

Tested on GC7000 with a reduced runtime delay of 50ms. Patches are against next-20200226.

I've already wondered if 200ms is too long, 50ms sounds more reasonable. Do you have any numbers on the power draw on the i.MX8M with idle GPU, vs. being fully power gated?

I don't have good numbers yet but i'll post here once i do. Cheers,
-- Guido

Quoted 18 lines

Regards,
Lucas

Quoted 13 lines

Thanks to Lucas Stach for pointing me in the right direction.

Guido Günther (5):
drm/etnaviv: Fix typo in comment
drm/etnaviv: Update idle bits
drm/etnaviv: Consider all kwnown idle bits in debugfs drm/etnaviv: Ignore MC when checking runtime suspend idleness drm/etnaviv: Warn when GPU doesn't idle fast enough

 drivers/gpu/drm/etnaviv/etnaviv_gpu.c  | 26 ++++++++++++++++++++++----
 drivers/gpu/drm/etnaviv/state_hi.xml.h |  7 +++++++
 2 files changed, 29 insertions(+), 4 deletions(-)

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#8 lore raw
In reply to Lucas Stach

Hi,

Quoted 6 lines · Lucas Stach
Quoted 4 lines · Guido Günther

At least GC7000 fails to enter runtime suspend for long periods of time since the MC becomes busy again even when the FE is idle. The rest of the series makes detecting similar issues easier to debug in the future by checking all known bits in debugfs and also warning in the EBUSY case.

Quoted 8 lines

Thanks, series applied to etnaviv/next.

Quoted 2 lines

Tested on GC7000 with a reduced runtime delay of 50ms. Patches are against next-20200226.

I've already wondered if 200ms is too long, 50ms sounds more reasonable. Do you have any numbers on the power draw on the i.MX8M with idle GPU, vs. being fully power gated?

The difference is at least 250mW. It makes a huge difference over here. We hit
https://lore.kernel.org/dri-devel/[email protected]/
recently and you notice instantly when that happens when looking at the SoC temperature.

Cheers,
-- Guido

Quoted 18 lines

Regards,
Lucas

Quoted 13 lines

Thanks to Lucas Stach for pointing me in the right direction.

Guido Günther (5):
drm/etnaviv: Fix typo in comment
drm/etnaviv: Update idle bits
drm/etnaviv: Consider all kwnown idle bits in debugfs drm/etnaviv: Ignore MC when checking runtime suspend idleness drm/etnaviv: Warn when GPU doesn't idle fast enough

 drivers/gpu/drm/etnaviv/etnaviv_gpu.c  | 26 ++++++++++++++++++++++----
 drivers/gpu/drm/etnaviv/state_hi.xml.h |  7 +++++++
 2 files changed, 29 insertions(+), 4 deletions(-)

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#9 lore raw

Would this power difference with the GPU also apply with the GC3000 in the i.MX6qp or the GC2000 in the i.MX6q?

Quoted 47 lines · Guido Günther

Hi,

Quoted 2 lines · Lucas Stach
Quoted 1 line · Guido Günther

At least GC7000 fails to enter runtime suspend for long periods of

time since

Quoted 1 line
Quoted 1 line

the MC becomes busy again even when the FE is idle. The rest of the

series

Quoted 1 line
Quoted 1 line

makes detecting similar issues easier to debug in the future by

checking

Quoted 10 lines
Quoted 1 line

all known bits in debugfs and also warning in the EBUSY case.

Thanks, series applied to etnaviv/next.

Quoted 2 lines

Tested on GC7000 with a reduced runtime delay of 50ms. Patches are against next-20200226.

I've already wondered if 200ms is too long, 50ms sounds more reasonable. Do you have any numbers on the power draw on the i.MX8M with idle GPU, vs. being fully power gated?

The difference is at least 250mW. It makes a huge difference over here. We hit

https://lore.kernel.org/dri-devel/[email protected]/
recently and you notice instantly when that happens when looking at the SoC temperature.

Cheers,
-- Guido

Quoted 18 lines

Regards,
Lucas

Quoted 13 lines

Thanks to Lucas Stach for pointing me in the right direction.

Guido Günther (5):
drm/etnaviv: Fix typo in comment
drm/etnaviv: Update idle bits
drm/etnaviv: Consider all kwnown idle bits in debugfs drm/etnaviv: Ignore MC when checking runtime suspend idleness drm/etnaviv: Warn when GPU doesn't idle fast enough

 drivers/gpu/drm/etnaviv/etnaviv_gpu.c  | 26 ++++++++++++++++++++++----
 drivers/gpu/drm/etnaviv/state_hi.xml.h |  7 +++++++
 2 files changed, 29 insertions(+), 4 deletions(-)
Quoted 5 lines

_______________________________________________
etnaviv mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/etnaviv

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel