Skip to contentSkip to conversation

drm/i915: Initialize HWS page address after GPU reset

PATCH Applied Arun Siluvery 5 participants 7 messages Jun 2, 2015 – Jun 18, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/i915: Initialize HWS page address after GPU reset

After GPU reset, HW is losing the address of HWS page in the register. The page itself is valid except that HW is not aware of its location.

[   64.368623] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000
[   64.368655] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000
[   64.368681] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000
[   64.368704] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000

This patch reloads this value into the register during ring init.

Change-Id Ibdd1e4645921b4deb02dfc8d0d8e6ba993ce7371
---
 drivers/gpu/drm/i915/intel_lrc.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/gpu/drm/i915/intel_lrc.c +6 -0
index 0413b8f..73033f5 100644
@@ -1085,6 +1085,12 @@ static int gen8_init_common_ring(struct intel_engine_cs *ring)
 	I915_WRITE_IMR(ring, ~(ring->irq_enable_mask | ring->irq_keep_mask));
 	I915_WRITE(RING_HWSTAM(ring->mmio_base), 0xffffffff);
 
+	if (ring->status_page.obj) {
+		I915_WRITE(RING_HWS_PGA(ring->mmio_base),
+			   (u32)ring->status_page.gfx_addr);
+		POSTING_READ(RING_HWS_PGA(ring->mmio_base));
+	}
+
 	I915_WRITE(RING_MODE_GEN7(ring),
 		   _MASKED_BIT_DISABLE(GFX_REPLAY_MODE) |
 		   _MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE));
Signature · 6 lines
2.3.0

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Tested-by on v1 1/1
#1 lore raw

Task id: 6526 -------------------------------------Summary-------------------------------------
Platform Delta drm-intel-nightly Series Applied

PNV                                  276/276              276/276
ILK                                  303/303              303/303
SNB                 -1              315/315              314/315
IVB                                  343/343              343/343
BYT                                  287/287              287/287
BDW                                  321/321              321/321

-------------------------------------Detailed-------------------------------------
Platform Test drm-intel-nightly Series Applied
*SNB igt@pm_rpm@dpms-mode-unset-non-lpsp PASS(1) DMESG_WARN(1)
(dmesg patch applied)WARNING:at_drivers/gpu/drm/i915/intel_uncore.c:#assert_device_not_suspended[i915]()@WARNING:.* at .* assert_device_not_suspended+0x
Note: You need to pay more attention to line start with '*'
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

Reviewed-by on v1 1/1
#2 lore raw
In reply to Arun Siluvery
Quoted 21 lines · Arun Siluvery

After GPU reset, HW is losing the address of HWS page in the register. The page itself is valid except that HW is not aware of its location.

[   64.368623] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000
[   64.368655] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000
[   64.368681] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000
[   64.368704] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000

This patch reloads this value into the register during ring init.

Change-Id Ibdd1e4645921b4deb02dfc8d0d8e6ba993ce7371
---
 drivers/gpu/drm/i915/intel_lrc.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/gpu/drm/i915/intel_lrc.c +0 -0
index 0413b8f..73033f5 100644

drivers/gpu/drm/i915/intel_lrc.c static int gen8_init_common_ring(struct intel_engine_cs *ring) · line 1092

Quoted 8 lines
drivers/gpu/drm/i915/intel_lrc.c +5 -0
@@ -1085,6 +1085,12 @@ static int gen8_init_common_ring(struct intel_engine_cs *ring)
 	I915_WRITE_IMR(ring, ~(ring->irq_enable_mask | ring->irq_keep_mask));
 	I915_WRITE(RING_HWSTAM(ring->mmio_base), 0xffffffff);
 
+	if (ring->status_page.obj) {
+		I915_WRITE(RING_HWS_PGA(ring->mmio_base),
+			   (u32)ring->status_page.gfx_addr);
+		POSTING_READ(RING_HWS_PGA(ring->mmio_base));
+	}

I was going to suggest removing the same thing from the lrc_setup_hardware_status_page(), but after another look it seems we sometimes call .init_hw() before the context setup. Would be nice to have a more consistent sequence for init and reset. But anyway the patch looks OK to me. I verified that we indeed lose this register on GPU reset.

Quoted 11 lines
drivers/gpu/drm/i915/intel_lrc.c +1 -0
+
 	I915_WRITE(RING_MODE_GEN7(ring),
 		   _MASKED_BIT_DISABLE(GFX_REPLAY_MODE) |
 		   _MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE));
Signature · 6 lines
2.3.0

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 6 lines
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
maintainer Applied
#3 lore raw
Quoted 38 lines · Ville Syrjälä
Quoted 21 lines · Arun Siluvery

After GPU reset, HW is losing the address of HWS page in the register. The page itself is valid except that HW is not aware of its location.

[   64.368623] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000
[   64.368655] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000
[   64.368681] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000
[   64.368704] [drm:gen8_init_common_ring [i915]] *ERROR* HWS Page address = 0x00000000

This patch reloads this value into the register during ring init.

Change-Id Ibdd1e4645921b4deb02dfc8d0d8e6ba993ce7371
---
 drivers/gpu/drm/i915/intel_lrc.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/gpu/drm/i915/intel_lrc.c +0 -0
index 0413b8f..73033f5 100644

drivers/gpu/drm/i915/intel_lrc.c static int gen8_init_common_ring(struct intel_engine_cs *ring) · line 1092

Quoted 8 lines
drivers/gpu/drm/i915/intel_lrc.c +5 -0
@@ -1085,6 +1085,12 @@ static int gen8_init_common_ring(struct intel_engine_cs *ring)
 	I915_WRITE_IMR(ring, ~(ring->irq_enable_mask | ring->irq_keep_mask));
 	I915_WRITE(RING_HWSTAM(ring->mmio_base), 0xffffffff);
 
+	if (ring->status_page.obj) {
+		I915_WRITE(RING_HWS_PGA(ring->mmio_base),
+			   (u32)ring->status_page.gfx_addr);
+		POSTING_READ(RING_HWS_PGA(ring->mmio_base));
+	}

I was going to suggest removing the same thing from the lrc_setup_hardware_status_page(), but after another look it seems we sometimes call .init_hw() before the context setup. Would be nice to have a more consistent sequence for init and reset. But anyway the patch looks OK to me. I verified that we indeed lose this register on GPU reset.

Quoted 1 line

Pushed to drm-intel-fixes, thanks for the patch and review.

BR,
Jani.

Quoted 20 lines
Quoted 12 lines
drivers/gpu/drm/i915/intel_lrc.c +1 -0
+
 	I915_WRITE(RING_MODE_GEN7(ring),
 		   _MASKED_BIT_DISABLE(GFX_REPLAY_MODE) |
 		   _MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE));
Signature · 6 lines
2.3.0

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 6 lines
Ville Syrjälä
Intel OTC
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 5 lines
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw
In reply to Ville Syrjälä
Quoted 6 lines · Ville Syrjälä

I was going to suggest removing the same thing from the lrc_setup_hardware_status_page(), but after another look it seems we sometimes call .init_hw() before the context setup. Would be nice to have a more consistent sequence for init and reset. But anyway the patch looks OK to me. I verified that we indeed lose this register on GPU reset.

Yep, this is a mess. And historically _any_ difference between driver load and gpu reset (or resume fwiw) has lead to hilarious bugs, so this difference is really troubling to me. Arun, can you please work on a patch to unify the setup sequence here, so that both driver load gpu resets work the same way? By the time we're calling gem_init_hw the default context should have been created already, and hence we should be able to write to HWS_PGA in ring->init_hw only.

Also I wonder about resume, where's the HWS_PGA restore for that case? -Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw
Quoted 9 lines · Daniel Vetter
Quoted 6 lines · Ville Syrjälä

I was going to suggest removing the same thing from the lrc_setup_hardware_status_page(), but after another look it seems we sometimes call .init_hw() before the context setup. Would be nice to have a more consistent sequence for init and reset. But anyway the patch looks OK to me. I verified that we indeed lose this register on GPU reset.

Quoted 8 lines

Yep, this is a mess. And historically _any_ difference between driver load and gpu reset (or resume fwiw) has lead to hilarious bugs, so this difference is really troubling to me. Arun, can you please work on a patch to unify the setup sequence here, so that both driver load gpu resets work the same way? By the time we're calling gem_init_hw the default context should have been created already, and hence we should be able to write to HWS_PGA in ring->init_hw only.

Hi Daniel,

I think the problem in this case was the code to init HWS page after reset was missing for Gen8+. For Gen7 we are doing this as part of ring->init_hw.

Gen7:
i915_reset()

+--> i915_gem_init_hw()
+------> ring->init_hw() which is init_render_ring()
+----------> init_ring_common()
+------------> intel_ring_setup_status_page()

Gen8:
i915_reset()

+--> i915_gem_init_hw()
+------> ring->init_hw() which is gen8_init_render_ring()
+--------> gen8_init_common_ring() - I added changes in this function.

We could probably use intel_ring_setup_status_page() for both cases, does it have to be Gen7 specific?

Quoted 1 line

Also I wonder about resume, where's the HWS_PGA restore for that case?

It is covered.

i915_drm_resume()
+-->i915_gem_init_hw

regards
Arun

Quoted 2 lines

-Daniel

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#6 lore raw
Quoted 31 lines · Arun
Quoted 9 lines · Daniel Vetter
Quoted 6 lines · Ville Syrjälä

I was going to suggest removing the same thing from the lrc_setup_hardware_status_page(), but after another look it seems we sometimes call .init_hw() before the context setup. Would be nice to have a more consistent sequence for init and reset. But anyway the patch looks OK to me. I verified that we indeed lose this register on GPU reset.

Quoted 8 lines

Yep, this is a mess. And historically _any_ difference between driver load and gpu reset (or resume fwiw) has lead to hilarious bugs, so this difference is really troubling to me. Arun, can you please work on a patch to unify the setup sequence here, so that both driver load gpu resets work the same way? By the time we're calling gem_init_hw the default context should have been created already, and hence we should be able to write to HWS_PGA in ring->init_hw only.

Hi Daniel,

I think the problem in this case was the code to init HWS page after reset was missing for Gen8+. For Gen7 we are doing this as part of ring->init_hw.

Gen7:
i915_reset()

+--> i915_gem_init_hw()
+------> ring->init_hw() which is init_render_ring()
+----------> init_ring_common()
+------------> intel_ring_setup_status_page()
Quoted 8 lines

Gen8:
i915_reset()

+--> i915_gem_init_hw()
+------> ring->init_hw() which is gen8_init_render_ring()
+--------> gen8_init_common_ring() - I added changes in this function.

We could probably use intel_ring_setup_status_page() for both cases, does it have to be Gen7 specific?

My concern isn't that we have two functions doing hws setup. My concern is that we now have 2 callsites for execlist mode doing hws setup, with slight differences between reset/driver load and resume.

I want one, unconditional call to set up the hws page at exactly the right place in the setup sequence. That might require some refactoring, I haven't looked that closely at intel_lrc.c

The usual approach is that gem_init does exclusively software setup, and gem_init_hw does all the register writes an actual enabling. I think the hws setup in the driver load code is currently called from gem_init(), which is the wrong place.

Quoted 5 lines
Quoted 1 line

Also I wonder about resume, where's the HWS_PGA restore for that case?

It is covered.

i915_drm_resume()
+-->i915_gem_init_hw

Ok, so should be covered with whatever fix we have for gpu reset.

Thanks, Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx