Skip to contentSkip to conversation

drm/i915: Ironlake GPU with VT-d fix

PATCH Superseded Ben Widawsky 3 participants 6 messages Sep 23, 2011

Subsystems (3)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/i915: Ironlake GPU with VT-d fix

Under certain circumstances, an IOTLB flush never completes and the hardware just locks up. The fix is meant to idle the GPU both before and after any map or unmap occurs.

It requires an additional IOMMU patch.

Cc Dave Airlie <[email protected]>
Cc David Woodhouse <[email protected]>
Cc Keith Packard <[email protected]>
---
 drivers/char/agp/intel-gtt.c        |   10 +++++++
 drivers/gpu/drm/i915/i915_gem.c     |    2 +-
 drivers/gpu/drm/i915/i915_gem_gtt.c |   50 +++++++++++++++++++++++++++++++++++
 include/drm/intel-gtt.h             |    2 +
 4 files changed, 63 insertions(+), 1 deletions(-)
drivers/char/agp/intel-gtt.c +10 -0
index 8515101..9d939f0 100644
@@ -650,6 +650,7 @@ static void intel_gtt_cleanup(void)
 static int intel_gtt_init(void)
 {
 	u32 gtt_map_size;
+	const unsigned short gpu_devid = intel_private.pcidev->device;
 	int ret;
 
 	ret = intel_private.driver->setup();
@@ -688,6 +689,11 @@ static int intel_gtt_init(void)
 
 	intel_private.base.needs_dmar = USE_PCI_DMA_API && INTEL_GTT_GEN > 2;
 
+	if ((gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_HB ||
+	     gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_IG) &&
+	     intel_private.base.needs_dmar)
+		intel_private.base.do_idle_maps = 1;
+
 	ret = intel_gtt_setup_scratch_page();
 	if (ret != 0) {
 		intel_gtt_cleanup();
@@ -923,6 +929,8 @@ static int intel_fake_agp_insert_entries(struct agp_memory *mem,
 {
 	int ret = -EINVAL;
 
+	BUG_ON(intel_private.base.do_idle_maps);
+
 	if (intel_private.clear_fake_agp) {
 		int start = intel_private.base.stolen_size / PAGE_SIZE;
 		int end = intel_private.base.gtt_mappable_entries;
@@ -985,6 +993,8 @@ static int intel_fake_agp_remove_entries(struct agp_memory *mem,
 	if (mem->page_count == 0)
 		return 0;
 
+	BUG_ON(intel_private.base.do_idle_maps);
+
 	intel_gtt_clear_range(pg_start, mem->page_count);
 
 	if (intel_private.base.needs_dmar) {
drivers/gpu/drm/i915/i915_gem.c +1 -1
index a546a71..785b84a 100644
@@ -2277,7 +2277,7 @@ i915_gpu_idle(struct drm_device *dev)
 
 	lists_empty = (list_empty(&dev_priv->mm.flushing_list) &&
 		       list_empty(&dev_priv->mm.active_list));
-	if (lists_empty)
+	if (lists_empty && !!dev_priv->mm.gtt->do_idle_maps)
 		return 0;
 
 	/* Flush everything onto the inactive list. */
drivers/gpu/drm/i915/i915_gem_gtt.c +50 -0

100 lines not on this page · raw message

include/drm/intel-gtt.h +2 -0
index 9e343c0..b174620 100644
@@ -13,6 +13,8 @@ const struct intel_gtt {
 	unsigned int gtt_mappable_entries;
 	/* Whether i915 needs to use the dmar apis or not. */
 	unsigned int needs_dmar : 1;
+	/* Whether we idle the gpu before mapping/unmapping */
+	unsigned int do_idle_maps : 1;
 } *intel_gtt_get(void);
 
 void intel_gtt_chipset_flush(void);
Signature · 1 line
1.7.6.3
#1 lore raw
Quoted 1 line · Ben Widawsky

It requires an additional IOMMU patch.

Can we collect those two patches into one sequence?

drivers/char/agp/intel-gtt.c static int intel_gtt_init(void) · line 696

Quoted 5 lines
drivers/char/agp/intel-gtt.c +5 -0
+	if ((gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_HB ||
+	     gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_IG) &&
+	     intel_private.base.needs_dmar)
+		intel_private.base.do_idle_maps = 1;
+

I'd like to make this conditional on whether IOMMU is actually in use; needs_dmar is based solely on whether the DMA_API is compiled into the kernel and the GTT gen is > 2.

drivers/gpu/drm/i915/i915_gem.c i915_gpu_idle(struct drm_device *dev) · line 2281

Quoted 3 lines
drivers/gpu/drm/i915/i915_gem.c +1 -1
-	if (lists_empty)
+	if (lists_empty && !!dev_priv->mm.gtt->do_idle_maps)
 		return 0;

Is it necessary to change the semantic of this function in cases which aren't related to GTT remapping? Seems like you're imposing a fairly high cost on operations which don't actually need it.

Signature · 1 line
[email protected]

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#2 lore raw
Quoted 7 lines · On Thu, 22 Sep 2011 21:50:49 -0700 Keith Packard <[email protected]> wrote:
Quoted 1 line · Ben Widawsky

It requires an additional IOMMU patch.

Can we collect those two patches into one sequence?

Yes. Although not sure who would do the pull request to Linus.

Quoted 9 lines

drivers/char/agp/intel-gtt.c static int intel_gtt_init(void) · line 696

Quoted 5 lines
drivers/char/agp/intel-gtt.c +5 -0
+	if ((gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_HB ||
+	     gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_IG) &&
+	     intel_private.base.needs_dmar)
+		intel_private.base.do_idle_maps = 1;
+

I'd like to make this conditional on whether IOMMU is actually in use; needs_dmar is based solely on whether the DMA_API is compiled into the kernel and the GTT gen is > 2.

David did mention a way for me to detect it. I wasn't sure if there was a deadline to get this patch out so I submitted it now. I will work on the detection portion of it while the rest gets review. It's something like, check if the bus address == the physical address and if not, assume the IOMMU is in use.

Quoted 9 lines

drivers/gpu/drm/i915/i915_gem.c i915_gpu_idle(struct drm_device *dev) · line 2281

Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +1 -1
-	if (lists_empty)
+	if (lists_empty && !!dev_priv->mm.gtt->do_idle_maps)
 		return 0;

Is it necessary to change the semantic of this function in cases which aren't related to GTT remapping? Seems like you're imposing a fairly high cost on operations which don't actually need it.

Not sure I follow. On that specific hunk it's apparently needed. I tried the patch without this and we got some data back which suggested it didn't work. It would be nice if we could reproduce this locally...

Ben

#3 lore raw
Quoted 23 lines · Ben Widawsky
Quoted 7 lines · On Thu, 22 Sep 2011 21:50:49 -0700 Keith Packard <[email protected]> wrote:
Quoted 1 line · Ben Widawsky

It requires an additional IOMMU patch.

Can we collect those two patches into one sequence?

Yes. Although not sure who would do the pull request to Linus.

Quoted 9 lines

drivers/char/agp/intel-gtt.c static int intel_gtt_init(void) · line 696

Quoted 5 lines
drivers/char/agp/intel-gtt.c +5 -0
+	if ((gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_HB ||
+	     gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_IG) &&
+	     intel_private.base.needs_dmar)
+		intel_private.base.do_idle_maps = 1;
+

I'd like to make this conditional on whether IOMMU is actually in use; needs_dmar is based solely on whether the DMA_API is compiled into the kernel and the GTT gen is > 2.

Quoted 5 lines

David did mention a way for me to detect it. I wasn't sure if there was a deadline to get this patch out so I submitted it now. I will work on the detection portion of it while the rest gets review. It's something like, check if the bus address == the physical address and if not, assume the IOMMU is in use.

And for starters, we can check no_iommu. But really we want intel_iommu.c to report whether or not it enabled VTd on the gfx.

Quoted 10 lines
Quoted 9 lines

drivers/gpu/drm/i915/i915_gem.c i915_gpu_idle(struct drm_device *dev) · line 2281

Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +1 -1
-	if (lists_empty)
+	if (lists_empty && !!dev_priv->mm.gtt->do_idle_maps)
 		return 0;

Is it necessary to change the semantic of this function in cases which aren't related to GTT remapping? Seems like you're imposing a fairly high cost on operations which don't actually need it.

Quoted 3 lines

Not sure I follow. On that specific hunk it's apparently needed. I tried the patch without this and we got some data back which suggested it didn't work. It would be nice if we could reproduce this locally...

Keith is right, there should be no reason to touch that function, and especially not for the normal code paths.

Also we should not introduce BUG_ON() trivially into user code paths (/dev/agp).
-Chris

Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre
#4 lore raw
Quoted 44 lines · On Fri, 23 Sep 2011 08:37:10 +0100 Chris Wilson <[email protected]>…
Quoted 23 lines · Ben Widawsky
Quoted 7 lines · On Thu, 22 Sep 2011 21:50:49 -0700 Keith Packard <[email protected]> wrote:
Quoted 1 line · Ben Widawsky

It requires an additional IOMMU patch.

Can we collect those two patches into one sequence?

Yes. Although not sure who would do the pull request to Linus.

Quoted 9 lines

drivers/char/agp/intel-gtt.c static int intel_gtt_init(void) · line 696

Quoted 5 lines
drivers/char/agp/intel-gtt.c +5 -0
+	if ((gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_HB ||
+	     gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_IG) &&
+	     intel_private.base.needs_dmar)
+		intel_private.base.do_idle_maps = 1;
+

I'd like to make this conditional on whether IOMMU is actually in use; needs_dmar is based solely on whether the DMA_API is compiled into the kernel and the GTT gen is > 2.

Quoted 5 lines

David did mention a way for me to detect it. I wasn't sure if there was a deadline to get this patch out so I submitted it now. I will work on the detection portion of it while the rest gets review. It's something like, check if the bus address == the physical address and if not, assume the IOMMU is in use.

And for starters, we can check no_iommu. But really we want intel_iommu.c to report whether or not it enabled VTd on the gfx.

Quoted 10 lines
Quoted 9 lines

drivers/gpu/drm/i915/i915_gem.c i915_gpu_idle(struct drm_device *dev) · line 2281

Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +1 -1
-	if (lists_empty)
+	if (lists_empty && !!dev_priv->mm.gtt->do_idle_maps)
 		return 0;

Is it necessary to change the semantic of this function in cases which aren't related to GTT remapping? Seems like you're imposing a fairly high cost on operations which don't actually need it.

Quoted 7 lines
Quoted 4 lines

Not sure I follow. On that specific hunk it's apparently needed. I tried the patch without this and we got some data back which suggested it didn't work. It would be nice if we could reproduce this locally...

Keith is right, there should be no reason to touch that function, and especially not for the normal code paths.

Crap... This is a bug in the patch.

Quoted 3 lines

Also we should not introduce BUG_ON() trivially into user code paths (/dev/agp).

Remove 'em then?

Quoted 2 lines

-Chris

#5 lore raw
Quoted 52 lines · Ben Widawsky
Quoted 49 lines · On Fri, 23 Sep 2011 08:37:10 +0100 Chris Wilson <[email protected]>…
Quoted 23 lines · Ben Widawsky
Quoted 7 lines · On Thu, 22 Sep 2011 21:50:49 -0700 Keith Packard <[email protected]> wrote:
Quoted 1 line · Ben Widawsky

It requires an additional IOMMU patch.

Can we collect those two patches into one sequence?

Yes. Although not sure who would do the pull request to Linus.

Quoted 9 lines

drivers/char/agp/intel-gtt.c static int intel_gtt_init(void) · line 696

Quoted 5 lines
drivers/char/agp/intel-gtt.c +5 -0
+	if ((gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_HB ||
+	     gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_IG) &&
+	     intel_private.base.needs_dmar)
+		intel_private.base.do_idle_maps = 1;
+

I'd like to make this conditional on whether IOMMU is actually in use; needs_dmar is based solely on whether the DMA_API is compiled into the kernel and the GTT gen is > 2.

Quoted 5 lines

David did mention a way for me to detect it. I wasn't sure if there was a deadline to get this patch out so I submitted it now. I will work on the detection portion of it while the rest gets review. It's something like, check if the bus address == the physical address and if not, assume the IOMMU is in use.

And for starters, we can check no_iommu. But really we want intel_iommu.c to report whether or not it enabled VTd on the gfx.

Quoted 14 lines
Quoted 9 lines

drivers/gpu/drm/i915/i915_gem.c i915_gpu_idle(struct drm_device *dev) · line 2281

Quoted 4 lines
drivers/gpu/drm/i915/i915_gem.c +1 -1
-	if (lists_empty)
+	if (lists_empty && !!dev_priv->mm.gtt->do_idle_maps)
 		return 0;

Is it necessary to change the semantic of this function in cases which aren't related to GTT remapping? Seems like you're imposing a fairly high cost on operations which don't actually need it.

Not sure I follow. On that specific hunk it's apparently needed. I tried the patch without this and we got some data back which suggested it didn't work. It would be nice if we could reproduce this locally...

Quoted 4 lines
Quoted 2 lines

Keith is right, there should be no reason to touch that function, and especially not for the normal code paths.

Crap... This is a bug in the patch.

I'm still can't explain the necessity of forcing the i915_ring_idle() here. If those two lists are empty, then each of the ring write/active lists should also be empty and it will be a no-op. Hence bafflement and time to be slightly afraid.

Quoted 5 lines
Quoted 3 lines

Also we should not introduce BUG_ON() trivially into user code paths (/dev/agp).

Remove 'em then?

Not quite, in this case I think returning -ENODEV is the sensible approach.
-Chris

Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre