Skip to contentSkip to conversation

Ironlake mobile GPU with VT-d, fix

PATCH In review Ben Widawsky 2 participants 6 messages Sep 26, 2011 – Oct 11, 2011

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v3 cover Ironlake mobile GPU with VT-d, fix

v1: (tested)
idle GPU before and after unmaps.

v2: (untested)
Read GMCH_CTRL to find if VT-d is enable
Pulled David's patch into the series
Only stall for unmaps
Add warning for list_empty && !rings_idle

v3: (still untested)
Removed warning stuff, got rid of early exit in i915_gpu_idle completely Pulled logic to detect effected chipsets into a new function Left the interruptible=true logic because I still think it's required,
and I can't see how it hurts terribly badly.

Ben Widawsky (2):
drm/i915: Remove early exit on i915_gpu_idle drm/i915: Fix hang on Ironlake mobile GPU with VT-d

David Woodhouse (1):
intel-iommu: IOTLB hang workaround

 drivers/char/agp/intel-gtt.c        |   33 ++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem.c     |    6 -----
 drivers/gpu/drm/i915/i915_gem_gtt.c |   36 +++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-iommu.c         |    6 ++++-
 include/drm/intel-gtt.h             |    2 +
 5 files changed, 76 insertions(+), 7 deletions(-)
Signature · 1 line
1.7.6.4
#1 lore raw

PATCH v3 1/3 drm/i915: Remove early exit on i915_gpu_idle

[Description from: Daniel Vetter]
I've just discussed this quickly with Chris on irc and it's probably best to just kill the list_empty early bailout. gpu_idle isn't a fastpath, so who cares. One candidate where we emit commands to the ring without adding anything onto these lists is e.g. pageflip. There are probably more.

Cc Daniel Vetter <[email protected]>
Cc Chris Wilson <[email protected]>
---
 drivers/gpu/drm/i915/i915_gem.c |    6 ------
 1 files changed, 0 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +0 -6
index a546a71..7867589 100644
@@ -2272,14 +2272,8 @@ int
 i915_gpu_idle(struct drm_device *dev)
 {
 	drm_i915_private_t *dev_priv = dev->dev_private;
-	bool lists_empty;
 	int ret, i;
 
-	lists_empty = (list_empty(&dev_priv->mm.flushing_list) &&
-		       list_empty(&dev_priv->mm.active_list));
-	if (lists_empty)
-		return 0;
-
 	/* Flush everything onto the inactive list. */
 	for (i = 0; i < I915_NUM_RINGS; i++) {
 		ret = i915_ring_idle(&dev_priv->ring[i]);
Signature · 1 line
1.7.6.4
#2 lore raw

PATCH v3 2/3 drm/i915: Fix hang on Ironlake mobile GPU with VT-d

In reply to Ben Widawsky

Under certain circumstances, an IOTLB flush never completes and the hardware just locks up. The fix is meant to idle the GPU both before and after any unmap occurs, which should prevent the possibity to hang.

---
 drivers/char/agp/intel-gtt.c        |   33 ++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_gtt.c |   36 +++++++++++++++++++++++++++++++++++
 include/drm/intel-gtt.h             |    2 +
 3 files changed, 71 insertions(+), 0 deletions(-)
drivers/char/agp/intel-gtt.c +33 -0
index 8515101..319bc3a 100644
@@ -923,6 +923,9 @@ static int intel_fake_agp_insert_entries(struct agp_memory *mem,
 {
 	int ret = -EINVAL;
 
+	if (intel_private.base.do_idle_maps)
+		return -ENODEV;
+
 	if (intel_private.clear_fake_agp) {
 		int start = intel_private.base.stolen_size / PAGE_SIZE;
 		int end = intel_private.base.gtt_mappable_entries;
@@ -985,6 +988,9 @@ static int intel_fake_agp_remove_entries(struct agp_memory *mem,
 	if (mem->page_count == 0)
 		return 0;
 
+	if (intel_private.base.do_idle_maps)
+		return -ENODEV;
+
 	intel_gtt_clear_range(pg_start, mem->page_count);
 
 	if (intel_private.base.needs_dmar) {
@@ -1177,6 +1183,30 @@ static void gen6_cleanup(void)
 {
 }
 
+/* Certain Gen5 chipsets require require idling the GPU before
+ * unmapping anything from the GTT when VT-d is enabled.
+ */
+static inline int needs_idle_maps(void)
+{
+	u16 gmch_ctrl;
+	const unsigned short gpu_devid = intel_private.pcidev->device;
+	bool needs_dmar = USE_PCI_DMA_API;
+
+	pci_read_config_word(intel_private.bridge_dev, I830_GMCH_CTRL,
+			     &gmch_ctrl);
+
+	/* Check if BIOS has enabled GTT w/VT-d. It may create some false
+	 * positives, but VT-d shouldn't be enabled unless this is set.
+	 */
+	if ((gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_HB ||
+	     gpu_devid == PCI_DEVICE_ID_INTEL_IRONLAKE_M_IG) &&
+	     (gmch_ctrl & G4x_GMCH_SIZE_VT_EN) &&
+	     USE_PCI_DMA_API)
+		return 1;
+
+	return 0;
+}
+
 static int i9xx_setup(void)
 {
 	u32 reg_addr;
@@ -1211,6 +1241,9 @@ static int i9xx_setup(void)
 		intel_private.gtt_bus_addr = reg_addr + gtt_offset;
 	}
 
+	if (needs_idle_maps());
+		intel_private.base.do_idle_maps = 1;
+
 	intel_i9xx_setup_flush();
 
 	return 0;
drivers/gpu/drm/i915/i915_gem_gtt.c +36 -0
index 7a709cd..905045d 100644
@@ -49,6 +49,34 @@ static unsigned int cache_level_to_agp_type(struct drm_device *dev,
 	}
 }
 
+static bool do_idling(struct drm_i915_private *dev_priv)
+{
+	bool ret = dev_priv->mm.interruptible;
+
+	if (dev_priv->mm.gtt->do_idle_maps) {
+		dev_priv->mm.interruptible = false;
+		if (i915_gpu_idle(dev_priv->dev))
+			DRM_ERROR("couldn't idle GPU %d\n", ret);
+	}
+
+	return ret;
+}
+
+static void undo_idling(struct drm_i915_private *dev_priv, bool idle)
+{
+	int ret;
+	if (!dev_priv->mm.gtt->do_idle_maps)
+		return;
+
+	/*NB: since GTT mappings don't actually touch the GPU, this is not
+	 * strictly necessary.
+	 */
+	ret = i915_gpu_idle(dev_priv->dev);
+	if (ret)
+		DRM_ERROR("couldn't idle GPU %d\n", ret);
+	dev_priv->mm.interruptible = idle;
+}
+
 void i915_gem_restore_gtt_mappings(struct drm_device *dev)
 {
 	struct drm_i915_private *dev_priv = dev->dev_private;
@@ -117,6 +145,12 @@ void i915_gem_gtt_rebind_object(struct drm_i915_gem_object *obj,
 
 void i915_gem_gtt_unbind_object(struct drm_i915_gem_object *obj)
 {
+	struct drm_device *dev = obj->base.dev;
+	struct drm_i915_private *dev_priv = dev->dev_private;
+	bool idle;
+
+	idle = do_idling(dev_priv);
+
 	intel_gtt_clear_range(obj->gtt_space->start >> PAGE_SHIFT,
 			      obj->base.size >> PAGE_SHIFT);
 
@@ -124,4 +158,6 @@ void i915_gem_gtt_unbind_object(struct drm_i915_gem_object *obj)
 		intel_gtt_unmap_memory(obj->sg_list, obj->num_sg);
 		obj->sg_list = NULL;
 	}
+
+	undo_idling(dev_priv, idle);
 }
include/drm/intel-gtt.h +2 -0
index 9e343c0..b174620 100644
@@ -13,6 +13,8 @@ const struct intel_gtt {
 	unsigned int gtt_mappable_entries;
 	/* Whether i915 needs to use the dmar apis or not. */
 	unsigned int needs_dmar : 1;
+	/* Whether we idle the gpu before mapping/unmapping */
+	unsigned int do_idle_maps : 1;
 } *intel_gtt_get(void);
 
 void intel_gtt_chipset_flush(void);
Signature · 1 line
1.7.6.4
maintainer
#3 lore raw

Re: [PATCH v3 2/3] drm/i915: Fix hang on Ironlake mobile GPU with VT-d

Quoted 3 lines

Under certain circumstances, an IOTLB flush never completes and the hardware just locks up.  The fix is meant to idle the GPU both before and after any unmap occurs, which should prevent the possibity to hang.

One comment below, noticed by building it for RHEL6.

drivers/char/agp/intel-gtt.c static void gen6_cleanup(void) · line 1193

Quoted 9 lines
drivers/char/agp/intel-gtt.c +8 -0
+/* Certain Gen5 chipsets require require idling the GPU before
+ * unmapping anything from the GTT when VT-d is enabled.
+ */
+static inline int needs_idle_maps(void)
+{
+       u16 gmch_ctrl;
+       const unsigned short gpu_devid = intel_private.pcidev->device;
+       bool needs_dmar = USE_PCI_DMA_API;

^^ you don't use needs_dmar anywhere.

Dave.

#4 lore raw

PATCH v3 3/3 intel-iommu: IOTLB hang workaround

In reply to Ben Widawsky

From: David Woodhouse <[email protected]>

To work around a hardware issue, we have to submit IOTLB flushes while the graphics engine is idle. The graphics driver will go to great lengths to ensure that it gets that right on the affected chipset(s)... so let's not screw it over by deferring the unmap and doing it later. That wouldn't be very helpful.

---
 drivers/iommu/intel-iommu.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)
drivers/iommu/intel-iommu.c +5 -1
index c621c98..dcdc3c7 100644
@@ -3950,7 +3950,11 @@ static void __devinit quirk_calpella_no_shadow_gtt(struct pci_dev *dev)
 	if (!(ggc & GGC_MEMORY_VT_ENABLED)) {
 		printk(KERN_INFO "DMAR: BIOS has allocated no shadow GTT; disabling IOMMU for graphics\n");
 		dmar_map_gfx = 0;
-	}
+	} else {
+		/* we have to ensure the gfx device is idle before we flush */
+		printk(KERN_INFO "DMAR: Disabling batched IOTLB flush on Ironlake\n");
+		intel_iommu_strict = 1;
+       }
 }
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x0040, quirk_calpella_no_shadow_gtt);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x0044, quirk_calpella_no_shadow_gtt);
Signature · 1 line
1.7.6.4
maintainer Tested-by on the v3 series
#5 lore raw
In reply to Ben Widawsky
Quoted 9 lines · Ben Widawsky

v1: (tested)
idle GPU before and after unmaps.

v2: (untested)
Read GMCH_CTRL to find if VT-d is enable
Pulled David's patch into the series
Only stall for unmaps
Add warning for list_empty && !rings_idle

Quoted 5 lines

v3: (still untested)
Removed warning stuff, got rid of early exit in i915_gpu_idle completely Pulled logic to detect effected chipsets into a new function Left the interruptible=true logic because I still think it's required, and I can't see how it hurts terribly badly.

On my crashy ILK HP laptop, ran my find + g-t + compiz test all night.

Dave.