Skip to contentSkip to conversation

Hack to avoid C-state reduction during graphics activity.

RFC In review Eric Anholt 5 participants 11 messages Nov 1, 2010 – Nov 2, 2010

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

This is a series I came up with as a result of KS discussions. The plan was to pin the CPU C state to keep the GPU going as fast as possible while it was active, since there's some relation between the two (we don't know for sure yet, per chipset, whether it's due to latency of DMA writes having to bring CPU state up, or reduced memory bandwidth due to bus frequency reduction, or other). The actual patch series doesn't achieve that -- a hint is provided to the scheduler and cpufreq/cpuidle, which may choose not to sleep as deeply as it would otherwise.

I found a 1% performance difference in nexuiz on my Ironlake system from the last patch in this series (HEAD~1 didn't have a statistically significant effect). If that's all the difference, I'll drop this. It may pay off higher for non-Ironlake, in which case I'll work on a "proper" solution of actually clamping C-states while graphics is active.

#1 lore raw

PATCH 1/4 sched: Export io_schedule_timeout()

This is the logical partner to the already exported io_schedule() and schedule_timeout(), and will be used by the DRM.

---
 kernel/sched.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
kernel/sched.c +1 -0
index aa14a56..ec853f9 100644
@@ -5287,6 +5287,7 @@ long __sched io_schedule_timeout(long timeout)
 	delayacct_blkio_end();
 	return ret;
 }
+EXPORT_SYMBOL(io_schedule_timeout);
 
 /**
  * sys_sched_get_priority_max - return maximum RT priority.
Signature · 1 line
1.7.2.3
#2 lore raw

PATCH 2/4 Add io_ variants of wait_event() and wait_event_interruptible()

In reply to Eric Anholt

These tell the scheduler that we're waiting on IO, and still "busy".

---
 include/linux/wait.h |   46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 46 insertions(+), 0 deletions(-)
include/linux/wait.h +46 -0
index 3efc9f3..b299c2b 100644
@@ -202,6 +202,19 @@ do {									\
 	finish_wait(&wq, &__wait);					\
 } while (0)
 
+#define __io_wait_event(wq, condition) 					\
+do {									\
+	DEFINE_WAIT(__wait);						\
+									\
+	for (;;) {							\
+		prepare_to_wait(&wq, &__wait, TASK_UNINTERRUPTIBLE);	\
+		if (condition)						\
+			break;						\
+		io_schedule();						\
+	}								\
+	finish_wait(&wq, &__wait);					\
+} while (0)
+
 /**
  * wait_event - sleep until a condition gets true
  * @wq: the waitqueue to wait on
@@ -221,6 +234,13 @@ do {									\
 	__wait_event(wq, condition);					\
 } while (0)
 
+#define io_wait_event(wq, condition) 					\
+do {									\
+	if (condition)	 						\
+		break;							\
+	__io_wait_event(wq, condition);					\
+} while (0)
+
 #define __wait_event_timeout(wq, condition, ret)			\
 do {									\
 	DEFINE_WAIT(__wait);						\
@@ -260,6 +280,24 @@ do {									\
 	__ret;								\
 })
 
+#define __io_wait_event_interruptible(wq, condition, ret)		\
+do {									\
+	DEFINE_WAIT(__wait);						\
+									\
+	for (;;) {							\
+		prepare_to_wait(&wq, &__wait, TASK_INTERRUPTIBLE);	\
+		if (condition)						\
+			break;						\
+		if (!signal_pending(current)) {				\
+			io_schedule();					\
+			continue;					\
+		}							\
+		ret = -ERESTARTSYS;					\
+		break;							\
+	}								\
+	finish_wait(&wq, &__wait);					\
+} while (0)
+
 #define __wait_event_interruptible(wq, condition, ret)			\
 do {									\
 	DEFINE_WAIT(__wait);						\
@@ -301,6 +339,14 @@ do {									\
 	__ret;								\
 })
 
+#define io_wait_event_interruptible(wq, condition)			\
+({									\
+	int __ret = 0;							\
+	if (!(condition))						\
+		__io_wait_event_interruptible(wq, condition, __ret);	\
+	__ret;								\
+})
+
 #define __wait_event_interruptible_timeout(wq, condition, ret)		\
 do {									\
 	DEFINE_WAIT(__wait);						\
Signature · 1 line
1.7.2.3
#3 lore raw

PATCH 3/4 drm/i915: Declare waits on GPU as io waits, to reduce C-state reduction.

In reply to Eric Anholt

Because of this, the C-state governor will consider our process "performance critical" and avoid dropping CPU power during these temporary sleeps. There are two benefits of this: We don't lose our cache contents (whose value is not well reflected in the metrics of the governor), and also don't reduce memory bandwidth, which can reduce GPU performance and increase the time we wait!

---
 drivers/gpu/drm/i915/i915_gem.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/i915_gem.c +2 -2
index 8eb8453..8991588 100644
@@ -1988,12 +1988,12 @@ i915_do_wait_request(struct drm_device *dev, uint32_t seqno,
 		ring->waiting_gem_seqno = seqno;
 		ring->user_irq_get(dev, ring);
 		if (interruptible)
-			ret = wait_event_interruptible(ring->irq_queue,
+			ret = io_wait_event_interruptible(ring->irq_queue,
 				i915_seqno_passed(
 					ring->get_seqno(dev, ring), seqno)
 				|| atomic_read(&dev_priv->mm.wedged));
 		else
-			wait_event(ring->irq_queue,
+			io_wait_event(ring->irq_queue,
 				i915_seqno_passed(
 					ring->get_seqno(dev, ring), seqno)
 				|| atomic_read(&dev_priv->mm.wedged));
Signature · 1 line
1.7.2.3
#4 lore raw

PATCH 4/4 cpuidle: Hack iowait weighting to avoid C-state reduction for graphics.

In reply to Eric Anholt

Improves nexuiz performance by about 1% on my system.

---
 drivers/cpuidle/governors/menu.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/cpuidle/governors/menu.c +1 -1
index f508690..9bb5654 100644
@@ -175,7 +175,7 @@ static inline int performance_multiplier(void)
 	mult += 2 * get_loadavg();
 
 	/* for IO wait tasks (per cpu!) we add 5x each */
-	mult += 10 * nr_iowait_cpu(smp_processor_id());
+	mult += 1000 * nr_iowait_cpu(smp_processor_id());
 
 	return mult;
 }
Signature · 1 line
1.7.2.3
#5 lore raw

Re: [Intel-gfx] [PATCH 4/4] cpuidle: Hack iowait weighting to avoid C-state reduction for graphics.

Quoted 1 line · Eric Anholt

Improves nexuiz performance by about 1% on my system.

CC Arjan and linux-acpi mailing list.

It seems that the selection of C-state will become very sensitive to IO wait after the multi factor is changed from 10 to 1000. Maybe only one IO/wait will bring that the system can't enter the deep C-state.

Quoted 19 lines
---
 drivers/cpuidle/governors/menu.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/cpuidle/governors/menu.c +1 -1
index f508690..9bb5654 100644
@@ -175,7 +175,7 @@ static inline int performance_multiplier(void)
 	mult += 2 * get_loadavg();
 
 	/* for IO wait tasks (per cpu!) we add 5x each */
-	mult += 10 * nr_iowait_cpu(smp_processor_id());
+	mult += 1000 * nr_iowait_cpu(smp_processor_id());
 
 	return mult;
 }
#6 lore raw

Re: [PATCH 4/4] cpuidle: Hack iowait weighting to avoid C-state reduction for graphics.

Quoted 9 lines · ykzhao
Quoted 1 line · Eric Anholt

Improves nexuiz performance by about 1% on my system.

CC Arjan and linux-acpi mailing list.

It seems that the selection of C-state will become very sensitive to IO wait after the multi factor is changed from 10 to 1000. Maybe only one IO/wait will bring that the system can't enter the deep C-state.

Note the original message that said "This patch series is a hack, if it helps anyone I'll rewrite it to do the right thing".

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#7 lore raw

Re: [Intel-gfx] [PATCH 4/4] cpuidle: Hack iowait weighting to avoid C-state reduction for graphics.

Am Dienstag, den 02.11.2010, 08:42 -0700 schrieb Eric Anholt:

Quoted 6 lines · ykzhao
Quoted 5 lines · Eric Anholt
Quoted 1 line

Improves nexuiz performance by about 1% on my system.

CC Arjan and linux-acpi mailing list.
Quoted 7 lines
Quoted 4 lines

It seems that the selection of C-state will become very sensitive to IO wait after the multi factor is changed from 10 to 1000. Maybe only one IO/wait will bring that the system can't enter the deep C-state.

Note the original message that said "This patch series is a hack, if it helps anyone I'll rewrite it to do the right thing".

If this patches about perforamnce issue on 9450gm and sleep state on CPU, than it do not work for me.

I applied your patches on the top of 3e7b033 (drm/i915: Use the agp_size determined from the GTT), the 3/4 was rejected so id it manually.

Same result: graphic perfomence drop on smp/ht system if cpu in C4.

Signature · 2 lines
Regards,
        Alexey
#8 lore raw

Re: [Intel-gfx] [PATCH 4/4] cpuidle: Hack iowait weighting to avoid C-state reduction for graphics.

Quoted 7 lines · Alexey Fisher

If this patches about perforamnce issue on 9450gm and sleep state on CPU, than it do not work for me.

I applied your patches on the top of 3e7b033 (drm/i915: Use the agp_size determined from the GTT), the 3/4 was rejected so id it manually.

Same result: graphic perfomence drop on smp/ht system if cpu in C4.

Useful check. The patches proposed should only affect readback and throttling, when the driver is stuck waiting for the GPU with the device mutex held. The scenario with missing vblank wakeups is that it is the application waiting in poll for the event to be sent from the driver. In order to have an effect, we would need to prevent the CPU from dropping below C2? C1? whilst the vblank interrupt is unmasked i.e drm_vblank_get(). That would also hopefully have a bigger impact on both performance and battery life for Intel GPUs/
-Chris

Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre
#9 lore raw
In reply to Eric Anholt

Out of curiousity, what happened to ACPI_BM_BREAK_EN here:

https://bugs.freedesktop.org/show_bug.cgi?id=30364

On my box, that register is (according to setpci) 0x00, but I'm testing on an ICH9M and I don't seem to be affected anyway.

--Andy

Quoted 22 lines · Eric Anholt

This is a series I came up with as a result of KS discussions.  The plan was to pin the CPU C state to keep the GPU going as fast as possible while it was active, since there's some relation between the two (we don't know for sure yet, per chipset, whether it's due to latency of DMA writes having to bring CPU state up, or reduced memory bandwidth due to bus frequency reduction, or other).  The actual patch series doesn't achieve that -- a hint is provided to the scheduler and cpufreq/cpuidle, which may choose not to sleep as deeply as it would otherwise.

I found a 1% performance difference in nexuiz on my Ironlake system from the last patch in this series (HEAD~1 didn't have a statistically significant effect).  If that's all the difference, I'll drop this. It may pay off higher for non-Ironlake, in which case I'll work on a "proper" solution of actually clamping C-states while graphics is active.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#10 lore raw
Quoted 6 lines · Andrew Lutomirski

Out of curiousity, what happened to ACPI_BM_BREAK_EN here:

https://bugs.freedesktop.org/show_bug.cgi?id=30364

On my box, that register is (according to setpci) 0x00, but I'm testing on an ICH9M and I don't seem to be affected anyway.

Those registers aren't there on Ironlake.

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx