Skip to contentSkip to conversation

nohz delay performance optimization

RFC PATCH New Martin Schwidefsky 1 participant 2 messages Aug 10, 2009

Subsystems (3)
s390 Nohz, dynticks support s390 common I/O layer
lore
Share & tools
lore

Martin Schwidefsky

author
#0 lore raw

cover nohz delay performance optimization

Greetings,
I'm still working on the cpu idle/wakeup path to improve the performance of some ping-pong workloads. This patch is the latest idea and proved rather effective. The idea is that the timer tick is not switched off if the cpu did some work in the last tick period. This will add an unnecessary tick interrupt if the cpu goes truly idle but will save some cycles for the nohz reprogramming if the cpu wakes up again for more work.

Currently the patch only works on s390, the arch_needs_cpu() will be true if there has been any other interrupt than timer interrupts in the last tick period. I think this could be improved with an explicit call to request a nohz delay. The call should be added to selective points in the code, e.g. whenever a process != idle is scheduled, whenever a bottom half is run, and so on.

Comments?

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.
author
#1 lore raw

RFC PATCH 1/1 nohz: introduce arch_needs_cpu

From: Martin Schwidefsky <[email protected]>

Allow the architecture to request a normal jiffy tick when the system goes idle and tick_nohz_stop_sched_tick is called . On s390 the hook is used to prevent the system going fully idle if there has been an interrupt other than a clock comparator interrupt since the last wakeup.

On s390 the HiperSockets response time for 1 connection ping-pong goes down from 42 to 34 microseconds. The CPU cost decreases by 27%.

Signed-off-by Martin Schwidefsky <[email protected]>

---

 arch/s390/include/asm/cputime.h |    8 ++++++++
 arch/s390/kernel/s390_ext.c     |    2 ++
 arch/s390/kernel/vtime.c        |    2 ++
 drivers/s390/cio/cio.c          |    1 +
 include/linux/tick.h            |    3 +++
 kernel/time/tick-sched.c        |   13 ++++++++-----
 6 files changed, 24 insertions(+), 5 deletions(-)
arch/s390/include/asm/cputime.h +8 -0
Index: quilt-2.6/arch/s390/include/asm/cputime.h
===================================================================
@@ -182,6 +182,7 @@ struct s390_idle_data {
 	unsigned long long idle_count;
 	unsigned long long idle_enter;
 	unsigned long long idle_time;
+	int nohz_delay;
 };
 
 DECLARE_PER_CPU(struct s390_idle_data, s390_idle);

@@ -197,4 +198,11 @@ static inline void s390_idle_check(void)
 		vtime_start_cpu();
 }
 
+static inline int s390_nohz_delay(int cpu)
+{
+	return per_cpu(s390_idle, cpu).nohz_delay != 0;
+}
+
+#define arch_needs_cpu(cpu) s390_nohz_delay(cpu)
+
 #endif /* _S390_CPUTIME_H */
arch/s390/kernel/s390_ext.c +2 -0
Index: quilt-2.6/arch/s390/kernel/s390_ext.c
===================================================================
@@ -126,6 +126,8 @@ void __irq_entry do_extint(struct pt_reg
 		/* Serve timer interrupts first. */
 		clock_comparator_work();
 	kstat_cpu(smp_processor_id()).irqs[EXTERNAL_INTERRUPT]++;
+	if (code != 0x1004)
+		__get_cpu_var(s390_idle).nohz_delay = 1;
         index = ext_hash(code);
 	for (p = ext_int_hash[index]; p; p = p->next) {
 		if (likely(p->code == code))
arch/s390/kernel/vtime.c +2 -0
Index: quilt-2.6/arch/s390/kernel/vtime.c
===================================================================
@@ -167,6 +167,8 @@ void vtime_stop_cpu(void)
 	/* Wait for external, I/O or machine check interrupt. */
 	psw.mask = psw_kernel_bits | PSW_MASK_WAIT | PSW_MASK_IO | PSW_MASK_EXT;
 
+	idle->nohz_delay = 0;
+
 	/* Check if the CPU timer needs to be reprogrammed. */
 	if (vq->do_spt) {
 		__u64 vmax = VTIMER_MAX_SLICE;
drivers/s390/cio/cio.c +1 -0
Index: quilt-2.6/drivers/s390/cio/cio.c
===================================================================
@@ -637,6 +637,7 @@ void __irq_entry do_IRQ(struct pt_regs *
 	old_regs = set_irq_regs(regs);
 	s390_idle_check();
 	irq_enter();
+	__get_cpu_var(s390_idle).nohz_delay = 1;
 	if (S390_lowcore.int_clock >= S390_lowcore.clock_comparator)
 		/* Serve timer interrupts first. */
 		clock_comparator_work();
include/linux/tick.h +3 -0
Index: quilt-2.6/include/linux/tick.h
===================================================================
@@ -98,6 +98,9 @@ extern int tick_check_oneshot_change(int
 extern struct tick_sched *tick_get_tick_sched(int cpu);
 extern void tick_check_idle(int cpu);
 extern int tick_oneshot_mode_active(void);
+#  ifndef arch_needs_cpu
+#   define arch_needs_cpu(cpu) (0)
+#  endif
 # else
 static inline void tick_clock_notify(void) { }
 static inline int tick_check_oneshot_change(int allow_nohz) { return 0; }
kernel/time/tick-sched.c +8 -5
Index: quilt-2.6/kernel/time/tick-sched.c
===================================================================
@@ -272,12 +272,15 @@ void tick_nohz_stop_sched_tick(int inidl
 		last_jiffies = jiffies;
 	} while (read_seqretry(&xtime_lock, seq));
 
-	/* Get the next timer wheel timer */
-	next_jiffies = get_next_timer_interrupt(last_jiffies);
-	delta_jiffies = next_jiffies - last_jiffies;
-
-	if (rcu_needs_cpu(cpu) || printk_needs_cpu(cpu))
+	if (rcu_needs_cpu(cpu) || printk_needs_cpu(cpu) ||
+	    arch_needs_cpu(cpu)) {
+		next_jiffies = last_jiffies + 1;
 		delta_jiffies = 1;
+	} else {
+		/* Get the next timer wheel timer */
+		next_jiffies = get_next_timer_interrupt(last_jiffies);
+		delta_jiffies = next_jiffies - last_jiffies;
+	}
 	/*
 	 * Do not stop the tick, if we are only one off
 	 * or if the cpu is required for rcu
Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.