Skip to contentSkip to conversation

remove put_cpu_no_resched() [ Resend ]

PATCH In review Thomas Gleixner 2 participants 2 messages May 2, 2009 – May 3, 2009 1 message not in the archive NFS client

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

PATCH 1/1 remove put_cpu_no_resched() [ Resend ]

Impact: remove a high latency source

put_cpu_no_resched() is an optimization of put_cpu() which unfortunately can cause high latencies.

The nfs iostats code uses put_cpu_no_resched() in a code sequence where a reschedule request caused by an interrupt between the get_cpu() and the put_cpu_no_resched() can delay the reschedule for at least HZ.

The other users of put_cpu_no_resched() optimize correctly in interrupt code, but there is no real harm in using the put_cpu() function which is an alias for preempt_enable(). The extra check of the preemmpt count is not as critical as the potential source of missing a reschedule.

Debugged in the preempt-rt tree and verified in mainline.

Cc Ingo Molnar <[email protected]>
Cc Tony Luck <[email protected]>
Cc Trond Myklebust <[email protected]>
---
 arch/ia64/kernel/perfmon.c |    2 +-
 fs/nfs/iostat.h            |    4 ++--
 include/linux/smp.h        |    1 -
 3 files changed, 3 insertions(+), 4 deletions(-)
arch/ia64/kernel/perfmon.c +1 -1
Index: linux-2.6/arch/ia64/kernel/perfmon.c
===================================================================
@@ -5595,7 +5595,7 @@ pfm_interrupt_handler(int irq, void *arg
 		(*pfm_alt_intr_handler->handler)(irq, arg, regs);
 	}
 
-	put_cpu_no_resched();
+	put_cpu();
 	return IRQ_HANDLED;
 }
 
fs/nfs/iostat.h +2 -2
Index: linux-2.6/fs/nfs/iostat.h
===================================================================
@@ -31,7 +31,7 @@ static inline void nfs_inc_server_stats(
 	cpu = get_cpu();
 	iostats = per_cpu_ptr(server->io_stats, cpu);
 	iostats->events[stat]++;
-	put_cpu_no_resched();
+	put_cpu();
 }
 
 static inline void nfs_inc_stats(const struct inode *inode,
@@ -50,7 +50,7 @@ static inline void nfs_add_server_stats(
 	cpu = get_cpu();
 	iostats = per_cpu_ptr(server->io_stats, cpu);
 	iostats->bytes[stat] += addend;
-	put_cpu_no_resched();
+	put_cpu();
 }
 
 static inline void nfs_add_stats(const struct inode *inode,
include/linux/smp.h +0 -1
Index: linux-2.6/include/linux/smp.h
===================================================================
@@ -177,7 +177,6 @@ static inline void init_call_single_data
 
 #define get_cpu()		({ preempt_disable(); smp_processor_id(); })
 #define put_cpu()		preempt_enable()
-#define put_cpu_no_resched()	preempt_enable_no_resched()
 
 /*
  * Callback to arch code if there's nosmp or maxcpus=0 on the
Acked-by on v1 1/1
#2 lore raw
Quoted 22 lines · Thomas Gleixner

Impact: remove a high latency source

put_cpu_no_resched() is an optimization of put_cpu() which unfortunately can cause high latencies.

The nfs iostats code uses put_cpu_no_resched() in a code sequence where a reschedule request caused by an interrupt between the get_cpu() and the put_cpu_no_resched() can delay the reschedule for at least HZ.

The other users of put_cpu_no_resched() optimize correctly in interrupt code, but there is no real harm in using the put_cpu() function which is an alias for preempt_enable(). The extra check of the preemmpt count is not as critical as the potential source of missing a reschedule.

Debugged in the preempt-rt tree and verified in mainline.

Cc Ingo Molnar <[email protected]>
Cc Tony Luck <[email protected]>
Cc Trond Myklebust <[email protected]>
Quoted 5 lines
---
 arch/ia64/kernel/perfmon.c |    2 +-
 fs/nfs/iostat.h            |    4 ++--
 include/linux/smp.h        |    1 -
 3 files changed, 3 insertions(+), 4 deletions(-)

Good catch!

Ingo