stop_machine stalls for a considerable period on large cpu count machines.
PATCH v2 1/1 stop_machine stalls for a considerable period on large cpu count machines.
I forgot again on the repost.
Sorry for the noise,
Robin
----- Forwarded message from Robin Holt <[email protected]> -----
Date: Sat, 27 Jun 2009 06:34:10 -0500
From: Robin Holt <[email protected]>
To: Linus Torvalds <[email protected]>
Cc: Mike Travis <[email protected]>, Rusty Russell <[email protected]>,
Stable Kernel Maintainers <[email protected]>
Subject: [Patch v2] stop_machine stalls for a considerable period on large
cpu count machines.
Mike Travis noted that a 2048 cpu machine booting would take hours to get through its modprobes. We would get numerous back traces from stop_cpu indicating they had not serviced interrupts.
A quick code review indicated we have a situation of heavy cacheline contention due to the 'state' (read-mostly) and 'thread_ack' (write-mostly) variables being located in the same cacheline.
---
My first attempt missed a 'quilt refresh' and did not work.
kernel/stop_machine.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
kernel/stop_machine.c +8 -2
@@ -13,6 +13,13 @@#include <asm/atomic.h> #include <asm/uaccess.h> +/* +* It is important to keep 'thread_ack' and 'state' in a seperate +* cachelines to prevent cacheline sharing between threads updating +* thread_ack and other threads spinning on state. +*/ +static atomic_t thread_ack____cacheline_aligned ; +/* This controls the threads on each CPU. */ enum stopmachine_state {/* Dummy starting state for thread. */
@@ -26,7 +33,7 @@ enum stopmachine_state {/* Exit */ STOPMACHINE_EXIT, }; -static enum stopmachine_state state; +static enum stopmachine_state state____cacheline_aligned ;struct stop_machine_data {int (*fn)(void *);
@@ -36,7 +43,6 @@ struct stop_machine_data {/* Like num_online_cpus(), but hotplug cpu uses us, so we need this. */ static unsigned int num_threads; -static atomic_t thread_ack;static DEFINE_MUTEX (lock);/* setup_lock protects refcount, stop_machine_wq and stop_machine_work. */ static DEFINE_MUTEX (setup_lock);
----- End forwarded message -----