Skip to contentSkip to conversation

x86 smp_call_function recent breakage

PATCH Andrea Arcangeli 2 participants 3 messages Oct 23, 2001

lore
Share & tools
lore
#0 lore raw

This isn't the right fix:

arch/i386/kernel/smp.c +24 -21
diff -urN 2.4.12ac5/arch/i386/kernel/smp.c 2.4.12ac6/arch/i386/kernel/smp.c
@@ -549,6 +549,7 @@
 	 * code desn't get confused if it gets an unexpected repeat
 	 * trigger of an old IPI while we're still setting up the new
 	 * one. */
+	wmb();
 	atomic_set(&data->started, 0);
 
 	local_bh_disable();
@@ -563,22 +564,20 @@
 		cpu_relax();
 	}
 
-	/* It is now safe to reuse the "call_data" global, but we need
-	 * to keep local bottom-halves disabled until after waiters have
-	 * been acknowledged to prevent reuse of the per-cpu call data
-	 * entry. */
+	/* We _must_ wait in all cases here, because we cannot drop the
+	 * call lock (and hence allow the call_data pointer to be
+	 * reused) until all CPUs have read the current value. */
+	while (atomic_read(&data->finished) != cpus)
+		barrier();
+
 	spin_unlock(&call_lock);
 
-	if (wait)
-	{
-		while (atomic_read(&data->finished) != cpus)
-		{
-			barrier();
-			cpu_relax();
-		}
-	}
-	local_bh_enable();
+	/* If any of the smp target functions are sufficiently expensive
+	 * that non-waiting IPI is important, we need to add a third
+	 * level to the handshake here to wait for completion of the
+	 * function. */
 
+	local_bh_enable();
 	return 0;
 }
 
@@ -621,9 +620,9 @@
 
 asmlinkage void smp_call_function_interrupt(void)
 {
-	void (*func) (void *info) = call_data->func;
-	void *info = call_data->info;
-	int wait = call_data->wait;
+	void (*func) (void *info);
+	void *info;
+	int wait;
 
 	ack_APIC_irq();
 	/*
@@ -633,11 +632,15 @@
 	 */
 	if (test_and_set_bit(smp_processor_id(), &call_data->started))
 		return;
-	/*
-	 * At this point the info structure may be out of scope unless wait==1
-	 */
+
+	/* We now know that the call_data is valid: */
+	func = call_data->func;
+	info = call_data->info;
+	wait = call_data->wait;
+
 	(*func)(info);
-	if (wait)
-		set_bit(smp_processor_id(), &call_data->finished);
+	/* Once we set this, all of the call_data information may go out
+	 * of scope. */
+	set_bit(smp_processor_id(), &call_data->finished);
 }
 

Right fix is to backout the broken changes that gone in 2.4.10pre12 because it's totally pointless to have a per-cpu array cacheline aligned, at least if you don't pass also a paramtere to the IPI routine so that it knows what entry to peek up so that we can scale the smp_call_function better and allow other to be execute while we wait for completion, but there's no parameter so it's just wasted memory and such a scalability optimization would be a very very minor one since if you want a chance to scale no IPI should run in first place.

So I'm backing out the below one instead of applying the above one:

arch/i386/kernel/smp.c +35 -23

95 lines not on this page · raw message

Robert, this explains the missed IPI during drain_cpu_caches, it isn't ram fault or IPI missed by the hardware, so I suggest to just backout the second diff above and try again. Will be fixed also in next -aa of course.

Andrea

#1 lore raw
Quoted 1 line

This isn't the right fix:

The cache thing you may be right on. The core fix is not about caching its about fixing races on IPI replay. We have to handle an IPI reoccuring potentially with a small time delay due to a retransmit of a message lost by one party on the bus. Furthermore it seems other messages can pass the message that then gets retransmitted.

Quoted 4 lines

Robert, this explains the missed IPI during drain_cpu_caches, it isn't ram fault or IPI missed by the hardware, so I suggest to just backout the second diff above and try again. Will be fixed also in next -aa of course.

I'm not sure I follow why it explains a missed IPI. Please explain in more detail.

Alan

#2 lore raw
In reply to Andrea Arcangeli
Quoted 8 lines · Alan Cox
Quoted 1 line

This isn't the right fix:

The cache thing you may be right on. The core fix is not about caching its about fixing races on IPI replay. We have to handle an IPI reoccuring potentially with a small time delay due to a retransmit of a message lost by one party on the bus. Furthermore it seems other messages can pass the message that then gets retransmitted.

Quoted 7 lines
Quoted 4 lines

Robert, this explains the missed IPI during drain_cpu_caches, it isn't ram fault or IPI missed by the hardware, so I suggest to just backout the second diff above and try again. Will be fixed also in next -aa of course.

I'm not sure I follow why it explains a missed IPI. Please explain in more detail.

The bug is very simple and it have nothing to do with hardware details, it is a plain software bug:

- if (wait) {
- mb();
- atomic_inc(&call_data->finished);
- }
+ if (wait)
+ set_bit(smp_processor_id(), &call_data->finished);

since 2.4.10pre12 (and in all previous -ac also) you still use the above "call_data" for notifying "finished", but you also allowed "call_data" to be changed under the IPI handler by another incoming IPI (the scalability optimization), so the first IPI handler will notify the completion of the second IPI (instead of the first one) generating the "missed IPI" generating kernel instability (deadlock for Roboert in drain_cpu_caches that waits all cpus to "finish").

So you wonder about reply IPI or whatever, and you add further hackery plus you left the NR_CPUS array that now is useless since there's nothing to scale and we hold the spinlock for the whole duration of the IPI cycle, while the only fix in your ac5<->ac6 diff is that you now spin_unlock after touching ->finished like the old code correctly did, so you notify completion of the right IPI and not of the wrong one, so it won't deadlock anymore.

I don't think it's necessary to read ->func and friends before checking ->started, also there's no need of testing ->started. we have the guarantee that only 1 IPI cycle can run at once because of the spinlock held for the whole duration of the IPI cycle, so if we are running in smp_call_function_interrupt we know all ->func,->started == 0 fields are just coherent and freely usable in memory, we only need to care to mb() between reading them and increasing ->started like the old code correctly did. So I think the old code was just perfect.

Andrea