Skip to contentSkip to conversation

x86: Is there still value in having a special tlb flush IPI vector?

DISCUSSION Jeremy Fitzhardinge 5 participants 22 messages Jul 28, 2008 – Aug 1, 2008

lore
Share & tools
lore
#0 lore raw

Now that normal smp_function_call is no longer an enormous bottleneck, is there still value in having a specialised IPI vector for tlb flushes? It seems like quite a lot of duplicate code.

The 64-bit tlb flush multiplexes the various cpus across 8 vectors to increase scalability. If this is a big issue, then the smp function call code can (and should) do the same thing. (Though looking at it more closely, the way the code uses the 8 vectors is actually a less general way of doing what smp_call_function is doing anyway.)

Thoughts?

(And uv should definitely be hooking pvops if it wants its own flush_tlb_others; vsmp sets the precedent for a subarch-like use of pvops.)

J

#1 lore raw

Resend to cc: Andi on an address which actually works.

Quoted 14 lines · Jeremy Fitzhardinge

Now that normal smp_function_call is no longer an enormous bottleneck, is there still value in having a specialised IPI vector for tlb flushes? It seems like quite a lot of duplicate code.

The 64-bit tlb flush multiplexes the various cpus across 8 vectors to increase scalability. If this is a big issue, then the smp function call code can (and should) do the same thing. (Though looking at it more closely, the way the code uses the 8 vectors is actually a less general way of doing what smp_call_function is doing anyway.)

Thoughts?

(And uv should definitely be hooking pvops if it wants its own flush_tlb_others; vsmp sets the precedent for a subarch-like use of pvops.)

J

Signature · 4 lines
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#2 lore raw
Quoted 4 lines · Jeremy Fitzhardinge

Resend to cc: Andi on an address which actually works.

Quoted 1 line · Jeremy Fitzhardinge

Now that normal smp_function_call is no longer an enormous bottleneck,

Hmm? It still uses a global lock at least as of current git tree.

-Andi

#3 lore raw
Quoted 4 lines · Andi Kleen
Quoted 2 lines
Quoted 2 lines

Now that normal smp_function_call is no longer an enormous bottleneck,

Hmm? It still uses a global lock at least as of current git tree.

Yes, but it's only held briefly to put things onto the list. It doesn't get held over the whole IPI transaction as the old smp_call_function did, and the tlb flush code still does. RCU is used to manage the list walk and freeing, so there's no long-held locks there either.

J

#4 lore raw
Quoted 10 lines · Jeremy Fitzhardinge
Quoted 4 lines · Andi Kleen
Quoted 2 lines
Quoted 2 lines

Now that normal smp_function_call is no longer an enormous bottleneck,

Hmm? It still uses a global lock at least as of current git tree.

Yes, but it's only held briefly to put things onto the list. It doesn't get held over the whole IPI transaction as the old smp_call_function did, and the tlb flush code still does. RCU is used to manage the list walk and freeing, so there's no long-held locks there either.

If it bounces regularly it will still hurt.

-Andi

#5 lore raw
Quoted 5 lines · Andi Kleen
Quoted 2 lines

Yes, but it's only held briefly to put things onto the list. It doesn't get held over the whole IPI transaction as the old smp_call_function did, and the tlb flush code still does. RCU is used to manage the list walk and freeing, so there's no long-held locks there either.

If it bounces regularly it will still hurt.

We could convert smp_call_function_mask to use a multi-vector scheme like tlb_64.c if that turns out to be an issue.

J

#6 lore raw
Quoted 7 lines · Jeremy Fitzhardinge
Quoted 6 lines · Andi Kleen
Quoted 5 lines

Yes, but it's only held briefly to put things onto the list. It doesn't get held over the whole IPI transaction as the old smp_call_function did, and the tlb flush code still does. RCU is used to manage the list walk and freeing, so there's no long-held locks there either.

Quoted 5 lines
Quoted 2 lines

If it bounces regularly it will still hurt.

We could convert smp_call_function_mask to use a multi-vector scheme like tlb_64.c if that turns out to be an issue.

Converting it first would be fine. Or rather in parallel because you would need to reuse the TLB vectors (there are not that many free)

But waiting first for a report would seem wrong to me.

I can just see some poor performance person spend a lot of work to track down such a regression. While there's a lot of development manpower available for Linux there's still no reason to waste i. I think if you want to change such performance critical paths you should make sure the new code is roughly performance equivalent first. And with the global lock I don't see that at all.

-Andi

#7 lore raw
In reply to Jeremy Fitzhardinge
Quoted 9 lines · Jeremy Fitzhardinge

Now that normal smp_function_call is no longer an enormous bottleneck, is there still value in having a specialised IPI vector for tlb flushes? It seems like quite a lot of duplicate code.

The 64-bit tlb flush multiplexes the various cpus across 8 vectors to increase scalability. If this is a big issue, then the smp function call code can (and should) do the same thing. (Though looking at it more closely, the way the code uses the 8 vectors is actually a less general way of doing what smp_call_function is doing anyway.)

yep, and we could eliminate the reschedule IPI as well.

Ingo

#8 lore raw
Quoted 10 lines · Ingo Molnar
Quoted 9 lines · Jeremy Fitzhardinge

Now that normal smp_function_call is no longer an enormous bottleneck, is there still value in having a specialised IPI vector for tlb flushes? It seems like quite a lot of duplicate code.

The 64-bit tlb flush multiplexes the various cpus across 8 vectors to increase scalability. If this is a big issue, then the smp function call code can (and should) do the same thing. (Though looking at it more closely, the way the code uses the 8 vectors is actually a less general way of doing what smp_call_function is doing anyway.)

It definitely is not a clear win. They do not have the same characteristics. So numbers will be needed.

smp_call_function is now properly scalable in smp_call_function_single form. The more general case of multiple targets is not so easy and it still takes a global lock and touches global cachelines.

I don't think it is a good use of time, honestly. Do you have a good reason?

Quoted 1 line

yep, and we could eliminate the reschedule IPI as well.

No. The rewrite makes it now very good at synchronously sending a function to a single other CPU.

Sending asynchronously requires a slab allocation and then a remote slab free (which is nasty for slab) at the other end, and bouncing of locks and cachelines. No way you want to do that in the reschedule IPI.

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

#9 lore raw
Quoted 9 lines · Nick Piggin

It definitely is not a clear win. They do not have the same characteristics. So numbers will be needed.

smp_call_function is now properly scalable in smp_call_function_single form. The more general case of multiple targets is not so easy and it still takes a global lock and touches global cachelines.

I don't think it is a good use of time, honestly. Do you have a good reason?

Code cleanup, unification. It took about 20 minutes to do. It probably won't take too much longer to unify kernel/tlb.c. It seems that if there's any performance loss in making the transition, then we can make it up again by tuning smp_call_function_mask, benefiting all users.

But, truth be told, the real reason is that I think there may be some correctness issue around smp_call_function* - I've seen occasional inexplicable crashes, all within generic_smp_call_function() - and I just can't exercise that code enough to get a solid reproducing case. But if it gets used for tlb flushes, then any bug is going to become pretty obvious. Regardless of whether these patches get accepted, I can use it as a test vehicle.

Quoted 10 lines

No. The rewrite makes it now very good at synchronously sending a function to a single other CPU.

Sending asynchronously requires a slab allocation and then a remote slab free (which is nasty for slab) at the other end, and bouncing of locks and cachelines. No way you want to do that in the reschedule IPI.

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

In the async case? Or because it can become spontaneously sync if there's an allocation failure?

J

#10 lore raw
Quoted 4 lines · Jeremy Fitzhardinge
Quoted 2 lines · Nick Piggin

It definitely is not a clear win. They do not have the same characteristics. So numbers will be needed.

Quoted 8 lines
Quoted 6 lines

smp_call_function is now properly scalable in smp_call_function_single form. The more general case of multiple targets is not so easy and it still takes a global lock and touches global cachelines.

I don't think it is a good use of time, honestly. Do you have a good reason?

Code cleanup, unification. It took about 20 minutes to do. It probably

OK, so nothing terribly important.

Quoted 3 lines

won't take too much longer to unify kernel/tlb.c. It seems that if there's any performance loss in making the transition, then we can make it up again by tuning smp_call_function_mask, benefiting all users.

No I don't think that is the right way to go for such an important functionality. There are no ifs, smp_call_function does touch global cachelines and locks.

smp_call_function is barely used, as should be very obvious because it was allowed to languish with such horrible performance for so long. So there aren't too many users.

But if you get smp_call_function_mask performance at the same time, then there is less to argue about I guess (although it will always be necessarily more complex than plain tlb flushing).

Quoted 7 lines

But, truth be told, the real reason is that I think there may be some correctness issue around smp_call_function* - I've seen occasional inexplicable crashes, all within generic_smp_call_function() - and I just can't exercise that code enough to get a solid reproducing case. But if it gets used for tlb flushes, then any bug is going to become pretty obvious. Regardless of whether these patches get accepted, I can use it as a test vehicle.

That's fair enough. Better still might be a test harness specifically to exercise it.

Quoted 7 lines
Quoted 7 lines

No. The rewrite makes it now very good at synchronously sending a function to a single other CPU.

Sending asynchronously requires a slab allocation and then a remote slab free (which is nasty for slab) at the other end, and bouncing of locks and cachelines. No way you want to do that in the reschedule IPI.

Quoted 5 lines
Quoted 2 lines

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

In the async case? Or because it can become spontaneously sync if there's an allocation failure?

In both sync and async case, yes.

#11 lore raw
In reply to Nick Piggin
Quoted 2 lines · Nick Piggin

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

__smp_call_function_single has potential though..

#12 lore raw
Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · Nick Piggin

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

__smp_call_function_single has potential though..

For reschedule interrupt? I don't really agree.

#13 lore raw
Quoted 8 lines · Nick Piggin
Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · Nick Piggin

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

__smp_call_function_single has potential though..

For reschedule interrupt? I don't really agree.

Not specifically, for not deadlocking from irq-off, more so.

#14 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 8 lines · Nick Piggin
Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · Nick Piggin

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

__smp_call_function_single has potential though..

For reschedule interrupt? I don't really agree.

Not specifically, for not deadlocking from irq-off, more so.

Oh, well yes it already does work from irq-off, so it has already realised its potential :)

Not sure exactly what kinds of users it is going to attract, but it should be interesting to see!

#15 lore raw
Quoted 11 lines · Nick Piggin
Quoted 10 lines · Peter Zijlstra
Quoted 8 lines · Nick Piggin
Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · Nick Piggin

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

__smp_call_function_single has potential though..

For reschedule interrupt? I don't really agree.

Quoted 7 lines
Quoted 1 line

Not specifically, for not deadlocking from irq-off, more so.

Oh, well yes it already does work from irq-off, so it has already realised its potential :)

Not sure exactly what kinds of users it is going to attract, but it should be interesting to see!

grep __smp_call_function_single kernel/sched.c

#16 lore raw
In reply to Nick Piggin
Quoted 8 lines · Nick Piggin
Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · Nick Piggin

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

__smp_call_function_single has potential though..

For reschedule interrupt? I don't really agree.

How about using just arch_send_call_function_single_ipi() to implement smp_send_reschedule() ?

The overhead of that is a smp_mb() and a list_empty() check in generic_smp_call_function_single_interrupt() if there is indeed no work to do.

#17 lore raw
Quoted 6 lines · Peter Zijlstra
Quoted 5 lines · Nick Piggin
Quoted 4 lines · Peter Zijlstra
Quoted 2 lines · Nick Piggin

Not to mention the minor problem that it still deadlocks when called with interrupts disabled ;)

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

__smp_call_function_single has potential though..

For reschedule interrupt? I don't really agree.

How about using just arch_send_call_function_single_ipi() to implement smp_send_reschedule() ?

agreed, that's just a single IPI which kicks the need_resched logic on return-from-interrupt.

Quoted 3 lines

The overhead of that is a smp_mb() and a list_empty() check in generic_smp_call_function_single_interrupt() if there is indeed no work to do.

that would be a miniscule cost - cacheline is read-shared amongst cpus so there's no real bouncing there. So i'm all for it ...

Ingo

#18 lore raw
Quoted 8 lines · Ingo Molnar
Quoted 3 lines · Peter Zijlstra

The overhead of that is a smp_mb() and a list_empty() check in generic_smp_call_function_single_interrupt() if there is indeed no work to do.

that would be a miniscule cost - cacheline is read-shared amongst cpus so there's no real bouncing there. So i'm all for it ...

smp_mb would cost some cycles. So would the branch mispredict because list_empty would otherwise normally be taken I think. q likely is not in cache either.

I'm not in favour.

#19 lore raw
In reply to Peter Zijlstra
Quoted 7 lines · Peter Zijlstra

How about using just arch_send_call_function_single_ipi() to implement smp_send_reschedule() ?

The overhead of that is a smp_mb() and a list_empty() check in generic_smp_call_function_single_interrupt() if there is indeed no work to do.

Is doing a no-op interrupt sufficient on all architectures? Is there some change a function call IPI might not go through the normal reschedule interrupt exit path?

J

#20 lore raw
Quoted 11 lines · Jeremy Fitzhardinge
Quoted 6 lines · Peter Zijlstra

How about using just arch_send_call_function_single_ipi() to implement smp_send_reschedule() ?

The overhead of that is a smp_mb() and a list_empty() check in generic_smp_call_function_single_interrupt() if there is indeed no work to do.

Is doing a no-op interrupt sufficient on all architectures? Is there some change a function call IPI might not go through the normal reschedule interrupt exit path?

We'd still use the smp_send_reschdule(cpu) API, so it's an architecture detail. On x86 we'd use arch_send_call_function_single_ipi().

Ingo

#21 lore raw
Quoted 10 lines · Ingo Molnar
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 6 lines · Peter Zijlstra

How about using just arch_send_call_function_single_ipi() to implement smp_send_reschedule() ?

The overhead of that is a smp_mb() and a list_empty() check in generic_smp_call_function_single_interrupt() if there is indeed no work to do.

Quoted 6 lines
Quoted 3 lines

Is doing a no-op interrupt sufficient on all architectures? Is there some change a function call IPI might not go through the normal reschedule interrupt exit path?

We'd still use the smp_send_reschdule(cpu) API, so it's an architecture detail. On x86 we'd use arch_send_call_function_single_ipi().

Also, all interrupts _should_ do the regular interrupt enter/exit paths, we fixup stuff there, like jiffies and such.

We had a fun NO_HZ bug the other day because some sparc64 IPIs didn't.