x86: Use clflush() instead of wbinvd() whenever possible when changing mapping
PATCH 1/1 x86: Use clflush() instead of wbinvd() whenever possible when changing mapping
The current code uses wbinvd() when the area to flush is > 4MB. Although this may be faster than using clflush() the effect of wbinvd() on irq latencies may be catastrophical on systems with large caches. Therefore use clflush() whenever possible and accept the slight performance hit.
--- arch/x86/mm/pageattr.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-)
arch/x86/mm/pageattr.c +2 -3
@@ -209,13 +209,12 @@ static void cpa_flush_array(unsigned long *start, int numpages, int cache, int in_flags, struct page **pages) {unsigned int i, level; -unsigned long do_wbinvd = cache && numpages >=1024 ;/* 4M threshold */ BUG_ON (irqs_disabled ()); -on_each_cpu (__cpa_flush_all, (void *) do_wbinvd,1 ); +on_each_cpu (__cpa_flush_all, (void *)0UL ,1 ); -if (!cache || do_wbinvd) +if (!cache)return ;/*
Signature · 1 line
1.6.1.3
Quoted 3 lines · Thomas Hellstrom
The current code uses
wbinvd()when the area to flush is > 4MB. Although this may be faster than usingclflush()the effect ofwbinvd()on irq latencies may be catastrophical on systems with large caches. Therefore useclflush()
may be? You seem to miss some hard data here.
I wouldn't expect wbinvd to take longer than a ms or two, which is probably not catastrophic yet.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Quoted 11 lines · Andi Kleen
Quoted 4 lines · Thomas Hellstrom
The current code uses
wbinvd()when the area to flush is > 4MB. Although this may be faster than usingclflush()the effect ofwbinvd()on irq latencies may be catastrophical on systems with large caches. Therefore useclflush()may be? You seem to miss some hard data here.
Admittedly.
However, the concept of flushing and invalidating the caches completely on systems with many processors and huge caches when we intend to only flush only small piece of the cache also sounds like a big overkill.
Furthermore, since the wbinvd() has been introduced as an optimization of the general clflush() case, did somebody ever check the effects on systems with many processors and huge caches?
/Thomas
Quoted 3 lines
-Andi
Quoted 9 lines · Thomas Hellstrom
Quoted 8 lines · Andi Kleen
Quoted 4 lines · Thomas Hellstrom
The current code uses
wbinvd()when the area to flush is > 4MB. Although this may be faster than usingclflush()the effect ofwbinvd()on irq latencies may be catastrophical on systems with large caches. Therefore useclflush()
Quoted 4 lines
Quoted 3 lines
may be? You seem to miss some hard data here.
Admittedly.
So was it motivated by a real problem?
Quoted 4 lines
However, the concept of flushing and invalidating the caches completely on systems with many
processors and huge caches when we intend to only flush only small piece of the cache also sounds like a big overkill.
The other CPUs will not block (just flush their caches in the background or in parallel), so the latency shouldn't scale with the number of sockets. Also number of cores also shouldn't impact it because these tend to have shared cache hierarchies.
That's just a theory, but not necessarily a worse one than yours :-)
Quoted 4 lines
Furthermore, since the
wbinvd()has been introduced as an optimization of the generalclflush()case, did somebody ever check the effects on systems with many processors and huge caches?
Typically systems with large caches flush faster too.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Quoted 13 lines · Andi Kleen
Quoted 11 lines · Thomas Hellstrom
Quoted 9 lines · Andi Kleen
Quoted 5 lines · Thomas Hellstrom
The current code uses
wbinvd()when the area to flush is > 4MB. Although this may be faster than usingclflush()the effect ofwbinvd()on irq latencies may be catastrophical on systems with large caches. Therefore useclflush()
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
may be? You seem to miss some hard data here.
Admittedly.
So was it motivated by a real problem?
No. It was motivated by the assumption that wbinvd() is just bad:
Qoute:
WBINVD is a very nasty operation. I was talking to some CPU people and they really recommended to get rid of it as far as possible. Stopping the CPU for msecs is just wrong and there are apparently even some theoretical live lock situations. - It is not interruptible in earlier VT versions and messes up real time in the hypervisor. Some people were doing KVM on rt kernels and had latency spikes from that.
/Qoute
(I believe you wrote that ?)
Quoted 9 lines
Quoted 3 lines
However, the concept of flushing and invalidating the caches completely on systems with many processors and huge caches when we intend to only flush only small piece of the cache also sounds like a big overkill.
The other CPUs will not block (just flush their caches in the background or in parallel), so the latency shouldn't scale with the number of sockets. Also number of cores also shouldn't impact it because these tend to have shared cache hierarchies.
Quoted 9 lines
That's just a theory, but not necessarily a worse one than yours :-)
Quoted 2 lines
Furthermore, since the
wbinvd()has been introduced as an optimization of the generalclflush()case, did somebody ever check the effects on systems with many processors and huge caches?Typically systems with large caches flush faster too.
OK. We should really test this at some point. I currently don't have the hardware to do so.
Quoted 3 lines
-Andi
/Thomas
Quoted 1 line
No. It was motivated by the assumption that
wbinvd()is just bad:
Ok, got it now.
Quoted 10 lines
Qoute:
WBINVD is a very nasty operation. I was talking to some CPU people and they really recommended to get rid of it as far as possible. Stopping the CPU for msecs is just wrong and there are apparently even some theoretical live lock situations. - It is not interruptible in earlier VT versions and messes up real time in the hypervisor. Some people were doing KVM on rt kernels and had latency spikes from that.
Quoted 2 lines
/Qoute
(I believe you wrote that ?)
Yes. That's still true and that's one reason to not use it.
-Andi
Quoted 3 lines · Thomas Hellstrom
The current code uses
wbinvd()when the area to flush is > 4MB. Although this may be faster than usingclflush()the effect ofwbinvd()on irq latencies
You are slowing the kernel down, and you did not write down the reasons. bad.
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Pavel Machek skrev:
Quoted 11 lines · Thomas Hellstrom
Quoted 5 lines
The current code uses
wbinvd()when the area to flush is > 4MB. Although this may be faster than usingclflush()the effect ofwbinvd()on irq latenciesYou are slowing the kernel down, and you did not write down the reasons. bad.
I may slow the kernel down or speed it up depending on your hardware configuration. The reasons should be quite obvious if you read the discussion following the patch.
/Thomas