x86/mm/cpa: Unconditionally avoid WBINDV when we can
PATCH 1/1 x86/mm/cpa: Unconditionally avoid WBINDV when we can
From: Peter Zijlstra <[email protected]>
commit ddd07b750382adc2b78fdfbec47af8a6e0d8ef37 upstream.
CAT has happened, WBINDV is bad (even before CAT blowing away the entire cache on a multi-core platform wasn't nice), try not to use it ever.
--- arch/x86/mm/pageattr.c | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-)
arch/x86/mm/pageattr.c +2 -16
@@ -239,26 +239,12 @@ static void cpa_flush_array(unsigned long *start, int numpages, int cache, int in_flags, struct page **pages) {unsigned int i, level; -#ifdef CONFIG_PREEMPT -/* -* Avoid wbinvd() because it causes latencies on all CPUs, -* regardless of any CPU isolation that may be in effect. -* -* This should be extended for CAT enabled systems independent of -* PREEMPT because wbinvd() does not respect the CAT partitions and -* this is exposed to unpriviledged users through the graphics -* subsystem. -*/ -unsigned long do_wbinvd =0 ; -#else -unsigned long do_wbinvd = cache && numpages >=1024 ;/* 4M threshold */ -#endif BUG_ON (irqs_disabled () && !early_boot_irqs_disabled); -on_each_cpu (__cpa_flush_all, (void *) do_wbinvd,1 ); +flush_tlb_all (); -if (!cache || do_wbinvd) +if (!cache)return ;/*
Signature · 1 line
2.19.1.6.gb485710b
Greg Kroah-Hartman
maintainerQuoted 56 lines · Wen Yang
From: Peter Zijlstra <[email protected]>
commit
ddd07b750382adc2b78fdfbec47af8a6e0d8ef37upstream.CAT has happened, WBINDV is bad (even before CAT blowing away the entire cache on a multi-core platform wasn't nice), try not to use it ever.
Signed-off-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Reviewed-by Dave Hansen <[email protected]>Cc Bin Yang <[email protected]>Cc Mark Gross <[email protected]>Cc <[email protected]> # 4.19.xSigned-off-by Wen Yang <[email protected]>--- arch/x86/mm/pageattr.c | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-)arch/x86/mm/pageattr.c +2
-16@@ -239,26 +239,12 @@ static void cpa_flush_array(unsigned long *start, int numpages, int cache, int in_flags, struct page **pages) {unsigned int i, level; -#ifdef CONFIG_PREEMPT -/* -* Avoid wbinvd() because it causes latencies on all CPUs, -* regardless of any CPU isolation that may be in effect. -* -* This should be extended for CAT enabled systems independent of -* PREEMPT because wbinvd() does not respect the CAT partitions and -* this is exposed to unpriviledged users through the graphics -* subsystem. -*/ -unsigned long do_wbinvd =0 ; -#else -unsigned long do_wbinvd = cache && numpages >=1024 ;/* 4M threshold */ -#endif BUG_ON (irqs_disabled () && !early_boot_irqs_disabled); -on_each_cpu (__cpa_flush_all, (void *) do_wbinvd,1 ); +flush_tlb_all (); -if (!cache || do_wbinvd) +if (!cache)return ;/* Signature · 1 line
2.19.1.6.gb485710b
Why is this needed on 4.19.y? What problem does it solve, it looks only like an optimization, not a bugfix.
And if it's a bugfix, why only 4.19.y, why not older kernels too?
We need more information here please.
thanks,
greg k-h
Quoted 61 lines · Greg Kroah-Hartman
Quoted 56 lines · Wen Yang
From: Peter Zijlstra <[email protected]>
commit
ddd07b750382adc2b78fdfbec47af8a6e0d8ef37upstream.CAT has happened, WBINDV is bad (even before CAT blowing away the entire cache on a multi-core platform wasn't nice), try not to use it ever.
Signed-off-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Reviewed-by Dave Hansen <[email protected]>Cc Bin Yang <[email protected]>Cc Mark Gross <[email protected]>Cc <[email protected]> # 4.19.xSigned-off-by Wen Yang <[email protected]>--- arch/x86/mm/pageattr.c | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-)arch/x86/mm/pageattr.c +2
-16@@ -239,26 +239,12 @@ static void cpa_flush_array(unsigned long *start, int numpages, int cache, int in_flags, struct page **pages) {unsigned int i, level; -#ifdef CONFIG_PREEMPT -/* -* Avoid wbinvd() because it causes latencies on all CPUs, -* regardless of any CPU isolation that may be in effect. -* -* This should be extended for CAT enabled systems independent of -* PREEMPT because wbinvd() does not respect the CAT partitions and -* this is exposed to unpriviledged users through the graphics -* subsystem. -*/ -unsigned long do_wbinvd =0 ; -#else -unsigned long do_wbinvd = cache && numpages >=1024 ;/* 4M threshold */ -#endif BUG_ON (irqs_disabled () && !early_boot_irqs_disabled); -on_each_cpu (__cpa_flush_all, (void *) do_wbinvd,1 ); +flush_tlb_all (); -if (!cache || do_wbinvd) +if (!cache)return ;/* Signature · 1 line
2.19.1.6.gb485710bWhy is this needed on 4.19.y? What problem does it solve, it looks only like an optimization, not a bugfix.
Quoted 4 lines
And if it's a bugfix, why only 4.19.y, why not older kernels too?
We need more information here please.
On a 128-core Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz server, when the user program frequently calls nv_alloc_system_pages to allocate large memory, it often causes a delay of about 200 milliseconds for the entire system. In this way, other latency-sensitive tasks on this system are heavily impacted, causing stability issues in large-scale clusters as well.
nv_alloc_system_pages
-> _set_memory_array
-> change_page_attr_set_clr
-> cpa_flush_array
-> on_each_cpu(__cpa_flush_all, (void *) do_wbinvd, 1);
This patch can be directly merged into the 4.19 kernel to solve this problem, and most of the machines in our production environment are 4.19 kernels.
We're also happy to apply it to the 4.14 and 4.9 kernels, and send the corresponding patches soon, although there are very few such servers in our production clusters.
--
Best wishes,
Wen
On 7/4/22 20:45, Wen Yang wrote:> On a 128-core Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz server,
Quoted 2 lines
when the user program frequently calls
nv_alloc_system_pagesto allocate large memory,
We seem to be repeating the same conversation we had back in November of last year:
Quoted 1 line
It's the binary nvidia driver doing unusual stuff again. Please talk to the folks you got that driver from.
Greg Kroah-Hartman
maintainerQuoted 78 lines · Wen Yang
Quoted 57 lines · Greg Kroah-Hartman
Quoted 52 lines · Wen Yang
From: Peter Zijlstra <[email protected]>
commit
ddd07b750382adc2b78fdfbec47af8a6e0d8ef37upstream.CAT has happened, WBINDV is bad (even before CAT blowing away the entire cache on a multi-core platform wasn't nice), try not to use it ever.
Signed-off-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Reviewed-by Dave Hansen <[email protected]>Cc Bin Yang <[email protected]>Cc Mark Gross <[email protected]>Cc <[email protected]> # 4.19.xSigned-off-by Wen Yang <[email protected]>--- arch/x86/mm/pageattr.c | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-)arch/x86/mm/pageattr.c +2
-16@@ -239,26 +239,12 @@ static void cpa_flush_array(unsigned long *start, int numpages, int cache, int in_flags, struct page **pages) {unsigned int i, level; -#ifdef CONFIG_PREEMPT -/* -* Avoid wbinvd() because it causes latencies on all CPUs, -* regardless of any CPU isolation that may be in effect. -* -* This should be extended for CAT enabled systems independent of -* PREEMPT because wbinvd() does not respect the CAT partitions and -* this is exposed to unpriviledged users through the graphics -* subsystem. -*/ -unsigned long do_wbinvd =0 ; -#else -unsigned long do_wbinvd = cache && numpages >=1024 ;/* 4M threshold */ -#endif BUG_ON (irqs_disabled () && !early_boot_irqs_disabled); -on_each_cpu (__cpa_flush_all, (void *) do_wbinvd,1 ); +flush_tlb_all (); -if (!cache || do_wbinvd) +if (!cache)return ;/* Signature · 1 line
2.19.1.6.gb485710bWhy is this needed on 4.19.y? What problem does it solve, it looks only like an optimization, not a bugfix.
Quoted 4 lines
And if it's a bugfix, why only 4.19.y, why not older kernels too?
We need more information here please.
On a 128-core Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz server, when the user program frequently calls
nv_alloc_system_pagesto allocate large memory, it often causes a delay of about 200 milliseconds for the entire system. In this way, other latency-sensitive tasks on this system are heavily impacted, causing stability issues in large-scale clusters as well.
nv_alloc_system_pages
->_set_memory_array
->change_page_attr_set_clr
->cpa_flush_array
->on_each_cpu(__cpa_flush_all, (void *) do_wbinvd, 1);
Quoted 3 lines
This patch can be directly merged into the 4.19 kernel to solve this problem, and most of the machines in our production environment are 4.19 kernels.
Ah. So what has changed from last year when I rejected this then:
https://lore.kernel.org/all/[email protected]/T/#m06369d080fa97eda3dd6a8eaf54a8ca2d430b3ab
Please do not try to submit previously-rejected patches, that is very disingenuous.
greg k-h