Skip to contentSkip to conversation

x86/mm/cpa: Unconditionally avoid WBINDV when we can

PATCH In review Wen Yang 3 participants 5 messages Jul 4, 2022 – Jul 5, 2022 Stable branch

lore
Share & tools
lore

Wen Yang

author
#0 lore raw

PATCH 1/1 x86/mm/cpa: Unconditionally avoid WBINDV when we can

From: Peter Zijlstra <[email protected]>

commit ddd07b750382adc2b78fdfbec47af8a6e0d8ef37 upstream.

CAT has happened, WBINDV is bad (even before CAT blowing away the entire cache on a multi-core platform wasn't nice), try not to use it ever.

---
 arch/x86/mm/pageattr.c | 18 ++----------------
 1 file changed, 2 insertions(+), 16 deletions(-)
arch/x86/mm/pageattr.c +2 -16
index 101f3ad0d6ad..ab87da7a6043 100644
@@ -239,26 +239,12 @@ static void cpa_flush_array(unsigned long *start, int numpages, int cache,
 			    int in_flags, struct page **pages)
 {
 	unsigned int i, level;
-#ifdef CONFIG_PREEMPT
-	/*
-	 * Avoid wbinvd() because it causes latencies on all CPUs,
-	 * regardless of any CPU isolation that may be in effect.
-	 *
-	 * This should be extended for CAT enabled systems independent of
-	 * PREEMPT because wbinvd() does not respect the CAT partitions and
-	 * this is exposed to unpriviledged users through the graphics
-	 * subsystem.
-	 */
-	unsigned long do_wbinvd = 0;
-#else
-	unsigned long do_wbinvd = cache && numpages >= 1024; /* 4M threshold */
-#endif
 
 	BUG_ON(irqs_disabled() && !early_boot_irqs_disabled);
 
-	on_each_cpu(__cpa_flush_all, (void *) do_wbinvd, 1);
+	flush_tlb_all();
 
-	if (!cache || do_wbinvd)
+	if (!cache)
 		return;
 
 	/*
Signature · 1 line
2.19.1.6.gb485710b
#1 lore raw
Quoted 56 lines · Wen Yang

From: Peter Zijlstra <[email protected]>

commit ddd07b750382adc2b78fdfbec47af8a6e0d8ef37 upstream.

CAT has happened, WBINDV is bad (even before CAT blowing away the entire cache on a multi-core platform wasn't nice), try not to use it ever.

---
 arch/x86/mm/pageattr.c | 18 ++----------------
 1 file changed, 2 insertions(+), 16 deletions(-)
arch/x86/mm/pageattr.c +2 -16
index 101f3ad0d6ad..ab87da7a6043 100644
@@ -239,26 +239,12 @@ static void cpa_flush_array(unsigned long *start, int numpages, int cache,
 			    int in_flags, struct page **pages)
 {
 	unsigned int i, level;
-#ifdef CONFIG_PREEMPT
-	/*
-	 * Avoid wbinvd() because it causes latencies on all CPUs,
-	 * regardless of any CPU isolation that may be in effect.
-	 *
-	 * This should be extended for CAT enabled systems independent of
-	 * PREEMPT because wbinvd() does not respect the CAT partitions and
-	 * this is exposed to unpriviledged users through the graphics
-	 * subsystem.
-	 */
-	unsigned long do_wbinvd = 0;
-#else
-	unsigned long do_wbinvd = cache && numpages >= 1024; /* 4M threshold */
-#endif
 
 	BUG_ON(irqs_disabled() && !early_boot_irqs_disabled);
 
-	on_each_cpu(__cpa_flush_all, (void *) do_wbinvd, 1);
+	flush_tlb_all();
 
-	if (!cache || do_wbinvd)
+	if (!cache)
 		return;
 
 	/*
Signature · 1 line
2.19.1.6.gb485710b

Why is this needed on 4.19.y? What problem does it solve, it looks only like an optimization, not a bugfix.

And if it's a bugfix, why only 4.19.y, why not older kernels too?

We need more information here please.

thanks,

greg k-h

author
#2 lore raw
Quoted 61 lines · Greg Kroah-Hartman
Quoted 56 lines · Wen Yang

From: Peter Zijlstra <[email protected]>

commit ddd07b750382adc2b78fdfbec47af8a6e0d8ef37 upstream.

CAT has happened, WBINDV is bad (even before CAT blowing away the entire cache on a multi-core platform wasn't nice), try not to use it ever.

---
  arch/x86/mm/pageattr.c | 18 ++----------------
  1 file changed, 2 insertions(+), 16 deletions(-)
arch/x86/mm/pageattr.c +2 -16
index 101f3ad0d6ad..ab87da7a6043 100644
@@ -239,26 +239,12 @@ static void cpa_flush_array(unsigned long *start, int numpages, int cache,
  			    int in_flags, struct page **pages)
  {
  	unsigned int i, level;
-#ifdef CONFIG_PREEMPT
-	/*
-	 * Avoid wbinvd() because it causes latencies on all CPUs,
-	 * regardless of any CPU isolation that may be in effect.
-	 *
-	 * This should be extended for CAT enabled systems independent of
-	 * PREEMPT because wbinvd() does not respect the CAT partitions and
-	 * this is exposed to unpriviledged users through the graphics
-	 * subsystem.
-	 */
-	unsigned long do_wbinvd = 0;
-#else
-	unsigned long do_wbinvd = cache && numpages >= 1024; /* 4M threshold */
-#endif
 
  	BUG_ON(irqs_disabled() && !early_boot_irqs_disabled);
 
-	on_each_cpu(__cpa_flush_all, (void *) do_wbinvd, 1);
+	flush_tlb_all();
 
-	if (!cache || do_wbinvd)
+	if (!cache)
  		return;
 
  	/*
Signature · 1 line
2.19.1.6.gb485710b

Why is this needed on 4.19.y? What problem does it solve, it looks only like an optimization, not a bugfix.

Quoted 4 lines

And if it's a bugfix, why only 4.19.y, why not older kernels too?

We need more information here please.

On a 128-core Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz server, when the user program frequently calls nv_alloc_system_pages to allocate large memory, it often causes a delay of about 200 milliseconds for the entire system. In this way, other latency-sensitive tasks on this system are heavily impacted, causing stability issues in large-scale clusters as well.

nv_alloc_system_pages
-> _set_memory_array
-> change_page_attr_set_clr
-> cpa_flush_array
-> on_each_cpu(__cpa_flush_all, (void *) do_wbinvd, 1);

This patch can be directly merged into the 4.19 kernel to solve this problem, and most of the machines in our production environment are 4.19 kernels.

We're also happy to apply it to the 4.14 and 4.9 kernels, and send the corresponding patches soon, although there are very few such servers in our production clusters.

--
Best wishes,
Wen

#3 lore raw

On 7/4/22 20:45, Wen Yang wrote:> On a 128-core Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz server,

Quoted 2 lines

when the user program frequently calls nv_alloc_system_pages to allocate large memory,

We seem to be repeating the same conversation we had back in November of last year:

Quoted 1 line

https://lore.kernel.org/all/[email protected]/T/#u

It's the binary nvidia driver doing unusual stuff again. Please talk to the folks you got that driver from.

#4 lore raw
In reply to Wen Yang
Quoted 78 lines · Wen Yang
Quoted 57 lines · Greg Kroah-Hartman
Quoted 52 lines · Wen Yang

From: Peter Zijlstra <[email protected]>

commit ddd07b750382adc2b78fdfbec47af8a6e0d8ef37 upstream.

CAT has happened, WBINDV is bad (even before CAT blowing away the entire cache on a multi-core platform wasn't nice), try not to use it ever.

---
  arch/x86/mm/pageattr.c | 18 ++----------------
  1 file changed, 2 insertions(+), 16 deletions(-)
arch/x86/mm/pageattr.c +2 -16
index 101f3ad0d6ad..ab87da7a6043 100644
@@ -239,26 +239,12 @@ static void cpa_flush_array(unsigned long *start, int numpages, int cache,
  			    int in_flags, struct page **pages)
  {
  	unsigned int i, level;
-#ifdef CONFIG_PREEMPT
-	/*
-	 * Avoid wbinvd() because it causes latencies on all CPUs,
-	 * regardless of any CPU isolation that may be in effect.
-	 *
-	 * This should be extended for CAT enabled systems independent of
-	 * PREEMPT because wbinvd() does not respect the CAT partitions and
-	 * this is exposed to unpriviledged users through the graphics
-	 * subsystem.
-	 */
-	unsigned long do_wbinvd = 0;
-#else
-	unsigned long do_wbinvd = cache && numpages >= 1024; /* 4M threshold */
-#endif
  	BUG_ON(irqs_disabled() && !early_boot_irqs_disabled);
-	on_each_cpu(__cpa_flush_all, (void *) do_wbinvd, 1);
+	flush_tlb_all();
-	if (!cache || do_wbinvd)
+	if (!cache)
  		return;
  	/*
Signature · 1 line
2.19.1.6.gb485710b

Why is this needed on 4.19.y? What problem does it solve, it looks only like an optimization, not a bugfix.

Quoted 4 lines

And if it's a bugfix, why only 4.19.y, why not older kernels too?

We need more information here please.

On a 128-core Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz server, when the user program frequently calls nv_alloc_system_pages to allocate large memory, it often causes a delay of about 200 milliseconds for the entire system. In this way, other latency-sensitive tasks on this system are heavily impacted, causing stability issues in large-scale clusters as well.

nv_alloc_system_pages
-> _set_memory_array
-> change_page_attr_set_clr
-> cpa_flush_array
-> on_each_cpu(__cpa_flush_all, (void *) do_wbinvd, 1);

Quoted 3 lines

This patch can be directly merged into the 4.19 kernel to solve this problem, and most of the machines in our production environment are 4.19 kernels.

Ah. So what has changed from last year when I rejected this then:

	https://lore.kernel.org/all/[email protected]/T/#m06369d080fa97eda3dd6a8eaf54a8ca2d430b3ab

Please do not try to submit previously-rejected patches, that is very disingenuous.

greg k-h