Skip to contentSkip to conversation

mm/vmstat: Defer the refresh_zone_stat_thresholds after all CPUs bringup

PATCH Applied Saurabh Sengar 6 participants 12 messages Aug 12, 2024 – Sep 24, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm/vmstat: Defer the refresh_zone_stat_thresholds after all CPUs bringup

refresh_zone_stat_thresholds function has two loops which is expensive for higher number of CPUs and NUMA nodes.

Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.

Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)

For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:

No NUMA = 1024*2*1*512 = 1,048,576 : Here refresh_zone_stat_thresholds
takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Here refresh_zone_stat_thresholds takes around 4.5 seconds total for all the CPUs in the system under test.

Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring refresh_zone_stat_thresholds to be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.

---
[V2]
- Move vmstat_late_init_done under CONFIG_SMP to fix variable 'defined but not used' warning.

 mm/vmstat.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
mm/vmstat.c +11 -1
index 4e2dc067a654..fa235c65c756 100644
@@ -1908,6 +1908,7 @@ static const struct seq_operations vmstat_op = {
 #ifdef CONFIG_SMP
 static DEFINE_PER_CPU(struct delayed_work, vmstat_work);
 int sysctl_stat_interval __read_mostly = HZ;
+static int vmstat_late_init_done;
 
 #ifdef CONFIG_PROC_FS
 static void refresh_vm_stats(struct work_struct *work)
@@ -2110,7 +2111,8 @@ static void __init init_cpu_node_state(void)
 
 static int vmstat_cpu_online(unsigned int cpu)
 {
-	refresh_zone_stat_thresholds();
+	if (vmstat_late_init_done)
+		refresh_zone_stat_thresholds();
 
 	if (!node_state(cpu_to_node(cpu), N_CPU)) {
 		node_set_state(cpu_to_node(cpu), N_CPU);
@@ -2142,6 +2144,14 @@ static int vmstat_cpu_dead(unsigned int cpu)
 	return 0;
 }
 
+static int __init vmstat_late_init(void)
+{
+	refresh_zone_stat_thresholds();
+	vmstat_late_init_done = 1;
+
+	return 0;
+}
+late_initcall(vmstat_late_init);
 #endif
 
 struct workqueue_struct *mm_percpu_wq;
Signature · 1 line
2.43.0
#1 lore raw
Quoted 34 lines

-----Original Message-----
From: Saurabh Sengar <[email protected]>
Sent: 12 August 2024 11:44
To: [email protected]; [email protected]; linux- [email protected]
Cc: Saurabh Singh Sengar <[email protected]>; [email protected]; [email protected]
Subject: [PATCH v2] mm/vmstat: Defer the refresh_zone_stat_thresholds after all CPUs bringup

refresh_zone_stat_thresholds function has two loops which is expensive for higher number of CPUs and NUMA nodes.

Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.

Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)

For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:

No NUMA = 1024*2*1*512 = 1,048,576 : Here refresh_zone_stat_thresholds
takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Here refresh_zone_stat_thresholds takes around 4.5 seconds total for all the CPUs in the system under test.

Quoted 7 lines

Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring refresh_zone_stat_thresholds to be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.

CC Mel Gorman and Christoph Lameter
#2 lore raw
Quoted 53 lines

-----Original Message-----
From: Saurabh Singh Sengar <[email protected]>
Sent: 23 August 2024 15:00
To: Saurabh Sengar <[email protected]>; akpm@linux- foundation.org; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXTERNAL] RE: [PATCH v2] mm/vmstat: Defer the refresh_zone_stat_thresholds after all CPUs bringup

Quoted 22 lines

-----Original Message-----
From: Saurabh Sengar <[email protected]>
Sent: 12 August 2024 11:44
To: [email protected]; [email protected]; linux- [email protected]
Cc: Saurabh Singh Sengar <[email protected]>; [email protected]; [email protected]
Subject: [PATCH v2] mm/vmstat: Defer the refresh_zone_stat_thresholds after all CPUs bringup

refresh_zone_stat_thresholds function has two loops which is expensive for higher number of CPUs and NUMA nodes.

Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.

Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)

Quoted 5 lines

For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:

No NUMA = 1024*2*1*512 = 1,048,576 : Here

refresh_zone_stat_thresholds

Quoted 11 lines

takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Here refresh_zone_stat_thresholds takes around 4.5 seconds total for all the CPUs in the system under test.

Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring refresh_zone_stat_thresholds to be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.

Quoted 3 lines
Quoted 1 line
CC Mel Gorman and Christoph Lameter

Adding [email protected] instead of [email protected] for Christoph Lameter

- Saurabh

#3 lore raw
Quoted 30 lines
Quoted 30 lines
Quoted 13 lines

refresh_zone_stat_thresholds function has two loops which is expensive for higher number of CPUs and NUMA nodes.

Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.

Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)

Quoted 5 lines

For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:

No NUMA = 1024*2*1*512 = 1,048,576 : Here

refresh_zone_stat_thresholds

Quoted 11 lines

takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Here refresh_zone_stat_thresholds takes around 4.5 seconds total for all the CPUs in the system under test.

Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring refresh_zone_stat_thresholds to be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line
CC Mel Gorman and Christoph Lameter

Adding [email protected] instead of [email protected] for Christoph Lameter

- Saurabh

Hi Andrew,

Can we get this merge in for next kernel release. Please let me know if there is any concern with this patch.

Regards,
Saurabh

maintainer
#4 lore raw
Quoted 11 lines · On Thu, 19 Sep 2024 19:52:45 +0000 Saurabh Singh Sengar…
Quoted 5 lines
Quoted 1 line
Quoted 1 line

Adding [email protected] instead of [email protected] for Christoph Lameter

- Saurabh

Hi Andrew,

Can we get this merge in for next kernel release. Please let me know if there is any concern with this patch.

Anshuman's review comment remains unaddressed:
https://lkml.kernel.org/r/[email protected]

Also, Christoph's observations from the v1 patch review haven't really been addressed.

So it sounds to me that an alternative implementation should be investigated?

#5 lore raw
Quoted 14 lines · Andrew Morton
Quoted 11 lines · On Thu, 19 Sep 2024 19:52:45 +0000 Saurabh Singh Sengar…
Quoted 5 lines
Quoted 1 line
Quoted 1 line

Adding [email protected] instead of [email protected] for Christoph Lameter

- Saurabh

Hi Andrew,

Can we get this merge in for next kernel release. Please let me know if there is any concern with this patch.

Quoted 8 lines

Anshuman's review comment remains unaddressed:
https://lkml.kernel.org/r/[email protected]

Also, Christoph's observations from the v1 patch review haven't really been addressed.

So it sounds to me that an alternative implementation should be investigated?

I believe Saurabh had a follow-up discussion in person with Christoph regarding this patch, following our talk on this topic at LPC:
https://lpc.events/event/18/contributions/1817/

@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?

Thanks a lot!

Regards,
Srivatsa
Microsoft Linux Systems Group

Acked-by Christoph Lameter
#6 lore raw
Quoted 2 lines · Srivatsa S. Bhat

@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?

V2 looks good to me (unitialized pcp values result in slow operation but no negative other effects) and the late_initcall() is always executed.

Reviewed-by
#7 lore raw
Quoted 9 lines · Christoph Lameter (Ampere)
Quoted 2 lines · Srivatsa S. Bhat

@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?

V2 looks good to me (unitialized pcp values result in slow operation but no negative other effects) and the late_initcall() is always executed.

Thanks a lot Christoph!

Andrew, could you please consider picking up the patch for the next release, now that all the review comments have been addressed? Thank you very much!

Also, I'd like to add to this patch v2:

Regards,
Srivatsa

#8 lore raw
Quoted 11 lines · Srivatsa S. Bhat
Quoted 9 lines · Christoph Lameter (Ampere)
Quoted 2 lines · Srivatsa S. Bhat

@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?

V2 looks good to me (unitialized pcp values result in slow operation but no negative other effects) and the late_initcall() is always executed.

Quoted 8 lines

Thanks a lot Christoph!

Andrew, could you please consider picking up the patch for the next release, now that all the review comments have been addressed? Thank you very much!

Also, I'd like to add to this patch v2:

Thanks Srivatsa !

- Saurabh

#9 lore raw
In reply to Christoph Lameter (Ampere)
Quoted 9 lines · Christoph Lameter (Ampere)
Quoted 2 lines · Srivatsa S. Bhat

@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?

V2 looks good to me (unitialized pcp values result in slow operation but no negative other effects) and the late_initcall() is always executed.

Thank you, Christoph. I truly appreciate your review and insights on this matter. Looking forward to collobrate more on similar space.

- Saurabh

#10 lore raw
In reply to Saurabh Sengar
Quoted 62 lines · Saurabh Sengar

refresh_zone_stat_thresholds function has two loops which is expensive for higher number of CPUs and NUMA nodes.

Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.

Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)

For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:

No NUMA = 1024*2*1*512 = 1,048,576 : Here refresh_zone_stat_thresholds
takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Here refresh_zone_stat_thresholds takes around 4.5 seconds total for all the CPUs in the system under test.

Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring refresh_zone_stat_thresholds to be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.

---
[V2]
- Move vmstat_late_init_done under CONFIG_SMP to fix variable 'defined but not used' warning.

 mm/vmstat.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
mm/vmstat.c +3 -1
index 4e2dc067a654..fa235c65c756 100644
@@ -1908,6 +1908,7 @@ static const struct seq_operations vmstat_op = {
 #ifdef CONFIG_SMP
 static DEFINE_PER_CPU(struct delayed_work, vmstat_work);
 int sysctl_stat_interval __read_mostly = HZ;
+static int vmstat_late_init_done;
 
 #ifdef CONFIG_PROC_FS
 static void refresh_vm_stats(struct work_struct *work)
@@ -2110,7 +2111,8 @@ static void __init init_cpu_node_state(void)
 
 static int vmstat_cpu_online(unsigned int cpu)
 {
-	refresh_zone_stat_thresholds();
+	if (vmstat_late_init_done)
+		refresh_zone_stat_thresholds();
 
 	if (!node_state(cpu_to_node(cpu), N_CPU)) {
 		node_set_state(cpu_to_node(cpu), N_CPU);
@@ -2142,6 +2144,14 @@ static int vmstat_cpu_dead(unsigned int cpu)
 	return 0;
 }
 

mm/vmstat.c static int vmstat_cpu_dead(unsigned int cpu) · line 2156

Quoted 10 lines
mm/vmstat.c +8 -0
+static int __init vmstat_late_init(void)
+{
+	refresh_zone_stat_thresholds();
+	vmstat_late_init_done = 1;
+
+	return 0;
+}
+late_initcall(vmstat_late_init);>  #endif
 
 struct workqueue_struct *mm_percpu_wq;

late_initcall() triggered vmstat_late_init() guaranteed to be called before the last call into vmstat_cpu_online() during a normal boot ? Otherwise refresh_zone_stat_thresholds() will never be called unless there is a CPU online event later.

#11 lore raw

Hey Anshuman,

Long time... :-) Hope you are doing great!

On Fri, Sep 20, 2024 at 12:28:44PM +0530, Anshuman Khandual wrote:
[...]

Quoted 31 lines
Quoted 31 lines
mm/vmstat.c +11 -1
@@ -1908,6 +1908,7 @@ static const struct seq_operations vmstat_op = {
 #ifdef CONFIG_SMP
 static DEFINE_PER_CPU(struct delayed_work, vmstat_work);
 int sysctl_stat_interval __read_mostly = HZ;
+static int vmstat_late_init_done;
 
 #ifdef CONFIG_PROC_FS
 static void refresh_vm_stats(struct work_struct *work)
@@ -2110,7 +2111,8 @@ static void __init init_cpu_node_state(void)
 
 static int vmstat_cpu_online(unsigned int cpu)
 {
-	refresh_zone_stat_thresholds();
+	if (vmstat_late_init_done)
+		refresh_zone_stat_thresholds();
 
 	if (!node_state(cpu_to_node(cpu), N_CPU)) {
 		node_set_state(cpu_to_node(cpu), N_CPU);
@@ -2142,6 +2144,14 @@ static int vmstat_cpu_dead(unsigned int cpu)
 	return 0;
 }
 
+static int __init vmstat_late_init(void)
+{
+	refresh_zone_stat_thresholds();
+	vmstat_late_init_done = 1;
+
+	return 0;
+}
+late_initcall(vmstat_late_init);>  #endif
 
Quoted 6 lines

mm/vmstat.c static int vmstat_cpu_dead(unsigned int cpu) · line 2157

Quoted 1 line
mm/vmstat.c +0 -0
 struct workqueue_struct *mm_percpu_wq;

late_initcall() triggered vmstat_late_init() guaranteed to be called before the last call into vmstat_cpu_online() during a normal boot ? Otherwise refresh_zone_stat_thresholds() will never be called unless there is a CPU online event later.

The vmstat_late_init() function itself calls refresh_zone_stat_thresholds(). So, we don't need another CPU online event to happen later just to invoke refresh_zone_stat_thresholds().

Does that address your concern?

Regards,
Srivatsa
Microsoft Linux Systems Group