mm/vmstat: Defer the refresh_zone_stat_thresholds after all CPUs bringup
Subsystems (2)
PATCH v2 1/1 mm/vmstat: Defer the refresh_zone_stat_thresholds after all CPUs bringup
refresh_zone_stat_thresholds function has two loops which is expensive for higher number of CPUs and NUMA nodes.
Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.
Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)
For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:
No NUMA = 1024*2*1*512 = 1,048,576 : Here refresh_zone_stat_thresholds
takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Here refresh_zone_stat_thresholds takes around 4.5 seconds total for all the CPUs in the system under test.
Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring refresh_zone_stat_thresholds to be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.
---
[V2]
- Move vmstat_late_init_done under CONFIG_SMP to fix variable 'defined but not used' warning.
mm/vmstat.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
mm/vmstat.c +11 -1
@@ -1908,6 +1908,7 @@ static const struct seq_operations vmstat_op = {#ifdef CONFIG_SMP static DEFINE_PER_CPU(struct delayed_work, vmstat_work);int sysctl_stat_interval__read_mostly =HZ ; +static int vmstat_late_init_done;#ifdef CONFIG_PROC_FS static void refresh_vm_stats (struct work_struct *work)
@@ -2110,7 +2111,8 @@ static void __init init_cpu_node_state(void)static int vmstat_cpu_online (unsigned int cpu) { -refresh_zone_stat_thresholds (); +if (vmstat_late_init_done) +refresh_zone_stat_thresholds ();if (!node_state (cpu_to_node (cpu),N_CPU )) {node_set_state (cpu_to_node (cpu),N_CPU );
@@ -2142,6 +2144,14 @@ static int vmstat_cpu_dead(unsigned int cpu) return 0; } +static int __init vmstat_late_init (void ) +{ +refresh_zone_stat_thresholds (); + vmstat_late_init_done =1 ; + +return 0 ; +} +late_initcall (vmstat_late_init);#endif struct workqueue_struct *mm_percpu_wq;
Signature · 1 line
2.43.0
Quoted 34 lines
-----Original Message-----
From: Saurabh Sengar <[email protected]>
Sent: 12 August 2024 11:44
To: [email protected]; [email protected]; linux- [email protected]
Cc: Saurabh Singh Sengar <[email protected]>; [email protected]; [email protected]
Subject: [PATCH v2] mm/vmstat: Defer therefresh_zone_stat_thresholdsafter all CPUs bringup
refresh_zone_stat_thresholdsfunction has two loops which is expensive for higher number of CPUs and NUMA nodes.Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.
Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:
No NUMA = 1024*2*1*512 = 1,048,576 : Here
refresh_zone_stat_thresholds
takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Hererefresh_zone_stat_thresholdstakes around 4.5 seconds total for all the CPUs in the system under test.
Quoted 7 lines
Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring
refresh_zone_stat_thresholdsto be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.Signed-off-by Saurabh Sengar <[email protected]>
Quoted 53 lines
-----Original Message-----
From: Saurabh Singh Sengar <[email protected]>
Sent: 23 August 2024 15:00
To: Saurabh Sengar <[email protected]>; akpm@linux- foundation.org; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXTERNAL] RE: [PATCH v2] mm/vmstat: Defer therefresh_zone_stat_thresholdsafter all CPUs bringupQuoted 22 lines
-----Original Message-----
From: Saurabh Sengar <[email protected]>
Sent: 12 August 2024 11:44
To: [email protected]; [email protected]; linux- [email protected]
Cc: Saurabh Singh Sengar <[email protected]>; [email protected]; [email protected]
Subject: [PATCH v2] mm/vmstat: Defer therefresh_zone_stat_thresholdsafter all CPUs bringup
refresh_zone_stat_thresholdsfunction has two loops which is expensive for higher number of CPUs and NUMA nodes.Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.
Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)Quoted 5 lines
For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:
No NUMA = 1024*2*1*512 = 1,048,576 : Here
refresh_zone_stat_thresholdsQuoted 11 lines
takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Here
refresh_zone_stat_thresholdstakes around 4.5 seconds total for all the CPUs in the system under test.Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring
refresh_zone_stat_thresholdsto be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.
Quoted 3 lines
Quoted 1 line
Signed-off-by Saurabh Sengar <[email protected]>CC Mel Gorman and Christoph Lameter
Adding [email protected] instead of [email protected] for Christoph Lameter
- Saurabh
Quoted 30 lines
Quoted 30 lines
Quoted 13 lines
refresh_zone_stat_thresholdsfunction has two loops which is expensive for higher number of CPUs and NUMA nodes.Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.
Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)Quoted 5 lines
For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:
No NUMA = 1024*2*1*512 = 1,048,576 : Here
refresh_zone_stat_thresholdsQuoted 11 lines
takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Here
refresh_zone_stat_thresholdstakes around 4.5 seconds total for all the CPUs in the system under test.Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring
refresh_zone_stat_thresholdsto be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Saurabh Sengar <[email protected]>CC Mel Gorman and Christoph LameterAdding [email protected] instead of [email protected] for Christoph Lameter
- Saurabh
Hi Andrew,
Can we get this merge in for next kernel release. Please let me know if there is any concern with this patch.
Regards,
Saurabh
Andrew Morton
maintainerQuoted 11 lines · On Thu, 19 Sep 2024 19:52:45 +0000 Saurabh Singh Sengar…
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Adding [email protected] instead of [email protected] for Christoph Lameter
- Saurabh
Hi Andrew,
Can we get this merge in for next kernel release. Please let me know if there is any concern with this patch.
Anshuman's review comment remains unaddressed:
https://lkml.kernel.org/r/[email protected]
Also, Christoph's observations from the v1 patch review haven't really been addressed.
So it sounds to me that an alternative implementation should be investigated?
Quoted 14 lines · Andrew Morton
Quoted 11 lines · On Thu, 19 Sep 2024 19:52:45 +0000 Saurabh Singh Sengar…
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Adding [email protected] instead of [email protected] for Christoph Lameter
- Saurabh
Hi Andrew,
Can we get this merge in for next kernel release. Please let me know if there is any concern with this patch.
Quoted 8 lines
Anshuman's review comment remains unaddressed:
https://lkml.kernel.org/r/[email protected]Also, Christoph's observations from the v1 patch review haven't really been addressed.
So it sounds to me that an alternative implementation should be investigated?
I believe Saurabh had a follow-up discussion in person with Christoph regarding this patch, following our talk on this topic at LPC:
https://lpc.events/event/18/contributions/1817/
@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?
Thanks a lot!
Regards,
Srivatsa
Microsoft Linux Systems Group
Christoph Lameter (Ampere)
Acked-by Christoph LameterQuoted 2 lines · Srivatsa S. Bhat
@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?
V2 looks good to me (unitialized pcp values result in slow operation but no negative other effects) and the late_initcall() is always executed.
Srivatsa S. Bhat
Reviewed-byQuoted 9 lines · Christoph Lameter (Ampere)
Quoted 2 lines · Srivatsa S. Bhat
@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?
V2 looks good to me (unitialized pcp values result in slow operation but no negative other effects) and the
late_initcall()is always executed.
Thanks a lot Christoph!
Andrew, could you please consider picking up the patch for the next release, now that all the review comments have been addressed? Thank you very much!
Also, I'd like to add to this patch v2:
Regards,
Srivatsa
Quoted 11 lines · Srivatsa S. Bhat
Quoted 9 lines · Christoph Lameter (Ampere)
Quoted 2 lines · Srivatsa S. Bhat
@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?
V2 looks good to me (unitialized pcp values result in slow operation but no negative other effects) and the
late_initcall()is always executed.
Quoted 8 lines
Thanks a lot Christoph!
Andrew, could you please consider picking up the patch for the next release, now that all the review comments have been addressed? Thank you very much!
Also, I'd like to add to this patch v2:
Reviewed-by Srivatsa S. Bhat (Microsoft) <[email protected]>
Thanks Srivatsa !
- Saurabh
Quoted 9 lines · Christoph Lameter (Ampere)
Quoted 2 lines · Srivatsa S. Bhat
@Christoph, would you mind giving your Ack if this patch v2 looks good to you, or kindly point out if there are any lingering concerns?
V2 looks good to me (unitialized pcp values result in slow operation but no negative other effects) and the
late_initcall()is always executed.
Thank you, Christoph. I truly appreciate your review and insights on this matter. Looking forward to collobrate more on similar space.
- Saurabh
Quoted 62 lines · Saurabh Sengar
refresh_zone_stat_thresholdsfunction has two loops which is expensive for higher number of CPUs and NUMA nodes.Below is the rough estimation of total iterations done by these loops based on number of NUMA and CPUs.
Total number of iterations: nCPU * 2 * Numa * mCPU Where:
nCPU = total number of CPUs
Numa = total number of NUMA nodes
mCPU = mean value of total CPUs (e.g., 512 for 1024 total CPUs)For the system under test with 16 NUMA nodes and 1024 CPUs, this results in a substantial increase in the number of loop iterations during boot-up when NUMA is enabled:
No NUMA = 1024*2*1*512 = 1,048,576 : Here
refresh_zone_stat_thresholds
takes around 224 ms total for all the CPUs in the system under test. 16 NUMA = 1024*2*16*512 = 16,777,216 : Hererefresh_zone_stat_thresholdstakes around 4.5 seconds total for all the CPUs in the system under test.Calling this for each CPU is expensive when there are large number of CPUs along with multiple NUMAs. Fix this by deferring
refresh_zone_stat_thresholdsto be called later at once when all the secondary CPUs are up. Also, register the DYN hooks to keep the existing hotplug functionality intact.Signed-off-by Saurabh Sengar <[email protected]>---
[V2]
- Movevmstat_late_init_doneunderCONFIG_SMPto fix variable 'defined but not used' warning.mm/vmstat.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)mm/vmstat.c +3
-1@@ -1908,6 +1908,7 @@ static const struct seq_operations vmstat_op = {#ifdef CONFIG_SMP static DEFINE_PER_CPU(struct delayed_work, vmstat_work);int sysctl_stat_interval__read_mostly =HZ ; +static int vmstat_late_init_done;#ifdef CONFIG_PROC_FS static void refresh_vm_stats (struct work_struct *work)@@ -2110,7 +2111,8 @@ static void __init init_cpu_node_state(void)static int vmstat_cpu_online (unsigned int cpu) { -refresh_zone_stat_thresholds (); +if (vmstat_late_init_done) +refresh_zone_stat_thresholds ();if (!node_state (cpu_to_node (cpu),N_CPU )) {node_set_state (cpu_to_node (cpu),N_CPU );@@ -2142,6 +2144,14 @@ static int vmstat_cpu_dead(unsigned int cpu)return 0 ; }
mm/vmstat.c static int vmstat_cpu_dead(unsigned int cpu) · line 2156
Quoted 10 lines
mm/vmstat.c +8
-0+static int __init vmstat_late_init (void ) +{ +refresh_zone_stat_thresholds (); + vmstat_late_init_done =1 ; + +return 0 ; +} +late_initcall(vmstat_late_init);> #endifstruct workqueue_struct *mm_percpu_wq;
late_initcall() triggered vmstat_late_init() guaranteed to be called before the last call into vmstat_cpu_online() during a normal boot ? Otherwise refresh_zone_stat_thresholds() will never be called unless there is a CPU online event later.
Hey Anshuman,
Long time... :-) Hope you are doing great!
On Fri, Sep 20, 2024 at 12:28:44PM +0530, Anshuman Khandual wrote:
[...]
Quoted 31 lines
Quoted 31 lines
mm/vmstat.c +11
-1@@ -1908,6 +1908,7 @@ static const struct seq_operations vmstat_op = {#ifdef CONFIG_SMP static DEFINE_PER_CPU(struct delayed_work, vmstat_work);int sysctl_stat_interval__read_mostly =HZ ; +static int vmstat_late_init_done;#ifdef CONFIG_PROC_FS static void refresh_vm_stats (struct work_struct *work)@@ -2110,7 +2111,8 @@ static void __init init_cpu_node_state(void)static int vmstat_cpu_online (unsigned int cpu) { -refresh_zone_stat_thresholds (); +if (vmstat_late_init_done) +refresh_zone_stat_thresholds ();if (!node_state (cpu_to_node (cpu),N_CPU )) {node_set_state (cpu_to_node (cpu),N_CPU );@@ -2142,6 +2144,14 @@ static int vmstat_cpu_dead(unsigned int cpu)return 0 ; } +static int __init vmstat_late_init (void ) +{ +refresh_zone_stat_thresholds (); + vmstat_late_init_done =1 ; + +return 0 ; +} +late_initcall(vmstat_late_init);> #endif
Quoted 6 lines
mm/vmstat.c static int vmstat_cpu_dead(unsigned int cpu) · line 2157
Quoted 1 line
mm/vmstat.c +0
-0struct workqueue_struct *mm_percpu_wq;
late_initcall()triggeredvmstat_late_init()guaranteed to be called before the last call intovmstat_cpu_online()during a normal boot ? Otherwiserefresh_zone_stat_thresholds()will never be called unless there is a CPU online event later.
The vmstat_late_init() function itself calls refresh_zone_stat_thresholds(). So, we don't need another CPU online event to happen later just to invoke refresh_zone_stat_thresholds().
Does that address your concern?
Regards,
Srivatsa
Microsoft Linux Systems Group