mm/mm_init.c: remove spinlock in early_pfn_to_nid()
PATCH 1/1 mm/mm_init.c: remove spinlock in early_pfn_to_nid()
When the system boots, only one cpu is enabled before smp_init(). So the spinlock is not needed in most cases, remove it.
Add spinlock in get_nid_for_pfn() because it is after smp_init().
--- drivers/base/node.c | 11 +++++++++-- mm/mm_init.c | 18 +++--------------- 2 files changed, 12 insertions(+), 17 deletions(-)
drivers/base/node.c +9 -2
@@ -748,8 +748,15 @@ int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)static int __ref get_nid_for_pfn (unsigned long pfn) {#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT -if (system_state <SYSTEM_RUNNING ) -return early_pfn_to_nid (pfn); +static DEFINE_SPINLOCK (early_pfn_lock); +int nid; + +if (system_state <SYSTEM_RUNNING ) { +spin_lock (&early_pfn_lock); + nid =early_pfn_to_nid (pfn); +spin_unlock (&early_pfn_lock); +return nid; + }#endif return pfn_to_nid (pfn); }
mm/mm_init.c +3 -15
@@ -584,11 +584,11 @@ struct mminit_pfnnid_cache {static struct mminit_pfnnid_cache early_pfnnid_cache__meminitdata ;/* -* Required by SPARSEMEM. Given a PFN, return what node the PFN is on. +* Given a PFN, return what node the PFN is on. */ -static int __meminit __early_pfn_to_nid (unsigned long pfn, -struct mminit_pfnnid_cache *state) +int __meminit early_pfn_to_nid (unsigned long pfn) { +struct mminit_pfnnid_cache *state = &early_pfnnid_cache;unsigned long start_pfn, end_pfn;int nid;
@@ -601,20 +601,8 @@ static int __meminit __early_pfn_to_nid(unsigned long pfn, state->last_end = end_pfn; state->last_nid = nid; } - -return nid; -} - -int __meminit early_pfn_to_nid (unsigned long pfn) -{ -static DEFINE_SPINLOCK (early_pfn_lock); -int nid; - -spin_lock (&early_pfn_lock); - nid =__early_pfn_to_nid (pfn, &early_pfnnid_cache);if (nid <0 ) nid = first_online_node; -spin_unlock (&early_pfn_lock);return nid; }
Signature · 1 line
2.25.1
Greg KH
maintainerQuoted 4 lines · Yajun Deng
When the system boots, only one cpu is enabled before
smp_init(). So the spinlock is not needed in most cases, remove it.Add spinlock in
get_nid_for_pfn()because it is aftersmp_init().
So this is two different things at once in the same patch?
Or are they the same problem and both need to go in to solve it?
And if a spinlock is not needed at early boot, is it really causing any problems?
Quoted 14 lines
Signed-off-by Yajun Deng <[email protected]>--- drivers/base/node.c | 11 +++++++++-- mm/mm_init.c | 18 +++--------------- 2 files changed, 12 insertions(+), 17 deletions(-)drivers/base/node.c +0
-0@@ -748,8 +748,15 @@ int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)static int __ref get_nid_for_pfn (unsigned long pfn) {
drivers/base/node.c int unregister_cpu_under_node(unsigned int cpu, unsigned int nid) · line 757
Quoted 10 lines
drivers/base/node.c +7
-2#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT -if (system_state <SYSTEM_RUNNING ) -return early_pfn_to_nid (pfn); +static DEFINE_SPINLOCK (early_pfn_lock); +int nid; + +if (system_state <SYSTEM_RUNNING ) { +spin_lock (&early_pfn_lock); + nid =early_pfn_to_nid (pfn); +spin_unlock (&early_pfn_lock);
Adding an external lock for when you call a function is VERY dangerous as you did not document this anywhere, and there's no way to enforce it properly at all.
Does your change actually result in any boot time changes? How was this tested?
thanks,
greg k-h
Quoted 5 lines · Greg KH
Quoted 3 lines · Yajun Deng
When the system boots, only one cpu is enabled before
smp_init(). So the spinlock is not needed in most cases, remove it.
Quoted 9 lines
Quoted 1 line
Add spinlock in
get_nid_for_pfn()because it is aftersmp_init().So this is two different things at once in the same patch?
Or are they the same problem and both need to go in to solve it?
And if a spinlock is not needed at early boot, is it really causing any problems?
They are the same problem.
I added pr_info in early_pfn_to_nid(), found get_nid_for_pfn() is the only case need to add spinlock.
This patch tested on my x86 system.
Quoted 19 lines
Quoted 19 lines
Signed-off-by Yajun Deng <[email protected]>---
drivers/base/node.c| 11 +++++++++--mm/mm_init.c| 18 +++---------------2 files changed, 12 insertions(+), 17 deletions(-)drivers/base/node.c +3
-2@@ -748,8 +748,15 @@ int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)static int __ref get_nid_for_pfn (unsigned long pfn) {#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT -if (system_state <SYSTEM_RUNNING ) -return early_pfn_to_nid (pfn); +static DEFINE_SPINLOCK (early_pfn_lock); +int nid; +
Quoted 9 lines
drivers/base/node.c int unregister_cpu_under_node(unsigned int cpu, unsigned int nid) · line 757
Quoted 4 lines
drivers/base/node.c +4
-0+if (system_state <SYSTEM_RUNNING ) { +spin_lock (&early_pfn_lock); + nid =early_pfn_to_nid (pfn); +spin_unlock (&early_pfn_lock);Adding an external lock for when you call a function is VERY dangerous as you did not document this anywhere, and there's no way to enforce it properly at all.
I should add a comment before early_pfn_to_nid().
Quoted 3 lines
Does your change actually result in any boot time changes? How was this tested?
Just a bit.
Quoted 3 lines
thanks,
greg k-h
Mike Rapoport
maintainerHi,
Quoted 13 lines · Yajun Deng
Quoted 11 lines · Greg KH
Quoted 4 lines · Yajun Deng
When the system boots, only one cpu is enabled before
smp_init(). So the spinlock is not needed in most cases, remove it.Add spinlock in
get_nid_for_pfn()because it is aftersmp_init().So this is two different things at once in the same patch?
Or are they the same problem and both need to go in to solve it?
Quoted 8 lines
Quoted 3 lines
And if a spinlock is not needed at early boot, is it really causing any problems?
They are the same problem.
I addedpr_infoinearly_pfn_to_nid(), foundget_nid_for_pfn()is the only case need to add spinlock.
This patch tested on my x86 system.
Are you sure it'll work on !x86?
Quoted 29 lines
Quoted 19 lines
Quoted 19 lines
Signed-off-by Yajun Deng <[email protected]>---
drivers/base/node.c| 11 +++++++++--mm/mm_init.c| 18 +++---------------2 files changed, 12 insertions(+), 17 deletions(-)drivers/base/node.c +3
-2@@ -748,8 +748,15 @@ int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)static int __ref get_nid_for_pfn (unsigned long pfn) {#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT -if (system_state <SYSTEM_RUNNING ) -return early_pfn_to_nid (pfn); +static DEFINE_SPINLOCK (early_pfn_lock); +int nid; +Quoted 9 lines
drivers/base/node.c int unregister_cpu_under_node(unsigned int cpu, unsigned int nid) · line 757
Quoted 4 lines
drivers/base/node.c +4
-0+if (system_state <SYSTEM_RUNNING ) { +spin_lock (&early_pfn_lock); + nid =early_pfn_to_nid (pfn); +spin_unlock (&early_pfn_lock);Adding an external lock for when you call a function is VERY dangerous as you did not document this anywhere, and there's no way to enforce it properly at all.
Quoted 7 lines
I should add a comment before
early_pfn_to_nid().Quoted 3 lines
Does your change actually result in any boot time changes? How was this tested?
Just a bit.
Just a bit tested? Or just a bit of boot time changes? For the latter, do you have numbers?
Signature · 2 lines
Sincerely yours, Mike.
Quoted 20 lines · Mike Rapoport
Hi,
Quoted 16 lines · Yajun Deng
June 14, 2023 7:09 PM, "Greg KH" <[email protected]> wrote:
On Wed, Jun 14, 2023 at 07:03:24PM +0800, Yajun Deng wrote:
When the system boots, only one cpu is enabled before
smp_init(). So the spinlock is not needed in most cases, remove it.Add spinlock in
get_nid_for_pfn()because it is aftersmp_init().So this is two different things at once in the same patch?
Or are they the same problem and both need to go in to solve it?
And if a spinlock is not needed at early boot, is it really causing any problems?
Quoted 7 lines
Quoted 4 lines
They are the same problem.
I addedpr_infoinearly_pfn_to_nid(), foundget_nid_for_pfn()is the only case need to add spinlock.
This patch tested on my x86 system.Are you sure it'll work on !x86?
I'm probably sure of that, although I don't have a !x86 machine.
early_pfn_to_nid() is called in smp_init() and kasan_init() on different architectures. If it works well on x86, it'll work on !x86.
Quoted 30 lines
Quoted 30 lines
Signed-off-by Yajun Deng <[email protected]>---
drivers/base/node.c| 11 +++++++++--mm/mm_init.c| 18 +++---------------2 files changed, 12 insertions(+), 17 deletions(-)drivers/base/node.c +7
-2@@ -748,8 +748,15 @@ int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)static int __ref get_nid_for_pfn (unsigned long pfn) {#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT -if (system_state <SYSTEM_RUNNING ) -return early_pfn_to_nid (pfn); +static DEFINE_SPINLOCK (early_pfn_lock); +int nid; + +if (system_state <SYSTEM_RUNNING ) { +spin_lock (&early_pfn_lock); + nid =early_pfn_to_nid (pfn); +spin_unlock (&early_pfn_lock);Adding an external lock for when you call a function is VERY dangerous as you did not document this anywhere, and there's no way to enforce it properly at all.
I should add a comment before
early_pfn_to_nid().
Quoted 8 lines
Quoted 4 lines
Does your change actually result in any boot time changes? How was this tested?
Just a bit.
Just a bit tested? Or just a bit of boot time changes? For the latter, do you have numbers?
For the latter, the most beneficial function is memmap_init_reserved_pages(), the boot time changes depending on whether DEFERRED_STRUCT_PAGE_INIT is defined or not.
-->memmap_init_reserved_pages()
-->for_each_reserved_mem_range()
reserve_bootmem_region()
-->for()
init_reserved_page()
--> early_pfn_to_nid()If define CONFIG_DEFERRED_STRUCT_PAGE_INIT:
before:memmap_init_reserved_pages() 1.87 seconds
after:memmap_init_reserved_pages() 1.27 seconds
32% time reduction.
If not define CONFIG_DEFERRED_STRUCT_PAGE_INIT:
early_pfn_to_nid() is called by few,
boot time didn't change.
By the way, this machine has 190GB RAM.
Quoted 3 lines
--
Sincerely yours,
Mike.
Mike Rapoport
maintainerQuoted 27 lines · Yajun Deng
Quoted 25 lines · Mike Rapoport
Hi,
Quoted 20 lines · Yajun Deng
June 14, 2023 7:09 PM, "Greg KH" <[email protected]> wrote:
On Wed, Jun 14, 2023 at 07:03:24PM +0800, Yajun Deng wrote:
When the system boots, only one cpu is enabled before
smp_init(). So the spinlock is not needed in most cases, remove it.Add spinlock in
get_nid_for_pfn()because it is aftersmp_init().So this is two different things at once in the same patch?
Or are they the same problem and both need to go in to solve it?
And if a spinlock is not needed at early boot, is it really causing any problems?
They are the same problem.
I addedpr_infoinearly_pfn_to_nid(), foundget_nid_for_pfn()is the only case need to add spinlock.
This patch tested on my x86 system.
Quoted 8 lines
Quoted 2 lines
Are you sure it'll work on !x86?
I'm probably sure of that, although I don't have a !x86 machine.
early_pfn_to_nid()is called insmp_init()andkasan_init()on different architectures. If it works well on x86, it'll work on !x86.
This is often not true. Please verify that other architectures do not call early_pfn_to_nid() after smp_init(). The explanation why it is safe should be a part of the changelog.
Quoted 43 lines
Quoted 30 lines
Quoted 30 lines
Signed-off-by Yajun Deng <[email protected]>---
drivers/base/node.c| 11 +++++++++--mm/mm_init.c| 18 +++---------------2 files changed, 12 insertions(+), 17 deletions(-)drivers/base/node.c +7
-2@@ -748,8 +748,15 @@ int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)static int __ref get_nid_for_pfn (unsigned long pfn) {#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT -if (system_state <SYSTEM_RUNNING ) -return early_pfn_to_nid (pfn); +static DEFINE_SPINLOCK (early_pfn_lock); +int nid; + +if (system_state <SYSTEM_RUNNING ) { +spin_lock (&early_pfn_lock); + nid =early_pfn_to_nid (pfn); +spin_unlock (&early_pfn_lock);Adding an external lock for when you call a function is VERY dangerous as you did not document this anywhere, and there's no way to enforce it properly at all.
I should add a comment before
early_pfn_to_nid().Quoted 8 lines
Quoted 4 lines
Does your change actually result in any boot time changes? How was this tested?
Just a bit.
Just a bit tested? Or just a bit of boot time changes? For the latter, do you have numbers?
For the latter, the most beneficial function is
memmap_init_reserved_pages(), the boot time changes depending on whetherDEFERRED_STRUCT_PAGE_INITis defined or not.
Quoted 6 lines
-->memmap_init_reserved_pages()
-->for_each_reserved_mem_range()reserve_bootmem_region() -->for() init_reserved_page() --> early_pfn_to_nid()
A better solution would be to pass nid to reserve_bootmem_range() and drop the call to early_pfn_to_nid() in init_reserved_page().
Then there won't be lock contention and no need for fragile changes in the locking.
Quoted 8 lines
If define
CONFIG_DEFERRED_STRUCT_PAGE_INIT:before:
memmap_init_reserved_pages()1.87 seconds
after:memmap_init_reserved_pages()1.27 seconds32% time reduction.
These measurements should be part of the changelog.
Quoted 11 lines
If not define
CONFIG_DEFERRED_STRUCT_PAGE_INIT:
early_pfn_to_nid()is called by few,
boot time didn't change.By the way, this machine has 190GB RAM.
Quoted 3 lines
--
Sincerely yours,
Mike.
Signature · 2 lines
Sincerely yours, Mike.
Quoted 93 lines · Mike Rapoport
Quoted 27 lines · Yajun Deng
June 14, 2023 7:53 PM, "Mike Rapoport" <[email protected]> wrote:
Hi,
On Wed, Jun 14, 2023 at 11:28:32AM +0000, Yajun Deng wrote:
June 14, 2023 7:09 PM, "Greg KH" <[email protected]> wrote:
On Wed, Jun 14, 2023 at 07:03:24PM +0800, Yajun Deng wrote:
When the system boots, only one cpu is enabled before
smp_init(). So the spinlock is not needed in most cases, remove it.Add spinlock in
get_nid_for_pfn()because it is aftersmp_init().So this is two different things at once in the same patch?
Or are they the same problem and both need to go in to solve it?
And if a spinlock is not needed at early boot, is it really causing any problems?
They are the same problem.
I addedpr_infoinearly_pfn_to_nid(), foundget_nid_for_pfn()is the only case need to add spinlock.
This patch tested on my x86 system.Quoted 7 lines
Are you sure it'll work on !x86?
I'm probably sure of that, although I don't have a !x86 machine.
early_pfn_to_nid()is called insmp_init()andkasan_init()on different architectures. If it works well on x86, it'll work on !x86.This is often not true. Please verify that other architectures do not call
early_pfn_to_nid()aftersmp_init(). The explanation why it is safe should be a part of the changelog.Quoted 42 lines
Signed-off-by Yajun Deng <[email protected]>---
drivers/base/node.c| 11 +++++++++--mm/mm_init.c| 18 +++---------------2 files changed, 12 insertions(+), 17 deletions(-)drivers/base/node.c +7
-2@@ -748,8 +748,15 @@ int unregister_cpu_under_node(unsigned int cpu, unsigned int nid)static int __ref get_nid_for_pfn (unsigned long pfn) {#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT -if (system_state <SYSTEM_RUNNING ) -return early_pfn_to_nid (pfn); +static DEFINE_SPINLOCK (early_pfn_lock); +int nid; + +if (system_state <SYSTEM_RUNNING ) { +spin_lock (&early_pfn_lock); + nid =early_pfn_to_nid (pfn); +spin_unlock (&early_pfn_lock);Adding an external lock for when you call a function is VERY dangerous as you did not document this anywhere, and there's no way to enforce it properly at all.
I should add a comment before
early_pfn_to_nid().Does your change actually result in any boot time changes? How was this tested?
Just a bit.
Just a bit tested? Or just a bit of boot time changes? For the latter, do you have numbers?
For the latter, the most beneficial function is
memmap_init_reserved_pages(), the boot time changes depending on whetherDEFERRED_STRUCT_PAGE_INITis defined or not.Quoted 6 lines
-->memmap_init_reserved_pages()
-->for_each_reserved_mem_range()reserve_bootmem_region()
-->for()init_reserved_page()
-->early_pfn_to_nid()A better solution would be to pass nid to
reserve_bootmem_range()and drop the call toearly_pfn_to_nid()ininit_reserved_page().
Quoted 3 lines
Then there won't be lock contention and no need for fragile changes in the locking.
Great, I will try it.
Quoted 25 lines
Quoted 8 lines
If define
CONFIG_DEFERRED_STRUCT_PAGE_INIT:before:
memmap_init_reserved_pages()1.87 seconds
after:memmap_init_reserved_pages()1.27 seconds32% time reduction.
These measurements should be part of the changelog.
Quoted 10 lines
If not define
CONFIG_DEFERRED_STRUCT_PAGE_INIT:
early_pfn_to_nid()is called by few,
boot time didn't change.By the way, this machine has 190GB RAM.
--
Sincerely yours,
Mike.--
Sincerely yours,
Mike.