Fixes "mm/sparsemem: support sub-section hotplug"
Subsystems (2)
cover Fixes "mm/sparsemem: support sub-section hotplug"
During reviewing David's code, I found current sub-section hotplug is broken in several places.
The following link fix one potential issue, while David and I still spot some suspicious points.
https://lkml.org/lkml/2020/2/6/334
The problem here is mainly about the memmap manipulation:
* memmap would be overwrite if not use SPARSEMEM_VMEMMAP
* only need to adjust memmap for SPARSEMEM_VMEMMAP
Wei Yang (3):
mm/sparsemem: adjust memmap only for SPARSEMEM_VMEMMAP mm/sparsemem: get physical address to page struct instead of virtual
address to pfn
mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP
mm/sparse.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)
Signature · 1 line
2.17.1
PATCH 1/3 mm/sparsemem: adjust memmap only for SPARSEMEM_VMEMMAP
Only when SPARSEMEM_VMEMMAP is set, memmap returned from section_activate() points to sub-section page struct. Otherwise, memmap already points to the whole section page struct.
This means only for SPARSEMEM_VMEMMAP, we need to adjust memmap for sub-section case.
--- mm/sparse.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mm/sparse.c +2 -1
@@ -886,7 +886,8 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,section_mark_present (ms);/* Align memmap to section boundary in the subsection case */ -if (section_nr_to_pfn (section_nr) != start_pfn) +if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) && +section_nr_to_pfn (section_nr) != start_pfn) memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr));sparse_init_one_section (ms, section_nr, memmap, ms->usage,0 );
Signature · 1 line
2.17.1
Re: [PATCH 1/3] mm/sparsemem: adjust memmap only for SPARSEMEM_VMEMMAP
Quoted 20 lines · Wei Yang
Only when
SPARSEMEM_VMEMMAPis set, memmap returned fromsection_activate()points to sub-section page struct. Otherwise, memmap already points to the whole section page struct.This means only for
SPARSEMEM_VMEMMAP, we need to adjust memmap for sub-section case.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/sparse.c +0
-0
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 890
Quoted 6 lines
mm/sparse.c +2
-1@@ -886,7 +886,8 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,section_mark_present (ms);/* Align memmap to section boundary in the subsection case */ -if (section_nr_to_pfn (section_nr) != start_pfn) +if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) && +section_nr_to_pfn (section_nr) != start_pfn)
Aren't we assured that start_pfn is always section aligned in the SPARSEMEM case? That's the role of check_pfn_span(). Does the change have a runtime impact or is this just theoretical?
Re: [PATCH 1/3] mm/sparsemem: adjust memmap only for SPARSEMEM_VMEMMAP
Quoted 23 lines · Dan Williams
Quoted 22 lines · Wei Yang
Only when
SPARSEMEM_VMEMMAPis set, memmap returned fromsection_activate()points to sub-section page struct. Otherwise, memmap already points to the whole section page struct.This means only for
SPARSEMEM_VMEMMAP, we need to adjust memmap for sub-section case.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)mm/sparse.c +0
-0@@ -886,7 +886,8 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,section_mark_present (ms);
Quoted 8 lines
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 890
Quoted 4 lines
mm/sparse.c +2
-1/* Align memmap to section boundary in the subsection case */ -if (section_nr_to_pfn (section_nr) != start_pfn) +if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) && +section_nr_to_pfn (section_nr) != start_pfn)Aren't we assured that
start_pfnis always section aligned in the SPARSEMEM case? That's the role ofcheck_pfn_span(). Does the change have a runtime impact or is this just theoretical?
You are right, I missed this point.
Thanks
Signature · 2 lines
Wei Yang Help you, Help me
PATCH 2/3 mm/sparsemem: get physical address to page struct instead of virtual address to pfn
In reply to Wei Yangmemmap should be the physical address to page struct instead of virtual address to pfn.
Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.
--- mm/sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/sparse.c +1 -1
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));sparse_init_one_section (ms, section_nr, memmap, ms->usage,0 );return 0 ;
Signature · 1 line
2.17.1
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Quoted 19 lines · Wei Yang
memmap should be the physical address to page struct instead of virtual address to pfn.
Since we call this only for
SPARSEMEM_VMEMMAP,pfn_to_page()is valid at this point.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/sparse.c +0
-0
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 5 lines
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));
Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Hi Dan,
Quoted 17 lines · Dan Williams
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines · Wei Yang
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.
The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into mem_section, and get memmap from the related mem_section to free it in section_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.
By the way, sub-section support is only valid in vmemmap case, right? Seems yes from code, but I don't find any document to prove it.
Thanks
Baoquan
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Quoted 22 lines · Baoquan He
Hi Dan,
Quoted 17 lines · Dan Williams
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines · Wei Yang
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.
Quoted 5 lines
The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into
mem_section, and get memmap from the relatedmem_sectionto free it insection_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.
Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?
Quoted 1 line
By the way, sub-section support is only valid in vmemmap case, right?
Yes.
Quoted 1 line
Seems yes from code, but I don't find any document to prove it.
check_pfn_span() enforces this requirement.
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Quoted 23 lines · Dan Williams
Quoted 22 lines · Baoquan He
Hi Dan,
Quoted 17 lines · Dan Williams
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines · Wei Yang
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.
Quoted 7 lines
Quoted 5 lines
The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into
mem_section, and get memmap from the relatedmem_sectionto free it insection_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?
I think no, the lost memmap should only happen in vmemmap case.
Quoted 8 lines
Quoted 2 lines
By the way, sub-section support is only valid in vmemmap case, right?
Yes.
Quoted 1 line
Seems yes from code, but I don't find any document to prove it.
check_pfn_span()enforces this requirement.
Thanks for your confirmation. Do you mind if I add some document sentences somewhere make clear this?
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Quoted 38 lines · Baoquan He
Quoted 23 lines · Dan Williams
Quoted 22 lines · Baoquan He
Hi Dan,
Quoted 17 lines · Dan Williams
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines · Wei Yang
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.Quoted 7 lines
Quoted 5 lines
The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into
mem_section, and get memmap from the relatedmem_sectionto free it insection_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?
I think no, the lost memmap should only happen in vmemmap case.
Quoted 3 lines
Quoted 2 lines
By the way, sub-section support is only valid in vmemmap case, right?
Quoted 9 lines
Quoted 5 lines
Yes.
Quoted 1 line
Seems yes from code, but I don't find any document to prove it.
check_pfn_span()enforces this requirement.Thanks for your confirmation. Do you mind if I add some document sentences somewhere make clear this?
Sure, I'd be happy to review as well.
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
In reply to Baoquan HeQuoted 30 lines · Baoquan He
Quoted 29 lines · Dan Williams
Quoted 27 lines · Baoquan He
Hi Dan,
Quoted 17 lines · Dan Williams
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines · Wei Yang
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into
mem_section, and get memmap from the relatedmem_sectionto free it insection_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.
Quoted 12 lines
Quoted 1 line
Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?
I think no, the lost memmap should only happen in vmemmap case.
Quoted 8 lines
Quoted 2 lines
By the way, sub-section support is only valid in vmemmap case, right?
Yes.
Quoted 1 line
Seems yes from code, but I don't find any document to prove it.
check_pfn_span()enforces this requirement.
I saw this function, but those combination of vmemmap and !vmemmap make my brain not work properly.
Quoted 3 lines
Thanks for your confirmation. Do you mind if I add some document sentences somewhere make clear this?
Thanks, hope this would help the future audience.
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
In reply to Dan WilliamsQuoted 29 lines · Dan Williams
Quoted 22 lines · Baoquan He
Hi Dan,
Quoted 17 lines · Dan Williams
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines · Wei Yang
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.Quoted 5 lines
The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into
mem_section, and get memmap from the relatedmem_sectionto free it insection_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.
Quoted 5 lines
Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?
Quoted 1 line
By the way, sub-section support is only valid in vmemmap case, right?
Yes.
Just one question from curiosity. Why we don't want sub-section for !vmemmap case? Because it will wast memory for memmap?
Quoted 4 lines
Quoted 2 lines
Seems yes from code, but I don't find any document to prove it.
check_pfn_span()enforces this requirement.
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Quoted 31 lines · Wei Yang
Quoted 29 lines · Dan Williams
Quoted 27 lines · Baoquan He
Hi Dan,
Quoted 17 lines · Dan Williams
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines · Wei Yang
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into
mem_section, and get memmap from the relatedmem_sectionto free it insection_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.
Quoted 8 lines
Quoted 5 lines
Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?
Quoted 1 line
By the way, sub-section support is only valid in vmemmap case, right?
Yes.
Just one question from curiosity. Why we don't want sub-section for !vmemmap case? Because it will wast memory for memmap?
The effort and maintenance burden outweighs the benefit.
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
In reply to Dan WilliamsQuoted 15 lines · Dan Williams
Quoted 14 lines · Wei Yang
memmap should be the physical address to page struct instead of virtual address to pfn.
Since we call this only for
SPARSEMEM_VMEMMAP,pfn_to_page()is valid at this point.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.
To summarize:
* vmemmap, ->section_mem_map is not used mostly
* !vmemmap, we are sure range is section aligned
Sounds we don't need to handle this?
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
In reply to Dan WilliamsQuoted 15 lines · Dan Williams
Quoted 14 lines · Wei Yang
memmap should be the physical address to page struct instead of virtual address to pfn.
Since we call this only for
SPARSEMEM_VMEMMAP,pfn_to_page()is valid at this point.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.
I took another look into the code. Looks there is no practical effect after this. Because in the vmemmap case, we don't need ->section_mem_map to retrieve the real memmap.
But leave a inconsistent data in section_mem_map is not a good practice.
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Quoted 33 lines · Wei Yang
Quoted 15 lines · Dan Williams
Quoted 14 lines · Wei Yang
memmap should be the physical address to page struct instead of virtual address to pfn.
Since we call this only for
SPARSEMEM_VMEMMAP,pfn_to_page()is valid at this point.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 16 lines
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.
Quoted 5 lines
I took another look into the code. Looks there is no practical effect after this. Because in the vmemmap case, we don't need ->section_mem_map to retrieve the real memmap.
But leave a inconsistent data in
section_mem_mapis not a good practice.
Yeah, it does has no pratical effect. I tried to create sub-section alighed namespace, then trigger crash, makedumpfile isn't impacted. Because pmem memory is only added, but not onlined. We don't report it to kdump, makedumpfile will ignore it.
I think it's worth fixing it to encode a correct memmap address. We don't know if in the future this will break anything.
Thanks
Baoquan
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Quoted 44 lines
Am 09.02.2020 um 14:50 schrieb Baoquan He <[email protected]>:
Quoted 33 lines · Wei Yang
Quoted 16 lines · Dan Williams
Quoted 14 lines · Wei Yang
memmap should be the physical address to page struct instead of virtual address to pfn.
Since we call this only for
SPARSEMEM_VMEMMAP,pfn_to_page()is valid at this point.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>---
mm/sparse.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)Quoted 16 lines
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.Quoted 5 lines
I took another look into the code. Looks there is no practical effect after this. Because in the vmemmap case, we don't need ->section_mem_map to retrieve the real memmap.
But leave a inconsistent data in
section_mem_mapis not a good practice.
Quoted 7 lines
Yeah, it does has no pratical effect. I tried to create sub-section alighed namespace, then trigger crash, makedumpfile isn't impacted. Because pmem memory is only added, but not onlined. We don't report it to kdump, makedumpfile will ignore it.
I think it's worth fixing it to encode a correct memmap address. We don't know if in the future this will break anything.
We can have system memory and devmem overlap within a section (which is still buggy and to be fixed in other regard - e.g., pfn_to_online_page() does not work correctly).
E.g., 64 mb of (boot) system memory in a section. Then you can hot-add devmem that spans the remaining 64 mb of that section.
So some of that memory will be kdumped - and should be fixed if broken.
Cheers
Quoted 3 lines
Thanks
Baoquan
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Quoted 52 lines · David Hildenbrand
Quoted 44 lines
Am 09.02.2020 um 14:50 schrieb Baoquan He <[email protected]>:
Quoted 33 lines · Wei Yang
Quoted 16 lines · Dan Williams
Quoted 14 lines · Wei Yang
memmap should be the physical address to page struct instead of virtual address to pfn.
Since we call this only for
SPARSEMEM_VMEMMAP,pfn_to_page()is valid at this point.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>---
mm/sparse.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)Quoted 16 lines
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 10 lines
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case
section_nr_to_pfn(section_nr)should always equalstart_pfn.Quoted 5 lines
I took another look into the code. Looks there is no practical effect after this. Because in the vmemmap case, we don't need ->section_mem_map to retrieve the real memmap.
But leave a inconsistent data in
section_mem_mapis not a good practice.Quoted 7 lines
Yeah, it does has no pratical effect. I tried to create sub-section alighed namespace, then trigger crash, makedumpfile isn't impacted. Because pmem memory is only added, but not onlined. We don't report it to kdump, makedumpfile will ignore it.
I think it's worth fixing it to encode a correct memmap address. We don't know if in the future this will break anything.
Quoted 7 lines
We can have system memory and devmem overlap within a section (which is still buggy and to be fixed in other regard - e.g.,
pfn_to_online_page()does not work correctly).E.g., 64 mb of (boot) system memory in a section. Then you can hot-add devmem that spans the remaining 64 mb of that section.
So some of that memory will be kdumped - and should be fixed if broken.
Cheers
Thanks for the explanation, I will add this in the changelog.
Quoted 5 lines
Quoted 5 lines
Thanks
Baoquan
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
In reply to Wei YangQuoted 2 lines · Wei Yang
memmap should be the physical address to page struct instead of virtual address to pfn.
Maybe not, memmap stores a virtual address.
Quoted 16 lines
Since we call this only for
SPARSEMEM_VMEMMAP,pfn_to_page()is valid at this point.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/sparse.c +0
-0
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 5 lines
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));
With Dan's confirmation, sub-section is only valid in vmemmap case. I think the old if (section_nr_to_pfn(section_nr) != start_pfn) is enough to filter out non vmemmap case. So only below code is good:
+ memmap = pfn_to_page(section_nr_to_pfn(section_nr));
Quoted 6 lines
mm/sparse.c +0
-0sparse_init_one_section (ms, section_nr, memmap, ms->usage,0 );return 0 ;Signature · 1 line
2.17.1
Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn
Quoted 33 lines · Baoquan He
Quoted 2 lines · Wei Yang
memmap should be the physical address to page struct instead of virtual address to pfn.
Maybe not, memmap stores a virtual address.
Quoted 16 lines
Since we call this only for
SPARSEMEM_VMEMMAP,pfn_to_page()is valid at this point.Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/sparse.c +0
-0mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 5 lines
mm/sparse.c +1
-1@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,/* Align memmap to section boundary in the subsection case */ if (IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) &§ion_nr_to_pfn (section_nr) != start_pfn) - memmap =pfn_to_kaddr (section_nr_to_pfn (section_nr)); + memmap =pfn_to_page (section_nr_to_pfn (section_nr));With Dan's confirmation, sub-section is only valid in vmemmap case. I think the old if (
section_nr_to_pfn(section_nr)!=start_pfn) is enough to filter out non vmemmap case. So only below code is good:
mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891
Quoted 1 line
mm/sparse.c +1
-0+ memmap =pfn_to_page (section_nr_to_pfn (section_nr));
You mean replace pfn_to_kaddr with pfn_to_page ?
Quoted 6 lines
Quoted 6 lines
mm/sparse.c +0
-0sparse_init_one_section (ms, section_nr, memmap, ms->usage,0 );return 0 ;Signature · 1 line
2.17.1
Signature · 2 lines
Wei Yang Help you, Help me
PATCH 3/3 mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP
In reply to Wei YangIn case of SPARSEMEM, populate_section_memmap() would allocate memmap for the whole section, even we just want a sub-section. This would lead to memmap overwrite if we a sub-section to an already populated section.
Just return the populated memmap for non-SPARSEMEM_VMEMMAP case.
--- mm/sparse.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
mm/sparse.c +10 -0
@@ -836,6 +836,16 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn,if (nr_pages <PAGES_PER_SECTION &&early_section (ms))return pfn_to_page (pfn); +/* +* If it is not SPARSEMEM_VMEMMAP, we always populate memmap for the +* whole section, even for a sub-section. +* +* Return its memmap if already populated to avoid memmap overwrite. +*/ +if (!IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) && +valid_section (ms)) +return __section_mem_map_addr (ms); + memmap =populate_section_memmap (pfn, nr_pages, nid, altmap);if (!memmap) {section_deactivate (pfn, nr_pages, altmap);
Signature · 1 line
2.17.1
Re: [PATCH 3/3] mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP
Quoted 21 lines · Wei Yang
In case of SPARSEMEM,
populate_section_memmap()would allocate memmap for the whole section, even we just want a sub-section. This would lead to memmap overwrite if we a sub-section to an already populated section.Just return the populated memmap for non-SPARSEMEM_VMEMMAP case.
Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 10 ++++++++++ 1 file changed, 10 insertions(+)mm/sparse.c +0
-0@@ -836,6 +836,16 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn,if (nr_pages <PAGES_PER_SECTION &&early_section (ms))return pfn_to_page (pfn);
mm/sparse.c static struct page * __meminit section_activate(int nid, unsigned long pfn, · line 847
Quoted 10 lines
mm/sparse.c +9
-0+/* +* If it is not SPARSEMEM_VMEMMAP, we always populate memmap for the +* whole section, even for a sub-section. +* +* Return its memmap if already populated to avoid memmap overwrite. +*/ +if (!IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) && +valid_section (ms)) +return __section_mem_map_addr (ms);
Again, is check_pfn_span() failing to prevent this path?
Re: [PATCH 3/3] mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP
Quoted 23 lines · Dan Williams
Quoted 22 lines · Wei Yang
In case of SPARSEMEM,
populate_section_memmap()would allocate memmap for the whole section, even we just want a sub-section. This would lead to memmap overwrite if we a sub-section to an already populated section.Just return the populated memmap for non-SPARSEMEM_VMEMMAP case.
Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 10 ++++++++++ 1 file changed, 10 insertions(+)mm/sparse.c +0
-0@@ -836,6 +836,16 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn,if (nr_pages <PAGES_PER_SECTION &&early_section (ms))return pfn_to_page (pfn);
Quoted 11 lines
mm/sparse.c static struct page * __meminit section_activate(int nid, unsigned long pfn, · line 847
Quoted 9 lines
mm/sparse.c +9
-0+/* +* If it is not SPARSEMEM_VMEMMAP, we always populate memmap for the +* whole section, even for a sub-section. +* +* Return its memmap if already populated to avoid memmap overwrite. +*/ +if (!IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) && +valid_section (ms)) +return __section_mem_map_addr (ms);Again, is
check_pfn_span()failing to prevent this path?
The answer should be yes, this patch is not needed.
Quoted 1 line
Re: [PATCH 3/3] mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP
In reply to Dan WilliamsQuoted 23 lines · Dan Williams
Quoted 22 lines · Wei Yang
In case of SPARSEMEM,
populate_section_memmap()would allocate memmap for the whole section, even we just want a sub-section. This would lead to memmap overwrite if we a sub-section to an already populated section.Just return the populated memmap for non-SPARSEMEM_VMEMMAP case.
Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")Signed-off-by Wei Yang <[email protected]>CC Dan Williams <[email protected]>--- mm/sparse.c | 10 ++++++++++ 1 file changed, 10 insertions(+)mm/sparse.c +0
-0@@ -836,6 +836,16 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn,if (nr_pages <PAGES_PER_SECTION &&early_section (ms))return pfn_to_page (pfn);
Quoted 11 lines
mm/sparse.c static struct page * __meminit section_activate(int nid, unsigned long pfn, · line 847
Quoted 9 lines
mm/sparse.c +9
-0+/* +* If it is not SPARSEMEM_VMEMMAP, we always populate memmap for the +* whole section, even for a sub-section. +* +* Return its memmap if already populated to avoid memmap overwrite. +*/ +if (!IS_ENABLED (CONFIG_SPARSEMEM_VMEMMAP ) && +valid_section (ms)) +return __section_mem_map_addr (ms);Again, is
check_pfn_span()failing to prevent this path?
Oh, you are right. Thanks
Signature · 2 lines
Wei Yang Help you, Help me