Skip to contentSkip to conversation

Fixes "mm/sparsemem: support sub-section hotplug"

PATCH In review Wei Yang 5 participants 24 messages Feb 6, 2020 – Feb 10, 2020

Subsystems (2)
lore
Share & tools
lore

Wei Yang

author
#0 lore raw

cover Fixes "mm/sparsemem: support sub-section hotplug"

During reviewing David's code, I found current sub-section hotplug is broken in several places.

The following link fix one potential issue, while David and I still spot some suspicious points.

https://lkml.org/lkml/2020/2/6/334

The problem here is mainly about the memmap manipulation:

* memmap would be overwrite if not use SPARSEMEM_VMEMMAP
* only need to adjust memmap for SPARSEMEM_VMEMMAP

Wei Yang (3):
mm/sparsemem: adjust memmap only for SPARSEMEM_VMEMMAP mm/sparsemem: get physical address to page struct instead of virtual
address to pfn
mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP

 mm/sparse.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/3 mm/sparsemem: adjust memmap only for SPARSEMEM_VMEMMAP

Only when SPARSEMEM_VMEMMAP is set, memmap returned from section_activate() points to sub-section page struct. Otherwise, memmap already points to the whole section page struct.

This means only for SPARSEMEM_VMEMMAP, we need to adjust memmap for sub-section case.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/sparse.c +2 -1
index 586d85662978..b5da121bdd6e 100644
@@ -886,7 +886,8 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
 	section_mark_present(ms);
 
 	/* Align memmap to section boundary in the subsection case */
-	if (section_nr_to_pfn(section_nr) != start_pfn)
+	if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
+		section_nr_to_pfn(section_nr) != start_pfn)
 		memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
 	sparse_init_one_section(ms, section_nr, memmap, ms->usage, 0);
 
Signature · 1 line
2.17.1
#2 lore raw

Re: [PATCH 1/3] mm/sparsemem: adjust memmap only for SPARSEMEM_VMEMMAP

Quoted 20 lines · Wei Yang

Only when SPARSEMEM_VMEMMAP is set, memmap returned from section_activate() points to sub-section page struct. Otherwise, memmap already points to the whole section page struct.

This means only for SPARSEMEM_VMEMMAP, we need to adjust memmap for sub-section case.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/sparse.c +0 -0
index 586d85662978..b5da121bdd6e 100644

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 890

Quoted 6 lines
mm/sparse.c +2 -1
@@ -886,7 +886,8 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        section_mark_present(ms);
 
        /* Align memmap to section boundary in the subsection case */
-       if (section_nr_to_pfn(section_nr) != start_pfn)
+       if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
+               section_nr_to_pfn(section_nr) != start_pfn)

Aren't we assured that start_pfn is always section aligned in the SPARSEMEM case? That's the role of check_pfn_span(). Does the change have a runtime impact or is this just theoretical?

#3 lore raw

Re: [PATCH 1/3] mm/sparsemem: adjust memmap only for SPARSEMEM_VMEMMAP

Quoted 23 lines · Dan Williams
Quoted 22 lines · Wei Yang

Only when SPARSEMEM_VMEMMAP is set, memmap returned from section_activate() points to sub-section page struct. Otherwise, memmap already points to the whole section page struct.

This means only for SPARSEMEM_VMEMMAP, we need to adjust memmap for sub-section case.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/sparse.c +0 -0
index 586d85662978..b5da121bdd6e 100644
@@ -886,7 +886,8 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        section_mark_present(ms);
 
Quoted 8 lines

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 890

Quoted 4 lines
mm/sparse.c +2 -1
        /* Align memmap to section boundary in the subsection case */
-       if (section_nr_to_pfn(section_nr) != start_pfn)
+       if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
+               section_nr_to_pfn(section_nr) != start_pfn)

Aren't we assured that start_pfn is always section aligned in the SPARSEMEM case? That's the role of check_pfn_span(). Does the change have a runtime impact or is this just theoretical?

You are right, I missed this point.

Thanks

Signature · 2 lines
Wei Yang
Help you, Help me
author
#4 lore raw

PATCH 2/3 mm/sparsemem: get physical address to page struct instead of virtual address to pfn

In reply to Wei Yang

memmap should be the physical address to page struct instead of virtual address to pfn.

Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
 	/* Align memmap to section boundary in the subsection case */
 	if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
 		section_nr_to_pfn(section_nr) != start_pfn)
-		memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+		memmap = pfn_to_page(section_nr_to_pfn(section_nr));
 	sparse_init_one_section(ms, section_nr, memmap, ms->usage, 0);
 
 	return 0;
Signature · 1 line
2.17.1
#5 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Quoted 19 lines · Wei Yang

memmap should be the physical address to page struct instead of virtual address to pfn.

Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/sparse.c +0 -0
index b5da121bdd6e..56816f653588 100644

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 5 lines
mm/sparse.c +1 -1
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

#6 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Hi Dan,

Quoted 17 lines · Dan Williams

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines · Wei Yang
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into mem_section, and get memmap from the related mem_section to free it in section_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.

By the way, sub-section support is only valid in vmemmap case, right? Seems yes from code, but I don't find any document to prove it.

Thanks
Baoquan

#7 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Quoted 22 lines · Baoquan He

Hi Dan,

Quoted 17 lines · Dan Williams

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines · Wei Yang
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

Quoted 5 lines

The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into mem_section, and get memmap from the related mem_section to free it in section_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.

Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?

Quoted 1 line

By the way, sub-section support is only valid in vmemmap case, right?

Yes.

Quoted 1 line

Seems yes from code, but I don't find any document to prove it.

check_pfn_span() enforces this requirement.

#8 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Quoted 23 lines · Dan Williams
Quoted 22 lines · Baoquan He

Hi Dan,

Quoted 17 lines · Dan Williams

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines · Wei Yang
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

Quoted 7 lines
Quoted 5 lines

The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into mem_section, and get memmap from the related mem_section to free it in section_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.

Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?

I think no, the lost memmap should only happen in vmemmap case.

Quoted 8 lines
Quoted 2 lines

By the way, sub-section support is only valid in vmemmap case, right?

Yes.

Quoted 1 line

Seems yes from code, but I don't find any document to prove it.

check_pfn_span() enforces this requirement.

Thanks for your confirmation. Do you mind if I add some document sentences somewhere make clear this?

#9 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Quoted 38 lines · Baoquan He
Quoted 23 lines · Dan Williams
Quoted 22 lines · Baoquan He

Hi Dan,

Quoted 17 lines · Dan Williams

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines · Wei Yang
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

Quoted 7 lines
Quoted 5 lines

The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into mem_section, and get memmap from the related mem_section to free it in section_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.

Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?

I think no, the lost memmap should only happen in vmemmap case.

Quoted 3 lines
Quoted 2 lines

By the way, sub-section support is only valid in vmemmap case, right?

Quoted 9 lines
Quoted 5 lines

Yes.

Quoted 1 line

Seems yes from code, but I don't find any document to prove it.

check_pfn_span() enforces this requirement.

Thanks for your confirmation. Do you mind if I add some document sentences somewhere make clear this?

Sure, I'd be happy to review as well.

#10 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

In reply to Baoquan He
Quoted 30 lines · Baoquan He
Quoted 29 lines · Dan Williams
Quoted 27 lines · Baoquan He

Hi Dan,

Quoted 17 lines · Dan Williams

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines · Wei Yang
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into mem_section, and get memmap from the related mem_section to free it in section_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.

Quoted 12 lines
Quoted 1 line

Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?

I think no, the lost memmap should only happen in vmemmap case.

Quoted 8 lines
Quoted 2 lines

By the way, sub-section support is only valid in vmemmap case, right?

Yes.

Quoted 1 line

Seems yes from code, but I don't find any document to prove it.

check_pfn_span() enforces this requirement.

I saw this function, but those combination of vmemmap and !vmemmap make my brain not work properly.

Quoted 3 lines

Thanks for your confirmation. Do you mind if I add some document sentences somewhere make clear this?

Thanks, hope this would help the future audience.

Signature · 2 lines
Wei Yang
Help you, Help me
#11 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

In reply to Dan Williams
Quoted 29 lines · Dan Williams
Quoted 22 lines · Baoquan He

Hi Dan,

Quoted 17 lines · Dan Williams

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines · Wei Yang
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

Quoted 5 lines

The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into mem_section, and get memmap from the related mem_section to free it in section_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.

Quoted 5 lines

Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?

Quoted 1 line

By the way, sub-section support is only valid in vmemmap case, right?

Yes.

Just one question from curiosity. Why we don't want sub-section for !vmemmap case? Because it will wast memory for memmap?

Quoted 4 lines
Quoted 2 lines

Seems yes from code, but I don't find any document to prove it.

check_pfn_span() enforces this requirement.

Signature · 2 lines
Wei Yang
Help you, Help me
#12 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Quoted 31 lines · Wei Yang
Quoted 29 lines · Dan Williams
Quoted 27 lines · Baoquan He

Hi Dan,

Quoted 17 lines · Dan Williams

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines · Wei Yang
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

The practical effects is that the memmap for the first unaligned section will be lost when destroy namespace to hot remove it. Because we encode the ->section_mem_map into mem_section, and get memmap from the related mem_section to free it in section_deactivate(). In fact in vmemmap, we don't need to encode the ->section_mem_map with memmap.

Quoted 8 lines
Quoted 5 lines

Right, but can you actually trigger that in the SPARSEMEM_VMEMMAP=n case?

Quoted 1 line

By the way, sub-section support is only valid in vmemmap case, right?

Yes.

Just one question from curiosity. Why we don't want sub-section for !vmemmap case? Because it will wast memory for memmap?

The effort and maintenance burden outweighs the benefit.

#13 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

In reply to Dan Williams
Quoted 15 lines · Dan Williams
Quoted 14 lines · Wei Yang

memmap should be the physical address to page struct instead of virtual address to pfn.

Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

To summarize:

* vmemmap, ->section_mem_map is not used mostly
* !vmemmap, we are sure range is section aligned

Sounds we don't need to handle this?

Signature · 2 lines
Wei Yang
Help you, Help me
#14 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

In reply to Dan Williams
Quoted 15 lines · Dan Williams
Quoted 14 lines · Wei Yang

memmap should be the physical address to page struct instead of virtual address to pfn.

Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

I took another look into the code. Looks there is no practical effect after this. Because in the vmemmap case, we don't need ->section_mem_map to retrieve the real memmap.

But leave a inconsistent data in section_mem_map is not a good practice.

Signature · 2 lines
Wei Yang
Help you, Help me
#15 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Quoted 33 lines · Wei Yang
Quoted 15 lines · Dan Williams
Quoted 14 lines · Wei Yang

memmap should be the physical address to page struct instead of virtual address to pfn.

Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
        /* Align memmap to section boundary in the subsection case */
        if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
                section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

Quoted 5 lines

I took another look into the code. Looks there is no practical effect after this. Because in the vmemmap case, we don't need ->section_mem_map to retrieve the real memmap.

But leave a inconsistent data in section_mem_map is not a good practice.

Yeah, it does has no pratical effect. I tried to create sub-section alighed namespace, then trigger crash, makedumpfile isn't impacted. Because pmem memory is only added, but not onlined. We don't report it to kdump, makedumpfile will ignore it.

I think it's worth fixing it to encode a correct memmap address. We don't know if in the future this will break anything.

Thanks
Baoquan

#16 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Quoted 44 lines

Am 09.02.2020 um 14:50 schrieb Baoquan He <[email protected]>:

Quoted 33 lines · Wei Yang
Quoted 16 lines · Dan Williams
Quoted 14 lines · Wei Yang

memmap should be the physical address to page struct instead of virtual address to pfn.

Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>

---
mm/sparse.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
       /* Align memmap to section boundary in the subsection case */
       if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
               section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

Quoted 5 lines

I took another look into the code. Looks there is no practical effect after this. Because in the vmemmap case, we don't need ->section_mem_map to retrieve the real memmap.

But leave a inconsistent data in section_mem_map is not a good practice.

Quoted 7 lines

Yeah, it does has no pratical effect. I tried to create sub-section alighed namespace, then trigger crash, makedumpfile isn't impacted. Because pmem memory is only added, but not onlined. We don't report it to kdump, makedumpfile will ignore it.

I think it's worth fixing it to encode a correct memmap address. We don't know if in the future this will break anything.

We can have system memory and devmem overlap within a section (which is still buggy and to be fixed in other regard - e.g., pfn_to_online_page() does not work correctly).

E.g., 64 mb of (boot) system memory in a section. Then you can hot-add devmem that spans the remaining 64 mb of that section.

So some of that memory will be kdumped - and should be fixed if broken.

Cheers

Quoted 3 lines

Thanks
Baoquan

author
#17 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Quoted 52 lines · David Hildenbrand
Quoted 44 lines

Am 09.02.2020 um 14:50 schrieb Baoquan He <[email protected]>:

Quoted 33 lines · Wei Yang
Quoted 16 lines · Dan Williams
Quoted 14 lines · Wei Yang

memmap should be the physical address to page struct instead of virtual address to pfn.

Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>

---
mm/sparse.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 10 lines
mm/sparse.c +1 -1
index b5da121bdd6e..56816f653588 100644
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
       /* Align memmap to section boundary in the subsection case */
       if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
               section_nr_to_pfn(section_nr) != start_pfn)
-               memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+               memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Yes, this looks obviously correct. This might be tripping up makedumpfile. Do you see any practical effects of this bug? The kernel mostly avoids ->section_mem_map in the vmemmap case and in the !vmemmap case section_nr_to_pfn(section_nr) should always equal start_pfn.

Quoted 5 lines

I took another look into the code. Looks there is no practical effect after this. Because in the vmemmap case, we don't need ->section_mem_map to retrieve the real memmap.

But leave a inconsistent data in section_mem_map is not a good practice.

Quoted 7 lines

Yeah, it does has no pratical effect. I tried to create sub-section alighed namespace, then trigger crash, makedumpfile isn't impacted. Because pmem memory is only added, but not onlined. We don't report it to kdump, makedumpfile will ignore it.

I think it's worth fixing it to encode a correct memmap address. We don't know if in the future this will break anything.

Quoted 7 lines

We can have system memory and devmem overlap within a section (which is still buggy and to be fixed in other regard - e.g., pfn_to_online_page() does not work correctly).

E.g., 64 mb of (boot) system memory in a section. Then you can hot-add devmem that spans the remaining 64 mb of that section.

So some of that memory will be kdumped - and should be fixed if broken.

Cheers

Thanks for the explanation, I will add this in the changelog.

Quoted 5 lines
Quoted 5 lines

Thanks
Baoquan

Signature · 2 lines
Wei Yang
Help you, Help me
#18 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

In reply to Wei Yang
Quoted 2 lines · Wei Yang

memmap should be the physical address to page struct instead of virtual address to pfn.

Maybe not, memmap stores a virtual address.

Quoted 16 lines

Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/sparse.c +0 -0
index b5da121bdd6e..56816f653588 100644

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 5 lines
mm/sparse.c +1 -1
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
 	/* Align memmap to section boundary in the subsection case */
 	if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
 		section_nr_to_pfn(section_nr) != start_pfn)
-		memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+		memmap = pfn_to_page(section_nr_to_pfn(section_nr));

With Dan's confirmation, sub-section is only valid in vmemmap case. I think the old if (section_nr_to_pfn(section_nr) != start_pfn) is enough to filter out non vmemmap case. So only below code is good:

+ memmap = pfn_to_page(section_nr_to_pfn(section_nr));

Quoted 6 lines
mm/sparse.c +0 -0
 	sparse_init_one_section(ms, section_nr, memmap, ms->usage, 0);
 
 	return 0;
Signature · 1 line
2.17.1
#19 lore raw

Re: [PATCH 2/3] mm/sparsemem: get physical address to page struct instead of virtual address to pfn

Quoted 33 lines · Baoquan He
Quoted 2 lines · Wei Yang

memmap should be the physical address to page struct instead of virtual address to pfn.

Maybe not, memmap stores a virtual address.

Quoted 16 lines

Since we call this only for SPARSEMEM_VMEMMAP, pfn_to_page() is valid at this point.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/sparse.c +0 -0
index b5da121bdd6e..56816f653588 100644

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 5 lines
mm/sparse.c +1 -1
@@ -888,7 +888,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
 	/* Align memmap to section boundary in the subsection case */
 	if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
 		section_nr_to_pfn(section_nr) != start_pfn)
-		memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+		memmap = pfn_to_page(section_nr_to_pfn(section_nr));

With Dan's confirmation, sub-section is only valid in vmemmap case. I think the old if (section_nr_to_pfn(section_nr) != start_pfn) is enough to filter out non vmemmap case. So only below code is good:

mm/sparse.c int __meminit sparse_add_section(int nid, unsigned long start_pfn, · line 891

Quoted 1 line
mm/sparse.c +1 -0
+		memmap = pfn_to_page(section_nr_to_pfn(section_nr));

You mean replace pfn_to_kaddr with pfn_to_page ?

Quoted 6 lines
Quoted 6 lines
mm/sparse.c +0 -0
 	sparse_init_one_section(ms, section_nr, memmap, ms->usage, 0);
 
 	return 0;
Signature · 1 line
2.17.1
Signature · 2 lines
Wei Yang
Help you, Help me
author
#20 lore raw

PATCH 3/3 mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP

In reply to Wei Yang

In case of SPARSEMEM, populate_section_memmap() would allocate memmap for the whole section, even we just want a sub-section. This would lead to memmap overwrite if we a sub-section to an already populated section.

Just return the populated memmap for non-SPARSEMEM_VMEMMAP case.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
mm/sparse.c +10 -0
index 56816f653588..c75ca40db513 100644
@@ -836,6 +836,16 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn,
 	if (nr_pages < PAGES_PER_SECTION && early_section(ms))
 		return pfn_to_page(pfn);
 
+	/*
+	 * If it is not SPARSEMEM_VMEMMAP, we always populate memmap for the
+	 * whole section, even for a sub-section.
+	 *
+	 * Return its memmap if already populated to avoid memmap overwrite.
+	 */
+	if (!IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
+		valid_section(ms))
+		return __section_mem_map_addr(ms);
+
 	memmap = populate_section_memmap(pfn, nr_pages, nid, altmap);
 	if (!memmap) {
 		section_deactivate(pfn, nr_pages, altmap);
Signature · 1 line
2.17.1
#21 lore raw

Re: [PATCH 3/3] mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP

Quoted 21 lines · Wei Yang

In case of SPARSEMEM, populate_section_memmap() would allocate memmap for the whole section, even we just want a sub-section. This would lead to memmap overwrite if we a sub-section to an already populated section.

Just return the populated memmap for non-SPARSEMEM_VMEMMAP case.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
mm/sparse.c +0 -0
index 56816f653588..c75ca40db513 100644
@@ -836,6 +836,16 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn,
        if (nr_pages < PAGES_PER_SECTION && early_section(ms))
                return pfn_to_page(pfn);

mm/sparse.c static struct page * __meminit section_activate(int nid, unsigned long pfn, · line 847

Quoted 10 lines
mm/sparse.c +9 -0
 
+       /*
+        * If it is not SPARSEMEM_VMEMMAP, we always populate memmap for the
+        * whole section, even for a sub-section.
+        *
+        * Return its memmap if already populated to avoid memmap overwrite.
+        */
+       if (!IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
+               valid_section(ms))
+               return __section_mem_map_addr(ms);

Again, is check_pfn_span() failing to prevent this path?

#22 lore raw

Re: [PATCH 3/3] mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP

Quoted 23 lines · Dan Williams
Quoted 22 lines · Wei Yang

In case of SPARSEMEM, populate_section_memmap() would allocate memmap for the whole section, even we just want a sub-section. This would lead to memmap overwrite if we a sub-section to an already populated section.

Just return the populated memmap for non-SPARSEMEM_VMEMMAP case.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
mm/sparse.c +0 -0
index 56816f653588..c75ca40db513 100644
@@ -836,6 +836,16 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn,
        if (nr_pages < PAGES_PER_SECTION && early_section(ms))
                return pfn_to_page(pfn);
 
Quoted 11 lines

mm/sparse.c static struct page * __meminit section_activate(int nid, unsigned long pfn, · line 847

Quoted 9 lines
mm/sparse.c +9 -0
+       /*
+        * If it is not SPARSEMEM_VMEMMAP, we always populate memmap for the
+        * whole section, even for a sub-section.
+        *
+        * Return its memmap if already populated to avoid memmap overwrite.
+        */
+       if (!IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
+               valid_section(ms))
+               return __section_mem_map_addr(ms);

Again, is check_pfn_span() failing to prevent this path?

The answer should be yes, this patch is not needed.

Quoted 1 line
#23 lore raw

Re: [PATCH 3/3] mm/sparsemem: avoid memmap overwrite for non-SPARSEMEM_VMEMMAP

In reply to Dan Williams
Quoted 23 lines · Dan Williams
Quoted 22 lines · Wei Yang

In case of SPARSEMEM, populate_section_memmap() would allocate memmap for the whole section, even we just want a sub-section. This would lead to memmap overwrite if we a sub-section to an already populated section.

Just return the populated memmap for non-SPARSEMEM_VMEMMAP case.

Fixes ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
CC Dan Williams <[email protected]>
---
 mm/sparse.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
mm/sparse.c +0 -0
index 56816f653588..c75ca40db513 100644
@@ -836,6 +836,16 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn,
        if (nr_pages < PAGES_PER_SECTION && early_section(ms))
                return pfn_to_page(pfn);
 
Quoted 11 lines

mm/sparse.c static struct page * __meminit section_activate(int nid, unsigned long pfn, · line 847

Quoted 9 lines
mm/sparse.c +9 -0
+       /*
+        * If it is not SPARSEMEM_VMEMMAP, we always populate memmap for the
+        * whole section, even for a sub-section.
+        *
+        * Return its memmap if already populated to avoid memmap overwrite.
+        */
+       if (!IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP) &&
+               valid_section(ms))
+               return __section_mem_map_addr(ms);

Again, is check_pfn_span() failing to prevent this path?

Oh, you are right. Thanks

Signature · 2 lines
Wei Yang
Help you, Help me