Skip to contentSkip to conversation

mm/rmap: do not add fully unmapped large folio to deferred split list

PATCH Superseded Zi Yan 7 participants 15 messages Apr 26, 2024 – May 2, 2024

Subsystems (3)

v5 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Zi Yan

author
#0 lore raw

PATCH v5 1/1 mm/rmap: do not add fully unmapped large folio to deferred split list

From: Zi Yan <[email protected]>

In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary.

For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue.

However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise.

Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration.

To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too.

Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable().

---
 mm/rmap.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
mm/rmap.c +9 -3
index 2608c40dffad..a9bd64ebdd9a 100644
@@ -1495,6 +1495,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
 {
 	atomic_t *mapped = &folio->_nr_pages_mapped;
 	int last, nr = 0, nr_pmdmapped = 0;
+	bool partially_mapped = false;
 	enum node_stat_item idx;
 
 	__folio_rmap_sanity_checks(folio, page, nr_pages, level);
@@ -1515,6 +1516,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
 					nr++;
 			}
 		} while (page++, --nr_pages > 0);
+
+		partially_mapped = !!nr && !!atomic_read(mapped);
 		break;
 	case RMAP_LEVEL_PMD:
 		atomic_dec(&folio->_large_mapcount);
@@ -1532,6 +1535,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
 				nr = 0;
 			}
 		}
+
+		partially_mapped = nr < nr_pmdmapped;
 		break;
 	}
 
@@ -1553,9 +1558,10 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
 		 * page of the folio is unmapped and at least one page
 		 * is still mapped.
 		 */
-		if (folio_test_large(folio) && folio_test_anon(folio))
-			if (level == RMAP_LEVEL_PTE || nr < nr_pmdmapped)
-				deferred_split_folio(folio);
+		if (folio_test_anon(folio) &&
+		    list_empty(&folio->_deferred_list) &&
+		    partially_mapped)
+			deferred_split_folio(folio);
 	}
 
 	/*

base-commit: 3dba658670af22074cc6f26dc92efe0013ac3359

Signature · 1 line
2.43.0
Reviewed-by on v5 1/1
#1 lore raw
Quoted 56 lines · Zi Yan

From: Zi Yan <[email protected]>

In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary.

For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue.

However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise.

Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration.

To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too.

Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable().

---
  mm/rmap.c | 12 +++++++++---
  1 file changed, 9 insertions(+), 3 deletions(-)
mm/rmap.c +1 -0
index 2608c40dffad..a9bd64ebdd9a 100644
@@ -1495,6 +1495,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
  {
  	atomic_t *mapped = &folio->_nr_pages_mapped;
  	int last, nr = 0, nr_pmdmapped = 0;
+	bool partially_mapped = false;
  	enum node_stat_item idx;
 
  	__folio_rmap_sanity_checks(folio, page, nr_pages, level);

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1520

Quoted 5 lines
mm/rmap.c +2 -0
@@ -1515,6 +1516,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
  					nr++;
  			}
  		} while (page++, --nr_pages > 0);
+
+		partially_mapped = !!nr && !!atomic_read(mapped);

Nit: The && should remove the need for both !!.

Signature · 3 lines
Cheers,

David / dhildenb
#2 lore raw
Quoted 55 lines · David Hildenbrand
Quoted 54 lines · Zi Yan

From: Zi Yan <[email protected]>

In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary.

For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue.

However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise.

Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration.

To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too.

Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable().

---
  mm/rmap.c | 12 +++++++++---
  1 file changed, 9 insertions(+), 3 deletions(-)
mm/rmap.c +1 -0
index 2608c40dffad..a9bd64ebdd9a 100644
@@ -1495,6 +1495,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
  {
  	atomic_t *mapped = &folio->_nr_pages_mapped;
  	int last, nr = 0, nr_pmdmapped = 0;
+	bool partially_mapped = false;
  	enum node_stat_item idx;
   	__folio_rmap_sanity_checks(folio, page, nr_pages, level);
Quoted 8 lines

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1520

Quoted 6 lines
mm/rmap.c +2 -0
@@ -1515,6 +1516,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
  					nr++;
  			}
  		} while (page++, --nr_pages > 0);
+
+		partially_mapped = !!nr && !!atomic_read(mapped);

Nit: The && should remove the need for both !!.

My impression was that !! is needed to convert from int to bool and I do find "!!int && !!int" use in the kernel. If this is unnecessary, Andrew can apply the fixup below. I can send a new version if it is really needed.

mm/rmap.c +1 -1
index a9bd64ebdd9a..c1fd5828409b 100644
@@ -1517,7 +1517,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                        }
                } while (page++, --nr_pages > 0);
 
-               partially_mapped = !!nr && !!atomic_read(mapped);
+               partially_mapped = nr && atomic_read(mapped);
                break;
        case RMAP_LEVEL_PMD:
                atomic_dec(&folio->_large_mapcount);
Quoted 2 lines

Thanks.

--
Best Regards,
Yan, Zi

#3 lore raw
Quoted 66 lines · Zi Yan
Quoted 55 lines · David Hildenbrand
Quoted 54 lines · Zi Yan

From: Zi Yan <[email protected]>

In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary.

For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue.

However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise.

Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration.

To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too.

Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable().

---
   mm/rmap.c | 12 +++++++++---
   1 file changed, 9 insertions(+), 3 deletions(-)
mm/rmap.c +1 -0
index 2608c40dffad..a9bd64ebdd9a 100644
@@ -1495,6 +1495,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
   {
   	atomic_t *mapped = &folio->_nr_pages_mapped;
   	int last, nr = 0, nr_pmdmapped = 0;
+	bool partially_mapped = false;
   	enum node_stat_item idx;
    	__folio_rmap_sanity_checks(folio, page, nr_pages, level);
Quoted 8 lines

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1520

Quoted 6 lines
mm/rmap.c +2 -0
@@ -1515,6 +1516,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
   					nr++;
   			}
   		} while (page++, --nr_pages > 0);
+
+		partially_mapped = !!nr && !!atomic_read(mapped);

Nit: The && should remove the need for both !!.

Quoted 2 lines

My impression was that !! is needed to convert from int to bool and I do find "!!int && !!int" use in the kernel.

I might be wrong about this, but if you wouldn't write

	if (!!nr && !!atomic_read(mapped))

then

bool partially_mapped = nr && atomic_read(mapped);

is sufficient.

&& would make sure that the result is either 0 or 1, which you can store safely in a bool, no matter which underlying type is used to store that value.

But I *think* nowdays, the compiler will always handle that correctly, even without the "&&" (ever since C99 added _Bool).

Likely, also

	bool partially_mapped = nr & atomic_read(mapped);

Would nowadays work, but looks stupid.

Related: https://lkml.org/lkml/2013/8/31/138

---
#include <stdio.h>
#include <stdbool.h>
#include <stdint.h>
#include <inttypes.h>

volatile uint64_t a = 0x8000000000000000ull;

void main (void) {
        printf("uint64_t a = a: 0x%" PRIx64 "\n", a);

        int i1 = a;
        printf("int i1 = a: %d\n", i1);

        int i2 = !!a;
        printf("int i2 = !!a: %d\n", i2);

        bool b1 = a;
        printf("bool b1 = a: %d\n", b1);

        bool b2 = !!a;
        printf("bool b2 = !!a: %d\n", b2);
}
---
$ ./test
uint64_t a = a: 0x8000000000000000
int i1 = a: 0
int i2 = !!a: 1
bool b1 = a: 1
bool b2 = !!a: 1
---

Note that if bool would be defined as "int", you would need the !!, otherwise you would lose information.

But even for b1, the gcc generates now:

 40118c:       48 8b 05 7d 2e 00 00    mov    0x2e7d(%rip),%rax        # 404010 <a>
 401193:       48 85 c0                test   %rax,%rax
 401196:       0f 95 c0                setne  %al

My stdbool.h contains

#define bool _Bool

And I think C99 added _Bool that makes that work.

But I didn't read the standard, and it's time for the weekend :)

Signature · 3 lines
Cheers,

David / dhildenb
#4 lore raw
Quoted 128 lines · David Hildenbrand
Quoted 66 lines · Zi Yan
Quoted 55 lines · David Hildenbrand
Quoted 54 lines · Zi Yan

From: Zi Yan <[email protected]>

In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary.

For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue.

However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise.

Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration.

To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too.

Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable().

---
   mm/rmap.c | 12 +++++++++---
   1 file changed, 9 insertions(+), 3 deletions(-)
mm/rmap.c +1 -0
index 2608c40dffad..a9bd64ebdd9a 100644
@@ -1495,6 +1495,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
   {
   	atomic_t *mapped = &folio->_nr_pages_mapped;
   	int last, nr = 0, nr_pmdmapped = 0;
+	bool partially_mapped = false;
   	enum node_stat_item idx;
    	__folio_rmap_sanity_checks(folio, page, nr_pages, level);
Quoted 8 lines

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1520

Quoted 6 lines
mm/rmap.c +2 -0
@@ -1515,6 +1516,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
   					nr++;
   			}
   		} while (page++, --nr_pages > 0);
+
+		partially_mapped = !!nr && !!atomic_read(mapped);

Nit: The && should remove the need for both !!.

Quoted 2 lines

My impression was that !! is needed to convert from int to bool and I do find "!!int && !!int" use in the kernel.

I might be wrong about this, but if you wouldn't write

	if (!!nr && !!atomic_read(mapped))

then

bool partially_mapped = nr && atomic_read(mapped);

is sufficient.

&& would make sure that the result is either 0 or 1, which you can store safely in a bool, no matter which underlying type is used to store that value.

But I *think* nowdays, the compiler will always handle that correctly, even without the "&&" (ever since C99 added _Bool).

Likely, also

	bool partially_mapped = nr & atomic_read(mapped);

Would nowadays work, but looks stupid.

Related: https://lkml.org/lkml/2013/8/31/138

---
#include <stdio.h>
#include <stdbool.h>
#include <stdint.h>
#include <inttypes.h>

volatile uint64_t a = 0x8000000000000000ull;

void main (void) {
        printf("uint64_t a = a: 0x%" PRIx64 "\n", a);

        int i1 = a;
        printf("int i1 = a: %d\n", i1);

        int i2 = !!a;
        printf("int i2 = !!a: %d\n", i2);

        bool b1 = a;
        printf("bool b1 = a: %d\n", b1);

        bool b2 = !!a;
        printf("bool b2 = !!a: %d\n", b2);
}
---
$ ./test
uint64_t a = a: 0x8000000000000000
int i1 = a: 0
int i2 = !!a: 1
bool b1 = a: 1
bool b2 = !!a: 1
---
Quoted 2 lines

Note that if bool would be defined as "int", you would need the !!, otherwise you would lose information.

Thank you for all above. Really appreciate it! And you are right about && and !!. My memory on !! must be from the old days and now is refreshed. :)

Quoted 8 lines

But even for b1, the gcc generates now:

 40118c:       48 8b 05 7d 2e 00 00    mov    0x2e7d(%rip),%rax        # 404010 <a>
 401193:       48 85 c0                test   %rax,%rax
 401196:       0f 95 c0                setne  %al
Quoted 7 lines

My stdbool.h contains

#define bool _Bool

And I think C99 added _Bool that makes that work.

But I didn't read the standard, and it's time for the weekend :)

Have a good weekend!

--
Best Regards,
Yan, Zi

#5 lore raw
In reply to David Hildenbrand
Quoted 3 lines · David Hildenbrand
Quoted 1 line · Zi Yan

On 26 Apr 2024, at 15:08, David Hildenbrand wrote:

[...]

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ bool partially_mapped = false;

[...]

Quoted 9 lines
Quoted 7 lines
Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+           partially_mapped = !!nr && !!atomic_read(mapped);

Nit: The && should remove the need for both !!.

My impression was that !! is needed to convert from int to bool and I do find "!!int && !!int" use in the kernel.

I might be wrong about this, but if you wouldn't write

I think you're correct.

Quoted 8 lines
        if (!!nr && !!atomic_read(mapped))

then

bool partially_mapped = nr && atomic_read(mapped);

is sufficient.

+1

Quoted 49 lines

&& would make sure that the result is either 0 or 1, which you can store safely in a bool, no matter which underlying type is used to store that value.

But I *think* nowdays, the compiler will always handle that correctly, even without the "&&" (ever since C99 added _Bool).

Likely, also

        bool partially_mapped = nr & atomic_read(mapped);

Would nowadays work, but looks stupid.

Related: https://lkml.org/lkml/2013/8/31/138

---
#include <stdio.h>
#include <stdbool.h>
#include <stdint.h>
#include <inttypes.h>

volatile uint64_t a = 0x8000000000000000ull;

void main (void) {
         printf("uint64_t a = a: 0x%" PRIx64 "\n", a);

         int i1 = a;
         printf("int i1 = a: %d\n", i1);

         int i2 = !!a;
         printf("int i2 = !!a: %d\n", i2);

         bool b1 = a;
         printf("bool b1 = a: %d\n", b1);

         bool b2 = !!a;
         printf("bool b2 = !!a: %d\n", b2);
}
---
$ ./test
uint64_t a = a: 0x8000000000000000
int i1 = a: 0
int i2 = !!a: 1
bool b1 = a: 1
bool b2 = !!a: 1
---
Quoted 2 lines

Note that if bool would be defined as "int", you would need the !!, otherwise you would lose information.

Agreed. We need to be careful in this case.

Quoted 8 lines

But even for b1, the gcc generates now:

  40118c:       48 8b 05 7d 2e 00 00    mov    0x2e7d(%rip),%rax        # 404010 <a>
  401193:       48 85 c0                test   %rax,%rax
  401196:       0f 95 c0                setne  %al
Quoted 7 lines

My stdbool.h contains

#define bool _Bool

And I think C99 added _Bool that makes that work.

But I didn't read the standard, and it's time for the weekend :)

I just read the C99 and found some interesting information as follows:

6.3.1.2 Boolean type
When any *scalar value* is converted to _Bool, the result is 0 if the value compares equal to 0; otherwise, the result is 1.

6.2.5 Types
21. Arithmetic types and pointer types are collectively called *scalar types*. Array and structure types are collectively called aggregate types.

6.5.13 Logical AND operator
Semantics
The && operator shall yield 1 if both of its operands compare unequal to 0; otherwise, it yields 0. The result has type int.

6.5.10 Bitwise AND operator
Constraints
Each of the operands shall have integer type.
Semantics
The result of the binary & operator is the bitwise AND of the operands (that is, each bit in the result is set if and only if each of the
corresponding
bits in the converted operands is set).

&& would ensure that the result is either 0 or 1, as David said, so no worries.

We defined partially_mapped as a bool(_Bool). IIUC, "partially_mapped = int & int;"
would work correctly as well. However, "partially_mapped = long & int;" might not.

Using && would be nicer as David suggested :p

Thanks,
Lance

Quoted 6 lines

--
Cheers,

David / dhildenb

#6 lore raw
Quoted 124 lines · Lance Yang
Quoted 3 lines · David Hildenbrand
Quoted 1 line · Zi Yan

On 26 Apr 2024, at 15:08, David Hildenbrand wrote:

[...]

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ bool partially_mapped = false;

[...]

Quoted 9 lines
Quoted 7 lines
Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+           partially_mapped = !!nr && !!atomic_read(mapped);

Nit: The && should remove the need for both !!.

My impression was that !! is needed to convert from int to bool and I do find "!!int && !!int" use in the kernel.

I might be wrong about this, but if you wouldn't write

I think you're correct.

Quoted 8 lines
         if (!!nr && !!atomic_read(mapped))

then

bool partially_mapped = nr && atomic_read(mapped);

is sufficient.

+1

Quoted 49 lines

&& would make sure that the result is either 0 or 1, which you can store safely in a bool, no matter which underlying type is used to store that value.

But I *think* nowdays, the compiler will always handle that correctly, even without the "&&" (ever since C99 added _Bool).

Likely, also

         bool partially_mapped = nr & atomic_read(mapped);

Would nowadays work, but looks stupid.

Related: https://lkml.org/lkml/2013/8/31/138

---
#include <stdio.h>
#include <stdbool.h>
#include <stdint.h>
#include <inttypes.h>

volatile uint64_t a = 0x8000000000000000ull;

void main (void) {
          printf("uint64_t a = a: 0x%" PRIx64 "\n", a);

          int i1 = a;
          printf("int i1 = a: %d\n", i1);

          int i2 = !!a;
          printf("int i2 = !!a: %d\n", i2);

          bool b1 = a;
          printf("bool b1 = a: %d\n", b1);

          bool b2 = !!a;
          printf("bool b2 = !!a: %d\n", b2);
}
---
$ ./test
uint64_t a = a: 0x8000000000000000
int i1 = a: 0
int i2 = !!a: 1
bool b1 = a: 1
bool b2 = !!a: 1
---
Quoted 2 lines

Note that if bool would be defined as "int", you would need the !!, otherwise you would lose information.

Agreed. We need to be careful in this case.

Quoted 8 lines

But even for b1, the gcc generates now:

   40118c:       48 8b 05 7d 2e 00 00    mov    0x2e7d(%rip),%rax        # 404010 <a>
   401193:       48 85 c0                test   %rax,%rax
   401196:       0f 95 c0                setne  %al
Quoted 7 lines

My stdbool.h contains

#define bool _Bool

And I think C99 added _Bool that makes that work.

But I didn't read the standard, and it's time for the weekend :)

I just read the C99 and found some interesting information as follows:

6.3.1.2 Boolean type
When any *scalar value* is converted to _Bool, the result is 0 if the value compares equal to 0; otherwise, the result is 1.

6.2.5 Types
21. Arithmetic types and pointer types are collectively called *scalar types*. Array and structure types are collectively called aggregate types.

6.5.13 Logical AND operator
Semantics
The && operator shall yield 1 if both of its operands compare unequal to 0; otherwise, it yields 0. The result has type int.

6.5.10 Bitwise AND operator
Constraints
Each of the operands shall have integer type.
Semantics
The result of the binary & operator is the bitwise AND of the operands (that is, each bit in the result is set if and only if each of the
corresponding
bits in the converted operands is set).

Quoted 2 lines

&& would ensure that the result is either 0 or 1, as David said, so no worries.

My example was flawed: I wanted to express that "if any bit is set, the bool value will be 1. That works for "| vs ||" but not for "& vs &&", obviously :)

Quoted 4 lines

We defined partially_mapped as a bool(_Bool). IIUC, "partially_mapped = int & int;"
would work correctly as well. However, "partially_mapped = long & int;" might not.

Implicit type conversion would convert "long & int" to "long & long" first, which should work just fine. I think really most concerns regarding the bool type are due to < C99 not supporting _Bool.

Great weekend everybody!

Signature · 3 lines
Cheers,

David / dhildenb
#7 lore raw
In reply to Zi Yan
Quoted 76 lines · Zi Yan
Quoted 55 lines · David Hildenbrand
Quoted 54 lines · Zi Yan

From: Zi Yan <[email protected]>

In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary.

For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue.

However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise.

Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration.

To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too.

Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable().

---
  mm/rmap.c | 12 +++++++++---
  1 file changed, 9 insertions(+), 3 deletions(-)
mm/rmap.c +1 -0
index 2608c40dffad..a9bd64ebdd9a 100644
@@ -1495,6 +1495,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
  {
     atomic_t *mapped = &folio->_nr_pages_mapped;
     int last, nr = 0, nr_pmdmapped = 0;
+    bool partially_mapped = false;
     enum node_stat_item idx;
     __folio_rmap_sanity_checks(folio, page, nr_pages, level);
Quoted 8 lines

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1520

Quoted 6 lines
mm/rmap.c +2 -0
@@ -1515,6 +1516,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                                     nr++;
                     }
             } while (page++, --nr_pages > 0);
+
+            partially_mapped = !!nr && !!atomic_read(mapped);

Nit: The && should remove the need for both !!.

My impression was that !! is needed to convert from int to bool and I do find "!!int && !!int" use in the kernel. If this is unnecessary, Andrew can apply the fixup below. I can send a new version if it is really needed.

mm/rmap.c +0 -0
index a9bd64ebdd9a..c1fd5828409b 100644

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1520

Quoted 5 lines
mm/rmap.c +1 -1
@@ -1517,7 +1517,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                        }
                } while (page++, --nr_pages > 0);
 
-               partially_mapped = !!nr && !!atomic_read(mapped);
+               partially_mapped = nr && atomic_read(mapped

Reviewed-by:Barry Song <[email protected]>

Quoted 12 lines
mm/rmap.c +0 -0
                break;
        case RMAP_LEVEL_PMD:
                atomic_dec(&folio->_large_mapcount);
Quoted 3 lines

Thanks.

--
Best Regards,
Yan, Zi

Reviewed-by on v5 1/1
#8 lore raw
In reply to Zi Yan
Quoted 38 lines · Zi Yan

From: Zi Yan <[email protected]>

In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary.

For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue.

However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise.

Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration.

To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too.

Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable().

Quoted 2 lines
Quoted 53 lines
---
 mm/rmap.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
mm/rmap.c +9 -3
index 2608c40dffad..a9bd64ebdd9a 100644
@@ -1495,6 +1495,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
 {
        atomic_t *mapped = &folio->_nr_pages_mapped;
        int last, nr = 0, nr_pmdmapped = 0;
+       bool partially_mapped = false;
        enum node_stat_item idx;
 
        __folio_rmap_sanity_checks(folio, page, nr_pages, level);
@@ -1515,6 +1516,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                                        nr++;
                        }
                } while (page++, --nr_pages > 0);
+
+               partially_mapped = !!nr && !!atomic_read(mapped);
                break;
        case RMAP_LEVEL_PMD:
                atomic_dec(&folio->_large_mapcount);
@@ -1532,6 +1535,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                                nr = 0;
                        }
                }
+
+               partially_mapped = nr < nr_pmdmapped;
                break;
        }
 
@@ -1553,9 +1558,10 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                 * page of the folio is unmapped and at least one page
                 * is still mapped.
                 */
-               if (folio_test_large(folio) && folio_test_anon(folio))
-                       if (level == RMAP_LEVEL_PTE || nr < nr_pmdmapped)
-                               deferred_split_folio(folio);
+               if (folio_test_anon(folio) &&
+                   list_empty(&folio->_deferred_list) &&
+                   partially_mapped)
+                       deferred_split_folio(folio);
        }
 
        /*

base-commit: 3dba658670af22074cc6f26dc92efe0013ac3359 --
2.43.0

Reviewed-by on v5 1/1
#9 lore raw
In reply to Zi Yan
Quoted 38 lines · Zi Yan

From: Zi Yan <[email protected]>

In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary.

For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue.

However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise.

Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration.

To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too.

Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable().

Quoted 2 lines

Thanks,
Lance

Quoted 53 lines
---
 mm/rmap.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
mm/rmap.c +9 -3
index 2608c40dffad..a9bd64ebdd9a 100644
@@ -1495,6 +1495,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
 {
        atomic_t *mapped = &folio->_nr_pages_mapped;
        int last, nr = 0, nr_pmdmapped = 0;
+       bool partially_mapped = false;
        enum node_stat_item idx;
 
        __folio_rmap_sanity_checks(folio, page, nr_pages, level);
@@ -1515,6 +1516,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                                        nr++;
                        }
                } while (page++, --nr_pages > 0);
+
+               partially_mapped = !!nr && !!atomic_read(mapped);
                break;
        case RMAP_LEVEL_PMD:
                atomic_dec(&folio->_large_mapcount);
@@ -1532,6 +1535,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                                nr = 0;
                        }
                }
+
+               partially_mapped = nr < nr_pmdmapped;
                break;
        }
 
@@ -1553,9 +1558,10 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                 * page of the folio is unmapped and at least one page
                 * is still mapped.
                 */
-               if (folio_test_large(folio) && folio_test_anon(folio))
-                       if (level == RMAP_LEVEL_PTE || nr < nr_pmdmapped)
-                               deferred_split_folio(folio);
+               if (folio_test_anon(folio) &&
+                   list_empty(&folio->_deferred_list) &&
+                   partially_mapped)
+                       deferred_split_folio(folio);
        }
 
        /*

base-commit: 3dba658670af22074cc6f26dc92efe0013ac3359 --
2.43.0

#10 lore raw
In reply to Zi Yan

On Fri, Apr 26, 2024 at 03:02:53PM -0400, Zi Yan wrote:

Hi Zi,

It increasingly looks like this commit is crashing on s390 since 2024-04-30 in linux-next. If I do not miss something - since it was included in mm-everything.

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1562

Quoted 9 lines
mm/rmap.c +2 -3
@@ -1553,9 +1558,10 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
 		 * page of the folio is unmapped and at least one page
 		 * is still mapped.
 		 */
-		if (folio_test_large(folio) && folio_test_anon(folio))
-			if (level == RMAP_LEVEL_PTE || nr < nr_pmdmapped)
-				deferred_split_folio(folio);
+		if (folio_test_anon(folio) &&
+		    list_empty(&folio->_deferred_list) &&

An attempt to reference folio->_deferred_list causes the crash below.

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1567

Quoted 5 lines
mm/rmap.c +2 -0
+		    partially_mapped)
+			deferred_split_folio(folio);
 	}
 
 	/*
[  507.227423] Unable to handle kernel pointer dereference in virtual kernel address space
[  507.227432] Failing address: 000001d689000000 TEID: 000001d689000803
[  507.227435] Fault in home space mode while using kernel ASCE.
[  507.227439] AS:0000000180788007 R3:00000001fe2cc007 S:0000000000000020 
[  507.227492] Oops: 0010 ilc:3 [#1] SMP 
[  507.227497] Modules linked in: vmur(E) kvm(E) algif_hash(E) af_alg(E) binfmt_misc(E) nft_fib_inet(E) nft_fib_ipv4(E) nft_fib_ipv6(E) nft_fib(E) nft_reject_inet(E) nf_reject_ipv4(E) nf_reject_ipv6(E) nft_reject(E) nft_ct(E) nft_chain_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) ip_set(E) nf_tables(E) nfnetlink(E) dm_service_time(E) s390_trng(E) vfio_ccw(E) mdev(E) vfio_iommu_type1(E) vfio(E) sch_fq_codel(E) loop(E) configfs(E) lcs(E) ctcm(E) fsm(E) zfcp(E) scsi_transport_fc(E) ghash_s390(E) prng(E) chacha_s390(E) libchacha(E) aes_s390(E) des_s390(E) libdes(E) sha3_512_s390(E) sha3_256_s390(E) sha512_s390(E) sha256_s390(E) sha1_s390(E) sha_common(E) scsi_dh_rdac(E) scsi_dh_emc(E) scsi_dh_alua(E) pkey(E) zcrypt(E) rng_core(E) dm_multipath(E) autofs4(E)
[  507.227546] Unloaded tainted modules: dcssblk(E):2 [last unloaded: dcssblk(E)]
[  507.230569] CPU: 0 PID: 36783 Comm: pahole Tainted: G            E      6.9.0-20240430.rc6.git237.d04466706db5.300.fc39.s390x+next #1
[  507.230574] Hardware name: IBM 3931 A01 703 (z/VM 7.3.0)
[  507.230576] Krnl PSW : 0704f00180000000 0000025e1092a430 (folio_remove_rmap_ptes+0xe0/0x140)
[  507.230588]            R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:3 PM:0 RI:0 EA:3
[  507.230592] Krnl GPRS: ffffffffffffe377 0000000000000000 0000025e122075b8 0000000000000000
[  507.230595]            ffffffffffffffff 0000025d8f613288 8800000000000000 00000157a38b8700
[  507.230598]            000000023fffe13f 0000000000000000 000001579ccd75c0 000001d688ffff80
[  507.230602]            000003ffb9cacf98 000001d688ffff80 0000025e1092a428 000001de11fab878
[  507.230610] Krnl Code: 0000025e1092a422: c0e500039f47        brasl   %r14,0000025e1099e2b0
[  507.230610]            0000025e1092a428: 9101b01f            tm      31(%r11),1
[  507.230610]           #0000025e1092a42c: a784ffb9            brc     8,0000025e1092a39e
[  507.230610]           >0000025e1092a430: e340b0900004        lg      %r4,144(%r11)
[  507.230610]            0000025e1092a436: 4150b090            la      %r5,144(%r11)
[  507.230610]            0000025e1092a43a: ec45ffb26064        cgrj    %r4,%r5,6,0000025e1092a39e
[  507.230610]            0000025e1092a440: a7910001            tmll    %r9,1
[  507.230610]            0000025e1092a444: a784ffad            brc     8,0000025e1092a39e
[  507.230672] Call Trace:
[  507.230678]  [<0000025e1092a430>] folio_remove_rmap_ptes+0xe0/0x140 
[  507.230682] ([<0000025e1092a428>] folio_remove_rmap_ptes+0xd8/0x140)
[  507.230685]  [<0000025e1090d76a>] zap_present_ptes.isra.0+0x222/0x918 
[  507.230689]  [<0000025e1090e008>] zap_pte_range+0x1a8/0x4e8 
[  507.230692]  [<0000025e1090e58c>] zap_p4d_range+0x244/0x480 
[  507.230695]  [<0000025e1090eb22>] unmap_page_range+0xea/0x2c0 
[  507.230698]  [<0000025e1090ed92>] unmap_single_vma.isra.0+0x9a/0xf0 
[  507.230701]  [<0000025e1090ee9e>] unmap_vmas+0xb6/0x1a0 
[  507.230705]  [<0000025e1091e0d4>] exit_mmap+0xc4/0x3d0 
[  507.230709]  [<0000025e10675c64>] __mmput+0x54/0x150 
[  507.230714]  [<0000025e1067f3ba>] exit_mm+0xca/0x138 
[  507.230717]  [<0000025e1067f690>] do_exit+0x268/0x520 
[  507.230721]  [<0000025e1067fb38>] do_group_exit+0x40/0xb8 
[  507.230725]  [<0000025e1067fc0e>] __s390x_sys_exit_group+0x2e/0x30 
[  507.230729]  [<0000025e1136ba4e>] __do_syscall+0x216/0x2d0 
[  507.230736]  [<0000025e1137c848>] system_call+0x70/0x98
[  507.230780] Last Breaking-Event-Address:
[  507.230783]  [<0000025e1099e32a>] __lruvec_stat_mod_folio+0x7a/0xb0
[  507.230789] Kernel panic - not syncing: Fatal exception: panic_on_oops

00: HCPGIR450W CP entered; disabled wait PSW 00020001 80000000 0000025E 10630B56

Thanks!

#11 lore raw
Quoted 8 lines · Alexander Gordeev

On Fri, Apr 26, 2024 at 03:02:53PM -0400, Zi Yan wrote:

Hi Zi,

It increasingly looks like this commit is crashing on s390 since 2024-04-30 in linux-next. If I do not miss something - since it was included in mm-everything.

Quoted 11 lines

Review static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1562

Quoted 9 lines
(quoted diff) +2 -3
@@ -1553,9 +1558,10 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
 		 * page of the folio is unmapped and at least one page
 		 * is still mapped.
 		 */
-		if (folio_test_large(folio) && folio_test_anon(folio))
-			if (level == RMAP_LEVEL_PTE || nr < nr_pmdmapped)
-				deferred_split_folio(folio);
+		if (folio_test_anon(folio) &&
+		    list_empty(&folio->_deferred_list) &&

An attempt to reference folio->_deferred_list causes the crash below.

So if you remove this line, the crash no longer happens? It looks strange to me that referencing a anonymous folio's _deferred_list would cause a crash. Hmm, unless the folio is order-0.

Can you try the patch below and see if it fixes the crash? It moves partially_mapped ahead to exclude order-0 folios.

mm/rmap.c +2 -3
index 087a79f1f611..2d27c92bb6d5 100644
@@ -1557,9 +1557,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                 * page of the folio is unmapped and at least one page
                 * is still mapped.
                 */
-               if (folio_test_anon(folio) &&
-                   list_empty(&folio->_deferred_list) &&
-                   partially_mapped)
+               if (folio_test_anon(folio) && partially_mapped &&
+                   list_empty(&folio->_deferred_list))
                        deferred_split_folio(folio);
        }
 
Quoted 50 lines

Review static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1567

Quoted 6 lines
(quoted diff) +2 -0
+		    partially_mapped)
+			deferred_split_folio(folio);
 	}
 
 	/*
[  507.227423] Unable to handle kernel pointer dereference in virtual kernel address space
[  507.227432] Failing address: 000001d689000000 TEID: 000001d689000803
[  507.227435] Fault in home space mode while using kernel ASCE.
[  507.227439] AS:0000000180788007 R3:00000001fe2cc007 S:0000000000000020
[  507.227492] Oops: 0010 ilc:3 [#1] SMP
[  507.227497] Modules linked in: vmur(E) kvm(E) algif_hash(E) af_alg(E) binfmt_misc(E) nft_fib_inet(E) nft_fib_ipv4(E) nft_fib_ipv6(E) nft_fib(E) nft_reject_inet(E) nf_reject_ipv4(E) nf_reject_ipv6(E) nft_reject(E) nft_ct(E) nft_chain_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) ip_set(E) nf_tables(E) nfnetlink(E) dm_service_time(E) s390_trng(E) vfio_ccw(E) mdev(E) vfio_iommu_type1(E) vfio(E) sch_fq_codel(E) loop(E) configfs(E) lcs(E) ctcm(E) fsm(E) zfcp(E) scsi_transport_fc(E) ghash_s390(E) prng(E) chacha_s390(E) libchacha(E) aes_s390(E) des_s390(E) libdes(E) sha3_512_s390(E) sha3_256_s390(E) sha512_s390(E) sha256_s390(E) sha1_s390(E) sha_common(E) scsi_dh_rdac(E) scsi_dh_emc(E) scsi_dh_alua(E) pkey(E) zcrypt(E) rng_core(E) dm_multipath(E) autofs4(E)
[  507.227546] Unloaded tainted modules: dcssblk(E):2 [last unloaded: dcssblk(E)]
[  507.230569] CPU: 0 PID: 36783 Comm: pahole Tainted: G            E      6.9.0-20240430.rc6.git237.d04466706db5.300.fc39.s390x+next #1
[  507.230574] Hardware name: IBM 3931 A01 703 (z/VM 7.3.0)
[  507.230576] Krnl PSW : 0704f00180000000 0000025e1092a430 (folio_remove_rmap_ptes+0xe0/0x140)
[  507.230588]            R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:3 PM:0 RI:0 EA:3
[  507.230592] Krnl GPRS: ffffffffffffe377 0000000000000000 0000025e122075b8 0000000000000000
[  507.230595]            ffffffffffffffff 0000025d8f613288 8800000000000000 00000157a38b8700
[  507.230598]            000000023fffe13f 0000000000000000 000001579ccd75c0 000001d688ffff80
[  507.230602]            000003ffb9cacf98 000001d688ffff80 0000025e1092a428 000001de11fab878
[  507.230610] Krnl Code: 0000025e1092a422: c0e500039f47        brasl   %r14,0000025e1099e2b0
[  507.230610]            0000025e1092a428: 9101b01f            tm      31(%r11),1
[  507.230610]           #0000025e1092a42c: a784ffb9            brc     8,0000025e1092a39e
[  507.230610]           >0000025e1092a430: e340b0900004        lg      %r4,144(%r11)
[  507.230610]            0000025e1092a436: 4150b090            la      %r5,144(%r11)
[  507.230610]            0000025e1092a43a: ec45ffb26064        cgrj    %r4,%r5,6,0000025e1092a39e
[  507.230610]            0000025e1092a440: a7910001            tmll    %r9,1
[  507.230610]            0000025e1092a444: a784ffad            brc     8,0000025e1092a39e
[  507.230672] Call Trace:
[  507.230678]  [<0000025e1092a430>] folio_remove_rmap_ptes+0xe0/0x140
[  507.230682] ([<0000025e1092a428>] folio_remove_rmap_ptes+0xd8/0x140)
[  507.230685]  [<0000025e1090d76a>] zap_present_ptes.isra.0+0x222/0x918
[  507.230689]  [<0000025e1090e008>] zap_pte_range+0x1a8/0x4e8
[  507.230692]  [<0000025e1090e58c>] zap_p4d_range+0x244/0x480
[  507.230695]  [<0000025e1090eb22>] unmap_page_range+0xea/0x2c0
[  507.230698]  [<0000025e1090ed92>] unmap_single_vma.isra.0+0x9a/0xf0
[  507.230701]  [<0000025e1090ee9e>] unmap_vmas+0xb6/0x1a0
[  507.230705]  [<0000025e1091e0d4>] exit_mmap+0xc4/0x3d0
[  507.230709]  [<0000025e10675c64>] __mmput+0x54/0x150
[  507.230714]  [<0000025e1067f3ba>] exit_mm+0xca/0x138
[  507.230717]  [<0000025e1067f690>] do_exit+0x268/0x520
[  507.230721]  [<0000025e1067fb38>] do_group_exit+0x40/0xb8
[  507.230725]  [<0000025e1067fc0e>] __s390x_sys_exit_group+0x2e/0x30
[  507.230729]  [<0000025e1136ba4e>] __do_syscall+0x216/0x2d0
[  507.230736]  [<0000025e1137c848>] system_call+0x70/0x98
[  507.230780] Last Breaking-Event-Address:
[  507.230783]  [<0000025e1099e32a>] __lruvec_stat_mod_folio+0x7a/0xb0
[  507.230789] Kernel panic - not syncing: Fatal exception: panic_on_oops
Quoted 3 lines

00: HCPGIR450W CP entered; disabled wait PSW 00020001 80000000 0000025E 10630B56

Thanks!

--
Best Regards,
Yan, Zi

#12 lore raw
Quoted 34 lines · Zi Yan
Quoted 8 lines · Alexander Gordeev

On Fri, Apr 26, 2024 at 03:02:53PM -0400, Zi Yan wrote:

Hi Zi,

It increasingly looks like this commit is crashing on s390 since 2024-04-30 in linux-next. If I do not miss something - since it was included in mm-everything.

Quoted 11 lines

Review static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1562

Quoted 9 lines
(quoted diff) +2 -3
@@ -1553,9 +1558,10 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
  		 * page of the folio is unmapped and at least one page
  		 * is still mapped.
  		 */
-		if (folio_test_large(folio) && folio_test_anon(folio))
-			if (level == RMAP_LEVEL_PTE || nr < nr_pmdmapped)
-				deferred_split_folio(folio);
+		if (folio_test_anon(folio) &&
+		    list_empty(&folio->_deferred_list) &&

An attempt to reference folio->_deferred_list causes the crash below.

So if you remove this line, the crash no longer happens? It looks strange to me that referencing a anonymous folio's _deferred_list would cause a crash. Hmm, unless the folio is order-0.

Can you try the patch below and see if it fixes the crash? It moves partially_mapped ahead to exclude order-0 folios.

mm/rmap.c +0 -0
index 087a79f1f611..2d27c92bb6d5 100644

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1562

Quoted 9 lines
mm/rmap.c +2 -3
@@ -1557,9 +1557,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                  * page of the folio is unmapped and at least one page
                  * is still mapped.
                  */
-               if (folio_test_anon(folio) &&
-                   list_empty(&folio->_deferred_list) &&
-                   partially_mapped)
+               if (folio_test_anon(folio) && partially_mapped &&
+                   list_empty(&folio->_deferred_list))
                         deferred_split_folio(folio);

Yes, that should fix it and is the right thing to do. For small folios, partially_mapped will always be false.

Signature · 3 lines
Cheers,

David / dhildenb
#13 lore raw
In reply to Zi Yan

On Wed, May 01, 2024 at 09:38:24AM -0400, Zi Yan wrote:
Hi Zi,

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1563

Quoted 11 lines
mm/rmap.c +2 -3
@@ -1557,9 +1557,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                 * page of the folio is unmapped and at least one page
                 * is still mapped.
                 */
-               if (folio_test_anon(folio) &&
-                   list_empty(&folio->_deferred_list) &&
-                   partially_mapped)
+               if (folio_test_anon(folio) && partially_mapped &&
+                   list_empty(&folio->_deferred_list))
                        deferred_split_folio(folio);
        }

That helps.

Quoted 2 lines
(quoted diff) +0 -0
 Best Regards,

Yan, Zi

Thanks!

#14 lore raw
Quoted 16 lines · Alexander Gordeev

On Wed, May 01, 2024 at 09:38:24AM -0400, Zi Yan wrote:
Hi Zi,

mm/rmap.c static __always_inline void __folio_remove_rmap(struct folio *folio, · line 1563

Quoted 11 lines
mm/rmap.c +2 -3
@@ -1557,9 +1557,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
                 * page of the folio is unmapped and at least one page
                 * is still mapped.
                 */
-               if (folio_test_anon(folio) &&
-                   list_empty(&folio->_deferred_list) &&
-                   partially_mapped)
+               if (folio_test_anon(folio) && partially_mapped &&
+                   list_empty(&folio->_deferred_list))
                        deferred_split_folio(folio);
        }

That helps.

Quoted 4 lines
Quoted 2 lines
(quoted diff) +0 -0
 Best Regards,

Yan, Zi

Thanks!

Great! I will send a v6.

--
Best Regards,
Yan, Zi