Skip to contentSkip to conversation

mm/gup: pgtable entry acccessors cleanup

RFC PATCH Superseded Alexander Gordeev 2 participants 5 messages Apr 20

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Alexander Gordeev

author maintainer
#0 lore raw

cover mm/gup: pgtable entry acccessors cleanup

Hi All,

This rework is necessary to allow a KASAN PTE access sanitizer (so far s390-only) that hits on direct PTE entries dereferences, like pte = *ptep as opposed to pte = ptep_get(ptep).

I am not posting the sanitizer itself (yet), since it does not fly without the suggested patches, but I am not sure whether these changes make sense for the generic case.

Thus, sending it as a RFC.

Thanks!

Alexander Gordeev (2):
mm/gup: add missing pXdp_get() conversions mm/gup: add lockless access semantics on entries validation

 mm/gup.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
Signature · 1 line
2.51.0
author maintainer
#1 lore raw

RFC PATCH 1/2 mm/gup: add missing pXdp_get() conversions

PMD and PUD entries revalidation has the same semantics as PTE entry revalidation. Convert the remaining direct entry dereferences to the corresponding pXdp_get() accessors.

---
 mm/gup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
mm/gup.c +3 -3
index 8e7dc2c6ee73..d149a4b0df71 100644
@@ -2865,7 +2865,7 @@ static int gup_fast_pte_range(pmd_t pmd, pmd_t *pmdp, unsigned long addr,
 		if (!folio)
 			goto pte_unmap;
 
-		if (unlikely(pmd_val(pmd) != pmd_val(*pmdp)) ||
+		if (unlikely(pmd_val(pmd) != pmd_val(pmdp_get(pmdp))) ||
 		    unlikely(pte_val(pte) != pte_val(ptep_get(ptep)))) {
 			gup_put_folio(folio, 1, flags);
 			goto pte_unmap;
@@ -2942,7 +2942,7 @@ static int gup_fast_pmd_leaf(pmd_t orig, pmd_t *pmdp, unsigned long addr,
 	if (!folio)
 		return 0;
 
-	if (unlikely(pmd_val(orig) != pmd_val(*pmdp))) {
+	if (unlikely(pmd_val(orig) != pmd_val(pmdp_get(pmdp)))) {
 		gup_put_folio(folio, refs, flags);
 		return 0;
 	}
@@ -2985,7 +2985,7 @@ static int gup_fast_pud_leaf(pud_t orig, pud_t *pudp, unsigned long addr,
 	if (!folio)
 		return 0;
 
-	if (unlikely(pud_val(orig) != pud_val(*pudp))) {
+	if (unlikely(pud_val(orig) != pud_val(pudp_get(pudp)))) {
 		gup_put_folio(folio, refs, flags);
 		return 0;
 	}
Signature · 1 line
2.51.0
#2 lore raw

Re: [RFC PATCH 1/2] mm/gup: add missing pXdp_get() conversions

Quoted 32 lines · Alexander Gordeev

PMD and PUD entries revalidation has the same semantics as PTE entry revalidation. Convert the remaining direct entry dereferences to the corresponding pXdp_get() accessors.

---
 mm/gup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
mm/gup.c +2 -2
index 8e7dc2c6ee73..d149a4b0df71 100644
@@ -2865,7 +2865,7 @@ static int gup_fast_pte_range(pmd_t pmd, pmd_t *pmdp, unsigned long addr,
 		if (!folio)
 			goto pte_unmap;
 
-		if (unlikely(pmd_val(pmd) != pmd_val(*pmdp)) ||
+		if (unlikely(pmd_val(pmd) != pmd_val(pmdp_get(pmdp))) ||
 		    unlikely(pte_val(pte) != pte_val(ptep_get(ptep)))) {
 			gup_put_folio(folio, 1, flags);
 			goto pte_unmap;
@@ -2942,7 +2942,7 @@ static int gup_fast_pmd_leaf(pmd_t orig, pmd_t *pmdp, unsigned long addr,
 	if (!folio)
 		return 0;
 
-	if (unlikely(pmd_val(orig) != pmd_val(*pmdp))) {
+	if (unlikely(pmd_val(orig) != pmd_val(pmdp_get(pmdp)))) {
 		gup_put_folio(folio, refs, flags);
 		return 0;
 	}

mm/gup.c static int gup_fast_pud_leaf(pud_t orig, pud_t *pudp, unsigned long addr, · line 2991

Quoted 8 lines
mm/gup.c +1 -1
@@ -2985,7 +2985,7 @@ static int gup_fast_pud_leaf(pud_t orig, pud_t *pudp, unsigned long addr,
 	if (!folio)
 		return 0;
 
-	if (unlikely(pud_val(orig) != pud_val(*pudp))) {
+	if (unlikely(pud_val(orig) != pud_val(pudp_get(pudp)))) {
 		gup_put_folio(folio, refs, flags);
 		return 0;
 	}

These are all on lockless code paths. Shouldn't we be using the lockless pte/pmd variants?

(not a matter of correctness, rather a matter of semantics)

Signature · 3 lines
Cheers,

David
author maintainer
#3 lore raw

RFC PATCH 2/2 mm/gup: add lockless access semantics on entries validation

In reply to Alexander Gordeev

The PTE validation in gup_fast_pte_range() is inconsistent with the prior value acquisition in the sense that it drops the lockless access semantics.

Although this is highly unlikely, prevent a future scenario in which a semantically mismatching ptep_get() incorrectly yields the same result as the preceding ptep_get_lockless(), while ptep_get_lockless() would otherwise return a different value.

Likewise the PMD validation is inconsistent with the prior value acquisition in gup_fast_pmd_range().

---
 mm/gup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
mm/gup.c +3 -3
index d149a4b0df71..236450feea9a 100644
@@ -2865,8 +2865,8 @@ static int gup_fast_pte_range(pmd_t pmd, pmd_t *pmdp, unsigned long addr,
 		if (!folio)
 			goto pte_unmap;
 
-		if (unlikely(pmd_val(pmd) != pmd_val(pmdp_get(pmdp))) ||
-		    unlikely(pte_val(pte) != pte_val(ptep_get(ptep)))) {
+		if (unlikely(pmd_val(pmd) != pmd_val(pmdp_get_lockless(pmdp))) ||
+		    unlikely(pte_val(pte) != pte_val(ptep_get_lockless(ptep)))) {
 			gup_put_folio(folio, 1, flags);
 			goto pte_unmap;
 		}
@@ -2942,7 +2942,7 @@ static int gup_fast_pmd_leaf(pmd_t orig, pmd_t *pmdp, unsigned long addr,
 	if (!folio)
 		return 0;
 
-	if (unlikely(pmd_val(orig) != pmd_val(pmdp_get(pmdp)))) {
+	if (unlikely(pmd_val(orig) != pmd_val(pmdp_get_lockless(pmdp)))) {
 		gup_put_folio(folio, refs, flags);
 		return 0;
 	}
Signature · 1 line
2.51.0
#4 lore raw

Re: [RFC PATCH 2/2] mm/gup: add lockless access semantics on entries validation

Quoted 33 lines · Alexander Gordeev

The PTE validation in gup_fast_pte_range() is inconsistent with the prior value acquisition in the sense that it drops the lockless access semantics.

Although this is highly unlikely, prevent a future scenario in which a semantically mismatching ptep_get() incorrectly yields the same result as the preceding ptep_get_lockless(), while ptep_get_lockless() would otherwise return a different value.

Likewise the PMD validation is inconsistent with the prior value acquisition in gup_fast_pmd_range().

---
 mm/gup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
mm/gup.c +2 -2
index d149a4b0df71..236450feea9a 100644
@@ -2865,8 +2865,8 @@ static int gup_fast_pte_range(pmd_t pmd, pmd_t *pmdp, unsigned long addr,
 		if (!folio)
 			goto pte_unmap;
 
-		if (unlikely(pmd_val(pmd) != pmd_val(pmdp_get(pmdp))) ||
-		    unlikely(pte_val(pte) != pte_val(ptep_get(ptep)))) {
+		if (unlikely(pmd_val(pmd) != pmd_val(pmdp_get_lockless(pmdp))) ||
+		    unlikely(pte_val(pte) != pte_val(ptep_get_lockless(ptep)))) {
 			gup_put_folio(folio, 1, flags);
 			goto pte_unmap;
 		}

mm/gup.c static int gup_fast_pmd_leaf(pmd_t orig, pmd_t *pmdp, unsigned long addr, · line 2948

Quoted 8 lines
mm/gup.c +1 -1
@@ -2942,7 +2942,7 @@ static int gup_fast_pmd_leaf(pmd_t orig, pmd_t *pmdp, unsigned long addr,
 	if (!folio)
 		return 0;
 
-	if (unlikely(pmd_val(orig) != pmd_val(pmdp_get(pmdp)))) {
+	if (unlikely(pmd_val(orig) != pmd_val(pmdp_get_lockless(pmdp)))) {
 		gup_put_folio(folio, refs, flags);
 		return 0;
 	}

Oh, that should be squashed into #1 :)

Signature · 3 lines
Cheers,

David