Skip to contentSkip to conversation

mm/gup: dereference page table entry using helper

PATCH In review Alexander Gordeev 3 participants 6 messages Apr 14, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/gup: dereference page table entry using helper

Commit 0005d20 ("mm/gup: Move page table entry dereference into helper function") wrapped access to page table entries larger than sizeof(long) into a race-aware accessor. One of the two dereferences in gup_fast path was however overlooked.

---
 mm/gup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/gup.c +1 -1
index d53f7dd..eceb98b 100644
@@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 		if (!head)
 			goto pte_unmap;
 
-		if (unlikely(pte_val(pte) != pte_val(*ptep))) {
+		if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) {
 			put_compound_head(head, 1, flags);
 			goto pte_unmap;
 		}
Signature · 1 line
1.8.3.1
#1 lore raw
Quoted 15 lines · Alexander Gordeev

Commit 0005d20 ("mm/gup: Move page table entry dereference into helper function") wrapped access to page table entries larger than sizeof(long) into a race-aware accessor. One of the two dereferences in gup_fast path was however overlooked.

 mm/gup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/gup.c +1 -0
index d53f7dd..eceb98b 100644
+++ b/mm/gup.c

mm/gup.c static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, · line 2211

Quoted 5 lines
mm/gup.c +1 -1
@@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 		if (!head)
 			goto pte_unmap;
 
-		if (unlikely(pte_val(pte) != pte_val(*ptep))) {
+		if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) {

It doesn't seem like this needs the special helper as it is just checking that the pte hasn't changed, it doesn't need to be read exactly.

But it probably should technically still be a READ_ONCE. Although I think the atomic inside try_grab_compound_head prevents any real problems.

Jason

#2 lore raw
Quoted 26 lines · Jason Gunthorpe
Quoted 15 lines · Alexander Gordeev

Commit 0005d20 ("mm/gup: Move page table entry dereference into helper function") wrapped access to page table entries larger than sizeof(long) into a race-aware accessor. One of the two dereferences in gup_fast path was however overlooked.

 mm/gup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/gup.c +1 -0
index d53f7dd..eceb98b 100644
+++ b/mm/gup.c

mm/gup.c static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, · line 2211

Quoted 5 lines
mm/gup.c +1 -1
@@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 		if (!head)
 			goto pte_unmap;
 
-		if (unlikely(pte_val(pte) != pte_val(*ptep))) {
+		if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) {

It doesn't seem like this needs the special helper as it is just checking that the pte hasn't changed, it doesn't need to be read exactly.

Quoted 3 lines

But it probably should technically still be a READ_ONCE. Although I think the atomic inside try_grab_compound_head prevents any real problems.

I think we should go for consistency here and use the helper function.

Ira

Quoted 3 lines

Jason

#3 lore raw
Quoted 27 lines · Ira Weiny
Quoted 26 lines · Jason Gunthorpe
Quoted 15 lines · Alexander Gordeev

Commit 0005d20 ("mm/gup: Move page table entry dereference into helper function") wrapped access to page table entries larger than sizeof(long) into a race-aware accessor. One of the two dereferences in gup_fast path was however overlooked.

 mm/gup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/gup.c +1 -0
index d53f7dd..eceb98b 100644
+++ b/mm/gup.c

mm/gup.c static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, · line 2211

Quoted 5 lines
mm/gup.c +1 -1
@@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 		if (!head)
 			goto pte_unmap;
 
-		if (unlikely(pte_val(pte) != pte_val(*ptep))) {
+		if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) {

It doesn't seem like this needs the special helper as it is just checking that the pte hasn't changed, it doesn't need to be read exactly.

Quoted 5 lines
Quoted 3 lines

But it probably should technically still be a READ_ONCE. Although I think the atomic inside try_grab_compound_head prevents any real problems.

I think we should go for consistency here and use the helper function.

It seems quite expensive to do two more unncessary barriers..

Jason

#4 lore raw
Quoted 28 lines · Jason Gunthorpe
Quoted 27 lines · Ira Weiny
Quoted 26 lines · Jason Gunthorpe
Quoted 15 lines · Alexander Gordeev

Commit 0005d20 ("mm/gup: Move page table entry dereference into helper function") wrapped access to page table entries larger than sizeof(long) into a race-aware accessor. One of the two dereferences in gup_fast path was however overlooked.

 mm/gup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/gup.c +1 -0
index d53f7dd..eceb98b 100644
+++ b/mm/gup.c

mm/gup.c static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, · line 2211

Quoted 5 lines
mm/gup.c +1 -1
@@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 		if (!head)
 			goto pte_unmap;
 
-		if (unlikely(pte_val(pte) != pte_val(*ptep))) {
+		if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) {

It doesn't seem like this needs the special helper as it is just checking that the pte hasn't changed, it doesn't need to be read exactly.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

But it probably should technically still be a READ_ONCE. Although I think the atomic inside try_grab_compound_head prevents any real problems.

I think we should go for consistency here and use the helper function.

It seems quite expensive to do two more unncessary barriers..

But won't a failure to read the 'real' pte result in falling back to GUP slow?

Not sure which is worse?

And most arch's don't suffer from this...

Ira

#5 lore raw
Quoted 33 lines · Ira Weiny
Quoted 32 lines · Jason Gunthorpe
Quoted 31 lines · Ira Weiny
Quoted 29 lines · Jason Gunthorpe
Quoted 15 lines · Alexander Gordeev

Commit 0005d20 ("mm/gup: Move page table entry dereference into helper function") wrapped access to page table entries larger than sizeof(long) into a race-aware accessor. One of the two dereferences in gup_fast path was however overlooked.

 mm/gup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/gup.c +1 -0
index d53f7dd..eceb98b 100644
+++ b/mm/gup.c

mm/gup.c static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, · line 2211

Quoted 5 lines
mm/gup.c +1 -1
@@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 		if (!head)
 			goto pte_unmap;
 
-		if (unlikely(pte_val(pte) != pte_val(*ptep))) {
+		if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) {

It doesn't seem like this needs the special helper as it is just checking that the pte hasn't changed, it doesn't need to be read exactly.

But it probably should technically still be a READ_ONCE. Although I think the atomic inside try_grab_compound_head prevents any real problems.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

I think we should go for consistency here and use the helper function.

It seems quite expensive to do two more unncessary barriers..

But won't a failure to read the 'real' pte result in falling back to GUP slow?

If there is no concurrent writer then the direct read will give the same result.

If there is a concurrent writer then it is a random race if fallback to gup slow is required.

Jason