Skip to contentSkip to conversation

mm/rmap: remove unnecessary page_table_lock

PATCH In review Yajun Deng 4 participants 8 messages Apr 22, 2024 – Apr 24, 2024

Subsystems (2)
lore
Share & tools
lore

Yajun Deng

author
#0 lore raw

PATCH 1/1 mm/rmap: remove unnecessary page_table_lock

page_table_lock is a lock that for page table, we won't change page table in __anon_vma_prepare(). As we can see, it works well in anon_vma_clone(). They do the same operation.

Remove unnecessary page_table_lock in __anon_vma_prepare().

---
 mm/rmap.c | 4 ----
 1 file changed, 4 deletions(-)
mm/rmap.c +0 -4
index 2608c40dffad..e894640a9327 100644
@@ -187,7 +187,6 @@ static void anon_vma_chain_link(struct vm_area_struct *vma,
  */
 int __anon_vma_prepare(struct vm_area_struct *vma)
 {
-	struct mm_struct *mm = vma->vm_mm;
 	struct anon_vma *anon_vma, *allocated;
 	struct anon_vma_chain *avc;
 
@@ -208,8 +207,6 @@ int __anon_vma_prepare(struct vm_area_struct *vma)
 	}
 
 	anon_vma_lock_write(anon_vma);
-	/* page_table_lock to protect against threads */
-	spin_lock(&mm->page_table_lock);
 	if (likely(!vma->anon_vma)) {
 		vma->anon_vma = anon_vma;
 		anon_vma_chain_link(vma, avc, anon_vma);
@@ -217,7 +214,6 @@ int __anon_vma_prepare(struct vm_area_struct *vma)
 		allocated = NULL;
 		avc = NULL;
 	}
-	spin_unlock(&mm->page_table_lock);
 	anon_vma_unlock_write(anon_vma);
 
 	if (unlikely(allocated))
Signature · 1 line
2.25.1
#1 lore raw
Quoted 3 lines · Yajun Deng

page_table_lock is a lock that for page table, we won't change page table in __anon_vma_prepare(). As we can see, it works well in anon_vma_clone(). They do the same operation.

We are reusing mm->page_table_lock to serialize, not the *actual* low-level page table locks that really protect PTEs.

With that locking gone, there would be nothing protection vma->anon_vma.

Note that anon_vma_clone() is likely called with the mmap_lock held in write mode, which is not the case for __anon_vma_prepare() ...

I think this change is wrong.

Signature · 3 lines
Cheers,

David / dhildenb
author
#2 lore raw
Quoted 8 lines · David Hildenbrand
Quoted 5 lines · Yajun Deng

page_table_lock is a lock that for page table, we won't change page

table in __anon_vma_prepare(). As we can see, it works well in

Quoted 8 lines
Quoted 2 lines

anon_vma_clone(). They do the same operation.

We are reusing mm->page_table_lock to serialize, not the *actual* low-level page table locks that really protect PTEs.

With that locking gone, there would be nothing protection vma->anon_vma.

Note that anon_vma_clone() is likely called with the mmap_lock held in write mode, which is not the case for __anon_vma_prepare() ...

Yes, anon_vma_clone() is called with the mmap_lock held. I added mmap_assert_write_locked(dst->vm_mm) to prove it. I added mmap_assert_write_locked(vma->vm_mm) in __anon_vma_prepare() at the same time, it shows __anon_vma_prepare() is also called with the mmap_lock held too.

Quoted 7 lines

I think this change is wrong.

-- Cheers,

David / dhildenb

#3 lore raw
Quoted 17 lines · Yajun Deng
Quoted 13 lines · David Hildenbrand
Quoted 7 lines · Yajun Deng

page_table_lock is a lock that for page table, we won't change page

table in __anon_vma_prepare(). As we can see, it works well in

anon_vma_clone(). They do the same operation.

We are reusing mm->page_table_lock to serialize, not the *actual* low-level page table locks that really protect PTEs.

Quoted 7 lines
Quoted 3 lines

With that locking gone, there would be nothing protection vma->anon_vma.

Note that anon_vma_clone() is likely called with the mmap_lock held in write mode, which is not the case for __anon_vma_prepare() ...

Yes, anon_vma_clone() is called with the mmap_lock held. I added mmap_assert_write_locked(dst->vm_mm) to prove it. I added mmap_assert_write_locked(vma->vm_mm) in __anon_vma_prepare() at the same time, it shows __anon_vma_prepare() is also called with the mmap_lock held too.

Make sure you actually have lockdep built in and enabled.

__anon_vma_prepare() is for example called from do_anonymous_page() where we might only hold the mmap_lock in read mode (or not at all IIRC with VMA in read mode).

Signature · 3 lines
Cheers,

David / dhildenb
author
#4 lore raw
Quoted 26 lines · David Hildenbrand
Quoted 23 lines · Yajun Deng
Quoted 5 lines · David Hildenbrand
Quoted 1 line
Quoted 1 line

On 22.04.24 12:52, Yajun Deng wrote:

page_table_lock is a lock that for page table, we won't change page

table in __anon_vma_prepare(). As we can see, it works well in

anon_vma_clone(). They do the same operation.

Quoted 7 lines

We are reusing mm->page_table_lock to serialize, not the *actual* low-level page table locks that really protect PTEs.

With that locking gone, there would be nothing protection vma->anon_vma.

Note that anon_vma_clone() is likely called with the mmap_lock held in write mode, which is not the case for __anon_vma_prepare() ...

Quoted 9 lines
Quoted 6 lines

Yes, anon_vma_clone() is called with the mmap_lock held. I added mmap_assert_write_locked(dst->vm_mm) to prove it.

I added mmap_assert_write_locked(vma->vm_mm) in __anon_vma_prepare() at the same time, it shows __anon_vma_prepare()

is also called with the mmap_lock held too.

Make sure you actually have lockdep built in and enabled.

This is my config.
CONFIG_LOCKDEP=n
CONFIG_DEBUG_VM=y

I did another test.
I put mmap_assert_write_locked(mm) before 'set_bit(MMF_OOM_SKIP, &mm->flags)' in mmap.c, it's outside the lock. It will crash when on boot. I think mmap_assert_write_locked() works.

Quoted 6 lines

__anon_vma_prepare() is for example called from do_anonymous_page() where we might only hold the mmap_lock in read mode (or not at all IIRC with VMA in read mode).

-- Cheers,

David / dhildenb

#5 lore raw
Quoted 40 lines · Yajun Deng
Quoted 26 lines · David Hildenbrand
Quoted 23 lines · Yajun Deng
Quoted 5 lines · David Hildenbrand
Quoted 1 line
Quoted 1 line

On 22.04.24 12:52, Yajun Deng wrote:

page_table_lock is a lock that for page table, we won't change page

table in __anon_vma_prepare(). As we can see, it works well in

anon_vma_clone(). They do the same operation.

Quoted 7 lines

We are reusing mm->page_table_lock to serialize, not the *actual* low-level page table locks that really protect PTEs.

With that locking gone, there would be nothing protection vma->anon_vma.

Note that anon_vma_clone() is likely called with the mmap_lock held in write mode, which is not the case for __anon_vma_prepare() ...

Quoted 9 lines
Quoted 6 lines

Yes, anon_vma_clone() is called with the mmap_lock held. I added mmap_assert_write_locked(dst->vm_mm) to prove it.

I added mmap_assert_write_locked(vma->vm_mm) in __anon_vma_prepare() at the same time, it shows __anon_vma_prepare()

is also called with the mmap_lock held too.

Make sure you actually have lockdep built in and enabled.

Quoted 7 lines

This is my config.
CONFIG_LOCKDEP=n
CONFIG_DEBUG_VM=y

I did another test.
I put mmap_assert_write_locked(mm) before 'set_bit(MMF_OOM_SKIP, &mm->flags)' in mmap.c, it's outside the lock. It will crash when on boot. I think mmap_assert_write_locked() works.

If you are changing locks, then please test with lockdep on.

Quoted 3 lines
Quoted 3 lines

__anon_vma_prepare() is for example called from do_anonymous_page() where we might only hold the mmap_lock in read mode (or not at all IIRC with VMA in read mode).

Consider two concurrent readers getting to this function with the same vma. There is no mergeable anon vma, so both create a new anon_vma.

You take the anon_vma lock to parallelize the linking to the vma - but they are different locks because they are both new anon_vma structs allocated by concurrent readers.

You now need a lock that you know cannot allow this to happen. Looking at the top of mm/rmap.c and see which one works. The next in line is the page table lock, so that one is used here.

What if we reverse the locks? We can deadlock.

What if we don't take the anon_vma lock? We can have two writers to the anon_vma.

Thanks,
Liam

author
#6 lore raw
Quoted 92 lines · Liam R. Howlett
Quoted 89 lines · Yajun Deng

April 23, 2024 at 4:18 PM, "David Hildenbrand" <[email protected]> wrote:

Quoted 50 lines · Yajun Deng
Quoted 9 lines · David Hildenbrand
Quoted 1 line
Quoted 1 line

On 22.04.24 12:52, Yajun Deng wrote:

page_table_lock is a lock that for page table, we won't change page

table in __anon_vma_prepare(). As we can see, it works well in

anon_vma_clone(). They do the same operation.

Quoted 6 lines

We are reusing mm->page_table_lock to serialize, not the *actual* low-level page table locks that really protect PTEs.

Quoted 7 lines

With that locking gone, there would be nothing protection vma->anon_vma.

Note that anon_vma_clone() is likely called with the mmap_lock held in write mode, which is not the case for __anon_vma_prepare() ...

Yes, anon_vma_clone() is called with the mmap_lock held. I added mmap_assert_write_locked(dst->vm_mm) to prove it.

Quoted 7 lines

I added mmap_assert_write_locked(vma->vm_mm) in __anon_vma_prepare() at the same time, it shows __anon_vma_prepare()

is also called with the mmap_lock held too.

Make sure you actually have lockdep built in and enabled.

This is my config.

CONFIG_LOCKDEP=n

CONFIG_DEBUG_VM=y

Quoted 9 lines
Quoted 6 lines

I did another test.

I put mmap_assert_write_locked(mm) before 'set_bit(MMF_OOM_SKIP, &mm->flags)' in mmap.c, it's outside the lock.

It will crash when on boot. I think mmap_assert_write_locked() works.

If you are changing locks, then please test with lockdep on.

It's my fault. It shows a warning with lockdep on.

Quoted 30 lines
Quoted 3 lines

__anon_vma_prepare() is for example called from do_anonymous_page() where we might only hold the mmap_lock in read mode (or not at all IIRC with VMA in read mode).

Consider two concurrent readers getting to this function with the same

vma. There is no mergeable anon vma, so both create a new anon_vma.

You take the anon_vma lock to parallelize the linking to the vma - but

they are different locks because they are both new anon_vma structs

allocated by concurrent readers.

You now need a lock that you know cannot allow this to happen. Looking

at the top of mm/rmap.c and see which one works. The next in line is

the page table lock, so that one is used here.

What if we reverse the locks? We can deadlock.

What if we don't take the anon_vma lock? We can have two writers to the

anon_vma.

Thanks,

Liam

#7 lore raw
In reply to Yajun Deng
Quoted 5 lines · Yajun Deng

page_table_lock is a lock that for page table, we won't change page table in __anon_vma_prepare(). As we can see, it works well in anon_vma_clone(). They do the same operation.

Remove unnecessary page_table_lock in __anon_vma_prepare().

IIUC, the page_table_lock here is not to protect page table, but as mentioned in the comments, to prevent concurrent modification by multiple threads.

Quoted 35 lines
---
  mm/rmap.c | 4 ----
  1 file changed, 4 deletions(-)
mm/rmap.c +0 -4
index 2608c40dffad..e894640a9327 100644
@@ -187,7 +187,6 @@ static void anon_vma_chain_link(struct vm_area_struct *vma,
   */
  int __anon_vma_prepare(struct vm_area_struct *vma)
  {
-	struct mm_struct *mm = vma->vm_mm;
  	struct anon_vma *anon_vma, *allocated;
  	struct anon_vma_chain *avc;
 
@@ -208,8 +207,6 @@ int __anon_vma_prepare(struct vm_area_struct *vma)
  	}
 
  	anon_vma_lock_write(anon_vma);
-	/* page_table_lock to protect against threads */
-	spin_lock(&mm->page_table_lock);
  	if (likely(!vma->anon_vma)) {
  		vma->anon_vma = anon_vma;
  		anon_vma_chain_link(vma, avc, anon_vma);
@@ -217,7 +214,6 @@ int __anon_vma_prepare(struct vm_area_struct *vma)
  		allocated = NULL;
  		avc = NULL;
  	}
-	spin_unlock(&mm->page_table_lock);
  	anon_vma_unlock_write(anon_vma);
 
  	if (unlikely(allocated))