Skip to contentSkip to conversation

THP: Use explicit memory barrier

PATCH In review Minchan Kim 6 participants 10 messages Mar 31, 2013 – Apr 4, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 THP: Use explicit memory barrier

__do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set set_pmd_at() write.

But lru_cache_add_lru uses pagevec so it could miss spinlock easily so above rule was broken so user may see inconsistent data.

This patch fixes it with using explict barrier rather than depending on lru spinlock.

Cc Mel Gorman <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Hugh Dickins <[email protected]>
---
 mm/huge_memory.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
mm/huge_memory.c +3 -4
index bfa142e..fad800e 100644
@@ -725,11 +725,10 @@ static int __do_huge_pmd_anonymous_page(struct mm_struct *mm,
 		pmd_t entry;
 		entry = mk_huge_pmd(page, vma);
 		/*
-		 * The spinlocking to take the lru_lock inside
-		 * page_add_new_anon_rmap() acts as a full memory
-		 * barrier to be sure clear_huge_page writes become
-		 * visible after the set_pmd_at() write.
+		 * clear_huge_page write become visible after the
+		 * set_pmd_at() write.
 		 */
+		smp_wmb();
 		page_add_new_anon_rmap(page, vma, haddr);
 		set_pmd_at(mm, haddr, pmd, entry);
 		pgtable_trans_huge_deposit(mm, pgtable);
Signature · 7 lines
1.8.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#1 lore raw
Quoted 9 lines · Minchan Kim

__do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set set_pmd_at() write.

But lru_cache_add_lru uses pagevec so it could miss spinlock easily so above rule was broken so user may see inconsistent data. This patch fixes it with using explict barrier rather than depending on lru spinlock.

Hmm...how about do_anonymous_page() ? there are no comments/locks/barriers. Users can see non-zero value after page fault in theory ?

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#2 lore raw

Hey Kame,

Quoted 5 lines · Kamezawa Hiroyuki
Quoted 4 lines · Minchan Kim

__do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set set_pmd_at() write.

Quoted 8 lines
Quoted 5 lines

But lru_cache_add_lru uses pagevec so it could miss spinlock easily so above rule was broken so user may see inconsistent data. This patch fixes it with using explict barrier rather than depending on lru spinlock.

Hmm...how about do_anonymous_page() ? there are no comments/locks/barriers. Users can see non-zero value after page fault in theory ?

Maybe, but as you know well, we didn't see any report about that until now so I'm not sure. Ccing people in this thread could have clear answer rather than me.

In THP, I just found inconsistency between Andrea's comment and code. That's why I sent a patch.

If your concern turns out truth, Ya, you might find ancient bug. :) Thanks.

Quoted 9 lines

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#3 lore raw
In reply to Minchan Kim
Quoted 10 lines · Minchan Kim

__do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set set_pmd_at() write.

But lru_cache_add_lru uses pagevec so it could miss spinlock easily so above rule was broken so user may see inconsistent data.

This patch fixes it with using explict barrier rather than depending on lru spinlock.

Is this the same issue that Andrea responded to in the "thp and memory barrier assumptions" thread at http://marc.info/?t=134333512700004 ?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#4 lore raw
Quoted 9 lines · David Rientjes
Quoted 7 lines · Minchan Kim

__do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set set_pmd_at() write.

But lru_cache_add_lru uses pagevec so it could miss spinlock easily so above rule was broken so user may see inconsistent data.

Quoted 6 lines
Quoted 3 lines

This patch fixes it with using explict barrier rather than depending on lru spinlock.

Is this the same issue that Andrea responded to in the "thp and memory barrier assumptions" thread at http://marc.info/?t=134333512700004 ?

Yes and Peter pointed out further step.
Thanks for pointing out.
Not that I know that Andrea alreay noticed it, I don't care about this patch.

Remaining question is Kame's one.
Isn't there anyone could answer it?

Quoted 6 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#5 lore raw
Quoted 5 lines · Minchan Kim

Yes and Peter pointed out further step.
Thanks for pointing out.
Not that I know that Andrea alreay noticed it, I don't care about this patch.

Andrea, do you have time to send

c08e0c9ee786 ("thp: add memory barrier to __do_huge_pmd_anonymous_page") b08d75a595ec ("thp: document barrier() in wrprotect THP fault path")

from the master branch of aa.git to Andrew? I would do it, but one isn't signed-off in your tree.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#6 lore raw
In reply to Minchan Kim
Quoted 22 lines · Minchan Kim
Quoted 9 lines · David Rientjes
Quoted 7 lines · Minchan Kim

__do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set set_pmd_at() write.

But lru_cache_add_lru uses pagevec so it could miss spinlock easily so above rule was broken so user may see inconsistent data.

Quoted 6 lines
Quoted 3 lines

This patch fixes it with using explict barrier rather than depending on lru spinlock.

Is this the same issue that Andrea responded to in the "thp and memory barrier assumptions" thread at http://marc.info/?t=134333512700004 ?

Yes and Peter pointed out further step.
Thanks for pointing out.
Not that I know that Andrea alreay noticed it, I don't care about this patch.

Quoted 4 lines

Remaining question is Kame's one.

Quoted 2 lines

Hmm...how about do_anonymous_page() ? there are no comments/locks/barriers. Users can see non-zero value after page fault in theory ?

Isn't there anyone could answer it?

See Nick's 2008 0ed361dec "mm: fix PageUptodate data race", which gave us

static inline void __SetPageUptodate(struct page *page)

{
	smp_wmb();
	__set_bit(PG_uptodate, &(page)->flags);
}

So both do_anonymous_page() and __do_huge_pmd_anonymous_page() look safe to me already, though the huge_memory one could do with a fixed comment.

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#7 lore raw
Quoted 30 lines · Hugh Dickins
Quoted 22 lines · Minchan Kim
Quoted 9 lines · David Rientjes
Quoted 7 lines · Minchan Kim

__do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set set_pmd_at() write.

But lru_cache_add_lru uses pagevec so it could miss spinlock easily so above rule was broken so user may see inconsistent data.

Quoted 6 lines
Quoted 3 lines

This patch fixes it with using explict barrier rather than depending on lru spinlock.

Is this the same issue that Andrea responded to in the "thp and memory barrier assumptions" thread at http://marc.info/?t=134333512700004 ?

Yes and Peter pointed out further step.
Thanks for pointing out.
Not that I know that Andrea alreay noticed it, I don't care about this patch.

Quoted 4 lines

Remaining question is Kame's one.

Quoted 2 lines

Hmm...how about do_anonymous_page() ? there are no comments/locks/barriers. Users can see non-zero value after page fault in theory ?

Isn't there anyone could answer it?

See Nick's 2008 0ed361dec "mm: fix PageUptodate data race", which gave us

Quoted 8 lines

static inline void __SetPageUptodate(struct page *page)

{
	smp_wmb();
	__set_bit(PG_uptodate, &(page)->flags);
}

So both do_anonymous_page() and __do_huge_pmd_anonymous_page() look safe to me already, though the huge_memory one could do with a fixed comment.

Thanks you very much!
That's one everybody are really missing.

Here it goes!

==================== 8< =====================

#8 lore raw

Review

Quoted 4 lines · Minchan Kim
(quoted diff) +3 -0
 	clear_huge_page(page, haddr, HPAGE_PMD_NR);
+	/*
+	 * The memory barrier inside __SetPageUptodate makes sure that
+	 * clear_huge_page writes become visible after the set_pmd_at()

s/after/before/

Quoted 23 lines
(quoted diff) +2 -6
+	 * write.
+	 */
 	__SetPageUptodate(page);
 
 	spin_lock(&mm->page_table_lock);
@@ -724,12 +729,6 @@ static int __do_huge_pmd_anonymous_page(struct mm_struct *mm,
 	} else {
 		pmd_t entry;
 		entry = mk_huge_pmd(page, vma);
-		/*
-		 * The spinlocking to take the lru_lock inside
-		 * page_add_new_anon_rmap() acts as a full memory
-		 * barrier to be sure clear_huge_page writes become
-		 * visible after the set_pmd_at() write.
-		 */
 		page_add_new_anon_rmap(page, vma, haddr);
 		set_pmd_at(mm, haddr, pmd, entry);
 		pgtable_trans_huge_deposit(mm, pgtable);
mm/memory.c +0 -0
index 494526a..d0da51e 100644

mm/memory.c static int do_anonymous_page(struct mm_struct *mm, struct vm_area_struct *vma, · line 3203

Quoted 8 lines
mm/memory.c +5 -0
@@ -3196,6 +3196,11 @@ static int do_anonymous_page(struct mm_struct *mm, struct vm_area_struct *vma,
 	page = alloc_zeroed_user_highpage_movable(vma, address);
 	if (!page)
 		goto oom;
+	/*
+	 * The memory barrier inside __SetPageUptodate makes sure that
+	 * preceeding stores to the page contents become visible after
+	 * the set_pte_at() write.
+	 */

s/after/before/

After the above correction it looks nice cleanup, thanks!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#9 lore raw
In reply to Minchan Kim

Hi Minchan,

Quoted 3 lines · Minchan Kim

__do_huge_pmd_anonymous_page depends on page_add_new_anon_rmap's spinlock for making sure that clear_huge_page write become visible after set set_pmd_at() write.

1. There are no pte modify, why take page_table_lock here?
2. What's the meaning of "clear_huge_page write become visible after set set_pmd_at() write"?

Quoted 24 lines

But lru_cache_add_lru uses pagevec so it could miss spinlock easily so above rule was broken so user may see inconsistent data.

This patch fixes it with using explict barrier rather than depending on lru spinlock.

Cc Mel Gorman <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Hugh Dickins <[email protected]>
---
  mm/huge_memory.c | 7 +++----
  1 file changed, 3 insertions(+), 4 deletions(-)
mm/huge_memory.c +0 -1
index bfa142e..fad800e 100644
@@ -725,11 +725,10 @@ static int __do_huge_pmd_anonymous_page(struct mm_struct *mm,
  		pmd_t entry;
  		entry = mk_huge_pmd(page, vma);
  		/*
-		 * The spinlocking to take the lru_lock inside

mm/huge_memory.c static int __do_huge_pmd_anonymous_page(struct mm_struct *mm, · line 734

Quoted 10 lines
mm/huge_memory.c +3 -3
-		 * page_add_new_anon_rmap() acts as a full memory
-		 * barrier to be sure clear_huge_page writes become
-		 * visible after the set_pmd_at() write.
+		 * clear_huge_page write become visible after the
+		 * set_pmd_at() write.
  		 */
+		smp_wmb();
  		page_add_new_anon_rmap(page, vma, haddr);
  		set_pmd_at(mm, haddr, pmd, entry);
  		pgtable_trans_huge_deposit(mm, pgtable);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>