Skip to contentSkip to conversation

mm: Fix possible PMD dirty bit lost in set_pmd_migration_entry()

PATCH Applied Huang, Ying 5 participants 5 messages Feb 20, 2020 – Feb 21, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm: Fix possible PMD dirty bit lost in set_pmd_migration_entry()

From: Huang Ying <[email protected]>

In set_pmd_migration_entry(), pmdp_invalidate() is used to change PMD atomically. But the PMD is read before that with an ordinary memory reading. If the THP (transparent huge page) is written between the PMD reading and pmdp_invalidate(), the PMD dirty bit may be lost, and cause data corruption. The race window is quite small, but still possible in theory, so need to be fixed.

The race is fixed via using the return value of pmdp_invalidate() to get the original content of PMD, which is a read/modify/write atomic operation. So no THP writing can occur in between.

The race has been introduced when the THP migration support is added in the commit 616b8371539a ("mm: thp: enable thp migration in generic path"). But this fix depends on the commit d52605d7cb30 ("mm: do not lose dirty and accessed bits in pmdp_invalidate()"). So it's easy to be backported after v4.16. But the race window is really small, so it may be fine not to backport the fix at all.

Cc Zi Yan <[email protected]>
Cc Kirill A. Shutemov <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 mm/huge_memory.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
mm/huge_memory.c +1 -2
index 580098e115bd..b1e069e68189 100644
@@ -3060,8 +3060,7 @@ void set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw,
 		return;
 
 	flush_cache_range(vma, address, address + HPAGE_PMD_SIZE);
-	pmdval = *pvmw->pmd;
-	pmdp_invalidate(vma, address, pvmw->pmd);
+	pmdval = pmdp_invalidate(vma, address, pvmw->pmd);
 	if (pmd_dirty(pmdval))
 		set_page_dirty(page);
 	entry = make_migration_entry(page, pmd_write(pmdval));
Signature · 1 line
2.25.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 4 lines · Ying

Looks good to me.

Quoted 27 lines
Cc Zi Yan <[email protected]>
Cc Kirill A. Shutemov <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>

---
mm/huge_memory.c | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)
mm/huge_memory.c +1 -2
index 580098e115bd..b1e069e68189 100644
@@ -3060,8 +3060,7 @@ void set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw,
 		return;
 
 	flush_cache_range(vma, address, address + HPAGE_PMD_SIZE);
-	pmdval = *pvmw->pmd;
-	pmdp_invalidate(vma, address, pvmw->pmd);
+	pmdval = pmdp_invalidate(vma, address, pvmw->pmd);
 	if (pmd_dirty(pmdval))
 		set_page_dirty(page);
 	entry = make_migration_entry(page, pmd_write(pmdval));
Signature · 1 line
2.25.0
Acked-by on v1 1/1
#2 lore raw
In reply to Huang, Ying
Quoted 20 lines · Ying

From: Huang Ying <[email protected]>

In set_pmd_migration_entry(), pmdp_invalidate() is used to change PMD atomically. But the PMD is read before that with an ordinary memory reading. If the THP (transparent huge page) is written between the PMD reading and pmdp_invalidate(), the PMD dirty bit may be lost, and cause data corruption. The race window is quite small, but still possible in theory, so need to be fixed.

The race is fixed via using the return value of pmdp_invalidate() to get the original content of PMD, which is a read/modify/write atomic operation. So no THP writing can occur in between.

The race has been introduced when the THP migration support is added in the commit 616b8371539a ("mm: thp: enable thp migration in generic path"). But this fix depends on the commit d52605d7cb30 ("mm: do not lose dirty and accessed bits in pmdp_invalidate()"). So it's easy to be backported after v4.16. But the race window is really small, so it may be fine not to backport the fix at all.

Quoted 6 lines
Cc Zi Yan <[email protected]>
Cc Kirill A. Shutemov <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>
Signature · 1 line
 Kirill A. Shutemov
reviewer Reviewed-by on v1 1/1
#3 lore raw
In reply to Huang, Ying
Quoted 45 lines · Ying

From: Huang Ying <[email protected]>

In set_pmd_migration_entry(), pmdp_invalidate() is used to change PMD atomically. But the PMD is read before that with an ordinary memory reading. If the THP (transparent huge page) is written between the PMD reading and pmdp_invalidate(), the PMD dirty bit may be lost, and cause data corruption. The race window is quite small, but still possible in theory, so need to be fixed.

The race is fixed via using the return value of pmdp_invalidate() to get the original content of PMD, which is a read/modify/write atomic operation. So no THP writing can occur in between.

The race has been introduced when the THP migration support is added in the commit 616b8371539a ("mm: thp: enable thp migration in generic path"). But this fix depends on the commit d52605d7cb30 ("mm: do not lose dirty and accessed bits in pmdp_invalidate()"). So it's easy to be backported after v4.16. But the race window is really small, so it may be fine not to backport the fix at all.

Cc Zi Yan <[email protected]>
Cc Kirill A. Shutemov <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 mm/huge_memory.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
mm/huge_memory.c +1 -3
index 580098e115bd..b1e069e68189 100644
@@ -3060,8 +3060,7 @@ void set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw,
                return;
 
        flush_cache_range(vma, address, address + HPAGE_PMD_SIZE);
-       pmdval = *pvmw->pmd;
-       pmdp_invalidate(vma, address, pvmw->pmd);
+       pmdval = pmdp_invalidate(vma, address, pvmw->pmd);
        if (pmd_dirty(pmdval))
                set_page_dirty(page);
        entry = make_migration_entry(page, pmd_write(pmdval));
--
Quoted 1 line

2.25.0

Looks good to me. Thanks.

—
Best Regards,
Yan Zi

maintainer
#4 lore raw
In reply to Huang, Ying
Quoted 13 lines · Ying

From: Huang Ying <[email protected]>

In set_pmd_migration_entry(), pmdp_invalidate() is used to change PMD atomically. But the PMD is read before that with an ordinary memory reading. If the THP (transparent huge page) is written between the PMD reading and pmdp_invalidate(), the PMD dirty bit may be lost, and cause data corruption. The race window is quite small, but still possible in theory, so need to be fixed.

The race is fixed via using the return value of pmdp_invalidate() to get the original content of PMD, which is a read/modify/write atomic operation. So no THP writing can occur in between.

Quoted 6 lines

The race has been introduced when the THP migration support is added in the commit 616b8371539a ("mm: thp: enable thp migration in generic path"). But this fix depends on the commit d52605d7cb30 ("mm: do not lose dirty and accessed bits in pmdp_invalidate()"). So it's easy to be backported after v4.16. But the race window is really small, so it may be fine not to backport the fix at all.

Thanks. I'm inclined to add a cc:stable to this one. Silent data corruption is pretty serious.