mm/page_alloc: fix freeing of MIGRATE_RESERVE migratetype pages

Pages allocated from MIGRATE_RESERVE migratetype pageblocks are not freed back to MIGRATE_RESERVE migratetype free lists in free_pcppages_bulk()->__free_one_page() if we got to free_pcppages_bulk() through drain_[zone_]pages(). The freeing through free_hot_cold_page() is okay because freepage migratetype is set to pageblock migratetype before calling free_pcppages_bulk().

If pages of MIGRATE_RESERVE
migratetype end up on the free lists of other migratetype whole Reserved pageblock may be later changed to the other migratetype in __rmqueue_fallback() and it will be never changed back to be a Reserved pageblock. Fix the issue by moving freepage migratetype setting from rmqueue_bulk() to __rmqueue[_fallback]() and preserving freepage migratetype as an original pageblock migratetype for MIGRATE_RESERVE migratetype pages.

The problem was introduced in v2.6.31 by commit ed0ae21 ("page allocator: do not call get_pageblock_migratetype() more than necessary").

Signed-off-by Bartlomiej Zolnierkiewicz <[email protected]>

Reported-by Yong-Taek Lee <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

---
v2:
- updated patch description, there is no __zone_pcp_update() in newer kernels
v3:
- set freepage migratetype in __rmqueue[_fallback]() instead of rmqueue_bulk() (per Mel's request)

  mm/page_alloc.c |   27 ++++++++++++++++++---------
  1 file changed, 18 insertions(+), 9 deletions(-)

Index: b/mm/page_alloc.c
===================================================================

mm/page_alloc.c +8 -0

  	/* Find the largest possible block of pages in the other list */
  	for (current_order = MAX_ORDER-1; current_order >= order;

@@ -1125,6 +1125,14 @@ __rmqueue_fallback(struct zone *zone, in
  			expand(zone, page, order, current_order, area,
  			       new_type);
 
+			if (IS_ENABLED(CONFIG_CMA)) {
+				mt = get_pageblock_migratetype(page);
+				if (!is_migrate_cma(mt) &&
+				    !is_migrate_isolate(mt))
+					mt = start_migratetype;
+			}
+			set_freepage_migratetype(page, mt);
+
  			trace_mm_page_alloc_extfrag(page, order, current_order,
  				start_migratetype, migratetype, new_type);
 

mm/page_alloc.c +7 -8

+	} else if (migratetype != MIGRATE_RESERVE) {
  		page = __rmqueue_fallback(zone, order, migratetype);
 
  		/*

@@ -1174,7 +1184,7 @@ static int rmqueue_bulk(struct zone *zon
  			unsigned long count, struct list_head *list,
  			int migratetype, int cold)
  {
-	int mt = migratetype, i;
+	int i;
 
  	spin_lock(&zone->lock);
  	for (i = 0; i < count; ++i) {

@@ -1195,16 +1205,15 @@ static int rmqueue_bulk(struct zone *zon
  			list_add(&page->lru, list);
  		else
  			list_add_tail(&page->lru, list);
+		list = &page->lru;
  		if (IS_ENABLED(CONFIG_CMA)) {
-			mt = get_pageblock_migratetype(page);
+			int mt = get_pageblock_migratetype(page);
  			if (!is_migrate_cma(mt) && !is_migrate_isolate(mt))
  				mt = migratetype;
+			if (is_migrate_cma(mt))
+				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+						      -(1 << order));
  		}
-		set_freepage_migratetype(page, mt);
-		list = &page->lru;
-		if (is_migrate_cma(mt))
-			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
-					      -(1 << order));
  	}
  	__mod_zone_page_state(zone, NR_FREE_PAGES, -(i << order));
  	spin_unlock(&zone->lock);
 
--

To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 7 lines · Bartlomiej Zolnierkiewicz

Pages allocated from MIGRATE_RESERVE migratetype pageblocks are not freed back to MIGRATE_RESERVE migratetype free lists in free_pcppages_bulk()->__free_one_page() if we got to free_pcppages_bulk() through drain_[zone_]pages(). The freeing through free_hot_cold_page() is okay because freepage migratetype is set to pageblock migratetype before calling free_pcppages_bulk().

I think this is somewhat misleading and got me confused for a while. It's not about the call path of free_pcppages_bulk(), but about the fact that rmqueue_bulk() has been called at some point to fill up the pcp lists, and had to resort to __rmqueue_fallback(). So, going through free_hot_cold_page() might give you correct migratetype for the last page freed, but the pcp lists may still contain misplaced pages from earlier rmqueue_bulk().

Quoted 9 lines

If pages of MIGRATE_RESERVE
migratetype end up on the free lists of other migratetype whole Reserved pageblock may be later changed to the other migratetype in __rmqueue_fallback() and it will be never changed back to be a Reserved pageblock. Fix the issue by moving freepage migratetype setting from rmqueue_bulk() to __rmqueue[_fallback]() and preserving freepage migratetype as an original pageblock migratetype for MIGRATE_RESERVE migratetype pages.

Actually wouldn't the easiest solution to this particular problem to check current pageblock migratetype in try_to_steal_freepages() and disallow changing it. However I agree that preventing the misplaced page in the first place would be even better.

Quoted 20 lines

The problem was introduced in v2.6.31 by commit ed0ae21 ("page allocator: do not call get_pageblock_migratetype() more than necessary").

Signed-off-by Bartlomiej Zolnierkiewicz <[email protected]>

Reported-by Yong-Taek Lee <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

---
v2:
- updated patch description, there is no __zone_pcp_update() in newer kernels
v3:
- set freepage migratetype in __rmqueue[_fallback]() instead of rmqueue_bulk() (per Mel's request)

  mm/page_alloc.c |   27 ++++++++++++++++++---------
  1 file changed, 18 insertions(+), 9 deletions(-)

mm/page_alloc.c __rmqueue_fallback(struct zone *zone, in · line 1097

Quoted 10 lines

Index: b/mm/page_alloc.c
===================================================================

mm/page_alloc.c +1 -1

@@ -1094,7 +1094,7 @@ __rmqueue_fallback(struct zone *zone, in
  	struct free_area *area;
  	int current_order;
  	struct page *page;
-	int migratetype, new_type, i;
+	int migratetype, new_type, mt = start_migratetype, i;

A better naming would help, "mt" and "migratetype" are the same thing and it gets too confusing.

Quoted 22 lines

mm/page_alloc.c +8 -0

  	/* Find the largest possible block of pages in the other list */
  	for (current_order = MAX_ORDER-1; current_order >= order;

@@ -1125,6 +1125,14 @@ __rmqueue_fallback(struct zone *zone, in
  			expand(zone, page, order, current_order, area,
  			       new_type);
 
+			if (IS_ENABLED(CONFIG_CMA)) {
+				mt = get_pageblock_migratetype(page);
+				if (!is_migrate_cma(mt) &&
+				    !is_migrate_isolate(mt))
+					mt = start_migratetype;
+			}
+			set_freepage_migratetype(page, mt);
+
  			trace_mm_page_alloc_extfrag(page, order, current_order,
  				start_migratetype, migratetype, new_type);
 

@@ -1147,7 +1155,9 @@ static struct page *__rmqueue(struct zon
  retry_reserve:
  	page = __rmqueue_smallest(zone, order, migratetype);
 

mm/page_alloc.c static struct page *__rmqueue(struct zon · line 1159

Quoted 3 lines

mm/page_alloc.c +2 -1

-	if (unlikely(!page) && migratetype != MIGRATE_RESERVE) {
+	if (likely(page)) {
+		set_freepage_migratetype(page, migratetype);

Are you sure that here the checking of of CMA and ISOLATE is not needed?

Did the original rmqueue_bulk() have this checking only for the __rmqueue_fallback() case? Why wouldn't the check already be only in __rmqueue_fallback() then?

Quoted 27 lines

mm/page_alloc.c +7 -2

+	} else if (migratetype != MIGRATE_RESERVE) {
  		page = __rmqueue_fallback(zone, order, migratetype);
 
  		/*

@@ -1174,7 +1184,7 @@ static int rmqueue_bulk(struct zone *zon
  			unsigned long count, struct list_head *list,
  			int migratetype, int cold)
  {
-	int mt = migratetype, i;
+	int i;
 
  	spin_lock(&zone->lock);
  	for (i = 0; i < count; ++i) {

@@ -1195,16 +1205,15 @@ static int rmqueue_bulk(struct zone *zon
  			list_add(&page->lru, list);
  		else
  			list_add_tail(&page->lru, list);
+		list = &page->lru;
  		if (IS_ENABLED(CONFIG_CMA)) {
-			mt = get_pageblock_migratetype(page);
+			int mt = get_pageblock_migratetype(page);
  			if (!is_migrate_cma(mt) && !is_migrate_isolate(mt))
  				mt = migratetype;
+			if (is_migrate_cma(mt))
+				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+						      -(1 << order));
  		}

mm/page_alloc.c static int rmqueue_bulk(struct zone *zon · line 1219

Quoted 8 lines

mm/page_alloc.c +0 -5

-		set_freepage_migratetype(page, mt);
-		list = &page->lru;
-		if (is_migrate_cma(mt))
-			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
-					      -(1 << order));
  	}
  	__mod_zone_page_state(zone, NR_FREE_PAGES, -(i << order));
  	spin_unlock(&zone->lock);

Hi,

Quoted 9 lines · Vlastimil Babka

Quoted 7 lines · Bartlomiej Zolnierkiewicz

Pages allocated from MIGRATE_RESERVE migratetype pageblocks are not freed back to MIGRATE_RESERVE migratetype free lists in free_pcppages_bulk()->__free_one_page() if we got to free_pcppages_bulk() through drain_[zone_]pages(). The freeing through free_hot_cold_page() is okay because freepage migratetype is set to pageblock migratetype before calling free_pcppages_bulk().

Quoted 7 lines

I think this is somewhat misleading and got me confused for a while. It's not about the call path of free_pcppages_bulk(), but about the fact that rmqueue_bulk() has been called at some point to fill up the pcp lists, and had to resort to __rmqueue_fallback(). So, going through free_hot_cold_page() might give you correct migratetype for the last page freed, but the pcp lists may still contain misplaced pages from earlier rmqueue_bulk().

Ok, you're right. I'll fix this.

Quoted 35 lines

Quoted 9 lines

If pages of MIGRATE_RESERVE
migratetype end up on the free lists of other migratetype whole Reserved pageblock may be later changed to the other migratetype in __rmqueue_fallback() and it will be never changed back to be a Reserved pageblock. Fix the issue by moving freepage migratetype setting from rmqueue_bulk() to __rmqueue[_fallback]() and preserving freepage migratetype as an original pageblock migratetype for MIGRATE_RESERVE migratetype pages.

Actually wouldn't the easiest solution to this particular problem to check current pageblock migratetype in try_to_steal_freepages() and disallow changing it. However I agree that preventing the misplaced page in the first place would be even better.

Quoted 20 lines

The problem was introduced in v2.6.31 by commit ed0ae21 ("page allocator: do not call get_pageblock_migratetype() more than necessary").

Signed-off-by Bartlomiej Zolnierkiewicz <[email protected]>

Reported-by Yong-Taek Lee <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

---
v2:
- updated patch description, there is no __zone_pcp_update() in newer kernels
v3:
- set freepage migratetype in __rmqueue[_fallback]() instead of rmqueue_bulk() (per Mel's request)

   mm/page_alloc.c |   27 ++++++++++++++++++---------
   1 file changed, 18 insertions(+), 9 deletions(-)

Quoted 13 lines

mm/page_alloc.c __rmqueue_fallback(struct zone *zone, in · line 1097

Quoted 10 lines

Index: b/mm/page_alloc.c
===================================================================

mm/page_alloc.c +1 -1

@@ -1094,7 +1094,7 @@ __rmqueue_fallback(struct zone *zone, in
   	struct free_area *area;
   	int current_order;
   	struct page *page;
-	int migratetype, new_type, i;
+	int migratetype, new_type, mt = start_migratetype, i;

A better naming would help, "mt" and "migratetype" are the same thing and it gets too confusing.

Well, yes, though 'mt' is short and the check code is consistent with the corresponding code in rmqueue_bulk().

Do you have a proposal for a better name for this variable?

Quoted 22 lines

Quoted 22 lines

mm/page_alloc.c +8 -0

   	/* Find the largest possible block of pages in the other list */
   	for (current_order = MAX_ORDER-1; current_order >= order;

@@ -1125,6 +1125,14 @@ __rmqueue_fallback(struct zone *zone, in
   			expand(zone, page, order, current_order, area,
   			       new_type);
 
+			if (IS_ENABLED(CONFIG_CMA)) {
+				mt = get_pageblock_migratetype(page);
+				if (!is_migrate_cma(mt) &&
+				    !is_migrate_isolate(mt))
+					mt = start_migratetype;
+			}
+			set_freepage_migratetype(page, mt);
+
   			trace_mm_page_alloc_extfrag(page, order, current_order,
   				start_migratetype, migratetype, new_type);
 

@@ -1147,7 +1155,9 @@ static struct page *__rmqueue(struct zon
   retry_reserve:
   	page = __rmqueue_smallest(zone, order, migratetype);
 

Quoted 5 lines

mm/page_alloc.c static struct page *__rmqueue(struct zon · line 1159

Quoted 3 lines

mm/page_alloc.c +2 -1

-	if (unlikely(!page) && migratetype != MIGRATE_RESERVE) {
+	if (likely(page)) {
+		set_freepage_migratetype(page, migratetype);

Are you sure that here the checking of of CMA and ISOLATE is not needed?

CMA and ISOLATE migratetype pages are always put back on the correct free lists (since set_freepage_migratetype() sets freepage migratetype to the original one for CMA and ISOLATE migratetype pages) and __rmqueue_smallest() can take page only from the 'migratetype' free list.

+ It was suggested to do it this way by Mel.

Quoted 3 lines

Did the original rmqueue_bulk() have this checking only for the __rmqueue_fallback() case? Why wouldn't the check already be only in __rmqueue_fallback() then?

Probably because of historical reasons. The rmqueue_bulk() contained set_page_private() call when CMA was introduced and added the special handling for CMA and ISOLATE migratetype pages, please see commit 47118af ("mm: mmzone: MIGRATE_CMA migration type added").

Quoted 27 lines

Quoted 27 lines

mm/page_alloc.c +7 -2

+	} else if (migratetype != MIGRATE_RESERVE) {
   		page = __rmqueue_fallback(zone, order, migratetype);
 
   		/*

@@ -1174,7 +1184,7 @@ static int rmqueue_bulk(struct zone *zon
   			unsigned long count, struct list_head *list,
   			int migratetype, int cold)
   {
-	int mt = migratetype, i;
+	int i;
 
   	spin_lock(&zone->lock);
   	for (i = 0; i < count; ++i) {

@@ -1195,16 +1205,15 @@ static int rmqueue_bulk(struct zone *zon
   			list_add(&page->lru, list);
   		else
   			list_add_tail(&page->lru, list);
+		list = &page->lru;
   		if (IS_ENABLED(CONFIG_CMA)) {
-			mt = get_pageblock_migratetype(page);
+			int mt = get_pageblock_migratetype(page);
   			if (!is_migrate_cma(mt) && !is_migrate_isolate(mt))
   				mt = migratetype;
+			if (is_migrate_cma(mt))
+				__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
+						      -(1 << order));
   		}

Quoted 8 lines

mm/page_alloc.c static int rmqueue_bulk(struct zone *zon · line 1219

Quoted 8 lines

mm/page_alloc.c +0 -5

-		set_freepage_migratetype(page, mt);
-		list = &page->lru;
-		if (is_migrate_cma(mt))
-			__mod_zone_page_state(zone, NR_FREE_CMA_PAGES,
-					      -(1 << order));
   	}
   	__mod_zone_page_state(zone, NR_FREE_PAGES, -(i << order));
   	spin_unlock(&zone->lock);

Best regards,
--
Bartlomiej Zolnierkiewicz
Samsung R&D Institute Poland
Samsung Electronics

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

mm/page_alloc.c +2 -0

index 2dbaba1..0ee9f8c 100644
@@ -697,6 +697,8 @@ static void free_pcppages_bulk(struct zone *zone, int count,
 			page = list_entry(list->prev, struct page, lru);
 			/* must delete as __free_one_page list manipulates */
 			list_del(&page->lru);
+
+			VM_BUG_ON(!check_freepage_migratetype(page));
 			mt = get_freepage_migratetype(page);
 			/* MIGRATE_MOVABLE list may include MIGRATE_RESERVEs */
 			__free_one_page(page, zone, 0, mt);

Quoted 12 lines · Vlastimil Babka

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

And second:

[snip]

Quoted 19 lines

mm/page_alloc.c +3 -0

index 2dbaba1..0ee9f8c 100644
@@ -697,6 +697,8 @@ static void free_pcppages_bulk(struct zone *zone, int count,
  			page = list_entry(list->prev, struct page, lru);
  			/* must delete as __free_one_page list manipulates */
  			list_del(&page->lru);
+
+			VM_BUG_ON(!check_freepage_migratetype(page));
  			mt = get_freepage_migratetype(page);
  			/* MIGRATE_MOVABLE list may include MIGRATE_RESERVEs */
  			__free_one_page(page, zone, 0, mt);

@@ -1190,6 +1192,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order,
  		struct page *page = __rmqueue(zone, order, migratetype);
  		if (unlikely(page == NULL))
  			break;
+		VM_BUG_ON(!check_freepage_migratetype(page));
 

mm/page_alloc.c static int rmqueue_bulk(struct zone *zone, unsigned int order, · line 1198

Quoted 3 lines

mm/page_alloc.c +0 -0

  		/*
  		 * Split buddy pages returned by expand() are received here

Could the VM_BUG_ON()s be VM_BUG_ON_PAGE() instead?

Quoted 81 lines · Sasha Levin

Quoted 81 lines · Vlastimil Babka

Quoted 12 lines

Quoted 12 lines

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

Quoted 8 lines

Quoted 8 lines

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

Hey, that's not an issue, that means the patch works as intended :) And I believe it's not a bug introduced by PATCH 1/2.

So, according to my decodecode reading, RAX is the results of get_pageblock_migratetype() and it's MIGRATE_RESERVE. RDX is the result of get_freepage_migratetype() and it's MIGRATE_UNMOVABLE. The freepage_migratetype has just been set either by __rmqueue_smallest() or __rmqueue_fallback(), according to the free_list the page has been taken from. So this looks like a page from MIGRATE_RESERVE pageblock found on the !MIGRATE_RESERVE free_list, which is exactly what the patch intends to catch.

I think there are two possible explanations.

1) the pageblock is genuinely MIGRATE_RESERVE and it was misplaced by mistake. I think it wasn't in free_pcppages_bulk() as there's the same VM_BUG_ON which would supposedly trigger at the moment of displacing. In theory it's possible that there's a race through __free_pages_ok() -> free_one_page() where the get_pageblock_migratetype() in __free_pages_ok() would race with set_pageblock_migratetype() and result in bogus value. But nobody should be calling set_pageblock_migratetype() on a MIGRATE_RESERVE pageblock.

2) the pageblock was marked as MIGRATE_RESERVE due to a race between set_pageblock_migratetype() and set_pageblock_skip(). The latter is currently not serialized by zone->lock, nor it uses atomic bit set. So it may result in lost updates in a racing set_pageblock_migratetype(). I think a well-placed race when changing pageblock from MIGRATE_MOVABLE to MIGRATE_RECLAIMABLE could result in MIGRATE_RESERVE value. Similar races have been already observed to be a problem where frequent changing to/from MIGRATE_ISOLATE is involved, and I did a patch series to address this, but it was not complete and I postponed it after Mel's changes that remove the racy for-cycles completely. So it might be that his "[PATCH 08/17] mm: page_alloc: Use word-based accesses for get/set pageblock bitmaps" already solves this bug (but maybe only on certain architectures where you don't need atomic operations). You might try that patch if you can reproduce this bug frequently enough?

Quoted 113 lines · Vlastimil Babka

Quoted 81 lines · Sasha Levin

Quoted 81 lines · Vlastimil Babka

Quoted 12 lines

Quoted 12 lines

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

Quoted 8 lines

Quoted 8 lines

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

Hey, that's not an issue, that means the patch works as intended :) And I believe it's not a bug introduced by PATCH 1/2.

So, according to my decodecode reading, RAX is the results of get_pageblock_migratetype() and it's MIGRATE_RESERVE. RDX is the result of get_freepage_migratetype() and it's MIGRATE_UNMOVABLE. The freepage_migratetype has just been set either by __rmqueue_smallest() or __rmqueue_fallback(), according to the free_list the page has been taken from. So this looks like a page from MIGRATE_RESERVE pageblock found on the !MIGRATE_RESERVE free_list, which is exactly what the patch intends to catch.

I think there are two possible explanations.

1) the pageblock is genuinely MIGRATE_RESERVE and it was misplaced by mistake. I think it wasn't in free_pcppages_bulk() as there's the same VM_BUG_ON which would supposedly trigger at the moment of displacing. In theory it's possible that there's a race through __free_pages_ok() -> free_one_page() where the get_pageblock_migratetype() in __free_pages_ok() would race with set_pageblock_migratetype() and result in bogus value. But nobody should be calling set_pageblock_migratetype() on a MIGRATE_RESERVE pageblock.

Quoted 14 lines

2) the pageblock was marked as MIGRATE_RESERVE due to a race between set_pageblock_migratetype() and set_pageblock_skip(). The latter is currently not serialized by zone->lock, nor it uses atomic bit set. So it may result in lost updates in a racing set_pageblock_migratetype(). I think a well-placed race when changing pageblock from MIGRATE_MOVABLE to MIGRATE_RECLAIMABLE could result in MIGRATE_RESERVE value. Similar races have been already observed to be a problem where frequent changing to/from MIGRATE_ISOLATE is involved, and I did a patch series to address this, but it was not complete and I postponed it after Mel's changes that remove the racy for-cycles completely. So it might be that his "[PATCH 08/17] mm: page_alloc: Use word-based accesses for get/set pageblock bitmaps" already solves this bug (but maybe only on certain architectures where you don't need atomic operations). You might try that patch if you can reproduce this bug frequently enough?

I've tried that patch, but still see the same BUG_ON.

Thanks,
Sasha

Quoted 129 lines · Sasha Levin

Quoted 127 lines · Vlastimil Babka

Quoted 81 lines · Sasha Levin

Quoted 81 lines · Vlastimil Babka

Quoted 12 lines

Quoted 12 lines

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

Quoted 8 lines

Quoted 8 lines

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

Hey, that's not an issue, that means the patch works as intended :) And I believe it's not a bug introduced by PATCH 1/2.

So, according to my decodecode reading, RAX is the results of get_pageblock_migratetype() and it's MIGRATE_RESERVE. RDX is the result of get_freepage_migratetype() and it's MIGRATE_UNMOVABLE. The freepage_migratetype has just been set either by __rmqueue_smallest() or __rmqueue_fallback(), according to the free_list the page has been taken from. So this looks like a page from MIGRATE_RESERVE pageblock found on the !MIGRATE_RESERVE free_list, which is exactly what the patch intends to catch.

I think there are two possible explanations.

1) the pageblock is genuinely MIGRATE_RESERVE and it was misplaced by mistake. I think it wasn't in free_pcppages_bulk() as there's the same VM_BUG_ON which would supposedly trigger at the moment of displacing. In theory it's possible that there's a race through __free_pages_ok() -> free_one_page() where the get_pageblock_migratetype() in __free_pages_ok() would race with set_pageblock_migratetype() and result in bogus value. But nobody should be calling set_pageblock_migratetype() on a MIGRATE_RESERVE pageblock.

2) the pageblock was marked as MIGRATE_RESERVE due to a race between set_pageblock_migratetype() and set_pageblock_skip(). The latter is currently not serialized by zone->lock, nor it uses atomic bit set. So it may result in lost updates in a racing set_pageblock_migratetype(). I think a well-placed race when changing pageblock from MIGRATE_MOVABLE to MIGRATE_RECLAIMABLE could result in MIGRATE_RESERVE value. Similar races have been already observed to be a problem where frequent changing to/from MIGRATE_ISOLATE is involved, and I did a patch series to address this, but it was not complete and I postponed it after Mel's changes that remove the racy for-cycles completely. So it might be that his "[PATCH 08/17] mm: page_alloc: Use word-based accesses for get/set pageblock bitmaps" already solves this bug (but maybe only on certain architectures where you don't need atomic operations). You might try that patch if you can reproduce this bug frequently enough?

Quoted 1 line

I've tried that patch, but still see the same BUG_ON.

Oh damn, I've realized that my assumptions about MIGRATE_RESERVE pageblocks being created only on zone init time were wrong. setup_zone_migrate_reserve() is called also from the handler of min_free_kbytes sysctl... does trinity try to change that while running?

Quoted 129 lines · Sasha Levin

Quoted 113 lines · Vlastimil Babka

Quoted 81 lines · Sasha Levin

Quoted 81 lines · Vlastimil Babka

Quoted 12 lines

Quoted 12 lines

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

Quoted 8 lines

Quoted 8 lines

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

Hey, that's not an issue, that means the patch works as intended :) And I believe it's not a bug introduced by PATCH 1/2.

So, according to my decodecode reading, RAX is the results of get_pageblock_migratetype() and it's MIGRATE_RESERVE. RDX is the result of get_freepage_migratetype() and it's MIGRATE_UNMOVABLE. The freepage_migratetype has just been set either by __rmqueue_smallest() or __rmqueue_fallback(), according to the free_list the page has been taken from. So this looks like a page from MIGRATE_RESERVE pageblock found on the !MIGRATE_RESERVE free_list, which is exactly what the patch intends to catch.

I think there are two possible explanations.

1) the pageblock is genuinely MIGRATE_RESERVE and it was misplaced by mistake. I think it wasn't in free_pcppages_bulk() as there's the same VM_BUG_ON which would supposedly trigger at the moment of displacing. In theory it's possible that there's a race through __free_pages_ok() -> free_one_page() where the get_pageblock_migratetype() in __free_pages_ok() would race with set_pageblock_migratetype() and result in bogus value. But nobody should be calling set_pageblock_migratetype() on a MIGRATE_RESERVE pageblock.

Quoted 14 lines

2) the pageblock was marked as MIGRATE_RESERVE due to a race between set_pageblock_migratetype() and set_pageblock_skip(). The latter is currently not serialized by zone->lock, nor it uses atomic bit set. So it may result in lost updates in a racing set_pageblock_migratetype(). I think a well-placed race when changing pageblock from MIGRATE_MOVABLE to MIGRATE_RECLAIMABLE could result in MIGRATE_RESERVE value. Similar races have been already observed to be a problem where frequent changing to/from MIGRATE_ISOLATE is involved, and I did a patch series to address this, but it was not complete and I postponed it after Mel's changes that remove the racy for-cycles completely. So it might be that his "[PATCH 08/17] mm: page_alloc: Use word-based accesses for get/set pageblock bitmaps" already solves this bug (but maybe only on certain architectures where you don't need atomic operations). You might try that patch if you can reproduce this bug frequently enough?

Quoted 1 line

I've tried that patch, but still see the same BUG_ON.

Oh damn, I've realized that my assumptions about MIGRATE_RESERVE pageblocks being created only on zone init time were wrong. setup_zone_migrate_reserve() is called also from the handler of min_free_kbytes sysctl... does trinity try to change that while running?
The function will change MOVABLE pageblocks to RESERVE and try to move all free pages to the RESERVE free_list, but of course pages on pcplists will remain MOVABLE and may trigger the VM_BUG_ON. You triggered the bug with page on MOVABLE free_list (in the first reply I said its UNMOVABLE by mistake) so this might be good explanation if trinity changes min_free_kbytes.

Furthermore, I think there's a problem that setup_zone_migrate_reserve() operates on pageblocks, but as MAX_ODER is higher than pageblock_order, RESERVE pages might be merged with buddies of different migratetype and end up on their free_list. That seems to me like a flaw in the design of reserves, but perhaps others won't think it's serious enough to fix?

So in the end this VM_DEBUG check probably cannot work anymore for MIGRATE_RESERVE, only for CMA. I'm not sure if it's worth keeping it only for CMA, what are the CMA guys' opinions on that?

Also this means that the 1/2 patch "prevent MIGRATE_RESERVE pages from being misplaced" still won't prevent stealing a MIGRATE_RESERVE pageblock when __rmqueue_fallback() encounters a strayed MIGRATE_RESERVE page on e.g. a MOVABLE freelist. This is fixable by having __rmqueue_fallback() not trusting the migratetype of free_list and checking for pageblock_migratetype. I hate that, but at least it's not on the fast path...

Quoted 3 lines

Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 151 lines · Vlastimil Babka

Quoted 129 lines · Sasha Levin

Quoted 113 lines · Vlastimil Babka

Quoted 81 lines · Sasha Levin

Quoted 81 lines · Vlastimil Babka

Quoted 12 lines

Quoted 12 lines

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

Quoted 8 lines

Quoted 8 lines

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

Hey, that's not an issue, that means the patch works as intended :) And I believe it's not a bug introduced by PATCH 1/2.

So, according to my decodecode reading, RAX is the results of get_pageblock_migratetype() and it's MIGRATE_RESERVE. RDX is the result of get_freepage_migratetype() and it's MIGRATE_UNMOVABLE. The freepage_migratetype has just been set either by __rmqueue_smallest() or __rmqueue_fallback(), according to the free_list the page has been taken from. So this looks like a page from MIGRATE_RESERVE pageblock found on the !MIGRATE_RESERVE free_list, which is exactly what the patch intends to catch.

I think there are two possible explanations.

1) the pageblock is genuinely MIGRATE_RESERVE and it was misplaced by mistake. I think it wasn't in free_pcppages_bulk() as there's the same VM_BUG_ON which would supposedly trigger at the moment of displacing. In theory it's possible that there's a race through __free_pages_ok() -> free_one_page() where the get_pageblock_migratetype() in __free_pages_ok() would race with set_pageblock_migratetype() and result in bogus value. But nobody should be calling set_pageblock_migratetype() on a MIGRATE_RESERVE pageblock.

Quoted 14 lines

2) the pageblock was marked as MIGRATE_RESERVE due to a race between set_pageblock_migratetype() and set_pageblock_skip(). The latter is currently not serialized by zone->lock, nor it uses atomic bit set. So it may result in lost updates in a racing set_pageblock_migratetype(). I think a well-placed race when changing pageblock from MIGRATE_MOVABLE to MIGRATE_RECLAIMABLE could result in MIGRATE_RESERVE value. Similar races have been already observed to be a problem where frequent changing to/from MIGRATE_ISOLATE is involved, and I did a patch series to address this, but it was not complete and I postponed it after Mel's changes that remove the racy for-cycles completely. So it might be that his "[PATCH 08/17] mm: page_alloc: Use word-based accesses for get/set pageblock bitmaps" already solves this bug (but maybe only on certain architectures where you don't need atomic operations). You might try that patch if you can reproduce this bug frequently enough?

Quoted 1 line

I've tried that patch, but still see the same BUG_ON.

Oh damn, I've realized that my assumptions about MIGRATE_RESERVE pageblocks being created only on zone init time were wrong. setup_zone_migrate_reserve() is called also from the handler of min_free_kbytes sysctl... does trinity try to change that while running?
The function will change MOVABLE pageblocks to RESERVE and try to move all free pages to the RESERVE free_list, but of course pages on pcplists will remain MOVABLE and may trigger the VM_BUG_ON. You triggered the bug with page on MOVABLE free_list (in the first reply I said its UNMOVABLE by mistake) so this might be good explanation if trinity changes min_free_kbytes.

Furthermore, I think there's a problem that setup_zone_migrate_reserve() operates on pageblocks, but as MAX_ODER is higher than pageblock_order, RESERVE pages might be merged with buddies of different migratetype and end up on their free_list. That seems to me like a flaw in the design of reserves, but perhaps others won't think it's serious enough to fix?

Quoted 3 lines

So in the end this VM_DEBUG check probably cannot work anymore for MIGRATE_RESERVE, only for CMA. I'm not sure if it's worth keeping it only for CMA, what are the CMA guys' opinions on that?

I really don't want it. That was I didn't add my Acked-by at that time. For a long time, I never wanted to add more overhead hot path due to CMA unless it's really critical. It's same to this.
Although such debug patch helps to notice something goes wrong for CMA, more information would be helpful to know why CMA failed because there are another potential reasons to fail CMA allocation.

One of the idea about that is to store alloc trace into somewhere(ex, naive idea is page description like page-owner) and then we could investigate what's the owner of that page so we could know why we fail to migrate it out. With that, we would figure out how on earth such page is allocated from CMA and it would be more helpful rather just VM_BUG_ON notice.

The whole point is I'd like to avoid adding more overhead to hot path for rare case although it's debugging feature.

Quoted 145 lines · Minchan Kim

Quoted 144 lines · Vlastimil Babka

Quoted 129 lines · Sasha Levin

Quoted 113 lines · Vlastimil Babka

Quoted 81 lines · Sasha Levin

Quoted 81 lines · Vlastimil Babka

Quoted 12 lines

Quoted 12 lines

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

Quoted 8 lines

Quoted 8 lines

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

Hey, that's not an issue, that means the patch works as intended :) And I believe it's not a bug introduced by PATCH 1/2.

So, according to my decodecode reading, RAX is the results of get_pageblock_migratetype() and it's MIGRATE_RESERVE. RDX is the result of get_freepage_migratetype() and it's MIGRATE_UNMOVABLE. The freepage_migratetype has just been set either by __rmqueue_smallest() or __rmqueue_fallback(), according to the free_list the page has been taken from. So this looks like a page from MIGRATE_RESERVE pageblock found on the !MIGRATE_RESERVE free_list, which is exactly what the patch intends to catch.

I think there are two possible explanations.

1) the pageblock is genuinely MIGRATE_RESERVE and it was misplaced by mistake. I think it wasn't in free_pcppages_bulk() as there's the same VM_BUG_ON which would supposedly trigger at the moment of displacing. In theory it's possible that there's a race through __free_pages_ok() -> free_one_page() where the get_pageblock_migratetype() in __free_pages_ok() would race with set_pageblock_migratetype() and result in bogus value. But nobody should be calling set_pageblock_migratetype() on a MIGRATE_RESERVE pageblock.

Quoted 14 lines

2) the pageblock was marked as MIGRATE_RESERVE due to a race between set_pageblock_migratetype() and set_pageblock_skip(). The latter is currently not serialized by zone->lock, nor it uses atomic bit set. So it may result in lost updates in a racing set_pageblock_migratetype(). I think a well-placed race when changing pageblock from MIGRATE_MOVABLE to MIGRATE_RECLAIMABLE could result in MIGRATE_RESERVE value. Similar races have been already observed to be a problem where frequent changing to/from MIGRATE_ISOLATE is involved, and I did a patch series to address this, but it was not complete and I postponed it after Mel's changes that remove the racy for-cycles completely. So it might be that his "[PATCH 08/17] mm: page_alloc: Use word-based accesses for get/set pageblock bitmaps" already solves this bug (but maybe only on certain architectures where you don't need atomic operations). You might try that patch if you can reproduce this bug frequently enough?

Quoted 1 line

I've tried that patch, but still see the same BUG_ON.

Oh damn, I've realized that my assumptions about MIGRATE_RESERVE pageblocks being created only on zone init time were wrong. setup_zone_migrate_reserve() is called also from the handler of min_free_kbytes sysctl... does trinity try to change that while running?
The function will change MOVABLE pageblocks to RESERVE and try to move all free pages to the RESERVE free_list, but of course pages on pcplists will remain MOVABLE and may trigger the VM_BUG_ON. You triggered the bug with page on MOVABLE free_list (in the first reply I said its UNMOVABLE by mistake) so this might be good explanation if trinity changes min_free_kbytes.

Quoted 6 lines

Quoted 6 lines

Furthermore, I think there's a problem that setup_zone_migrate_reserve() operates on pageblocks, but as MAX_ODER is higher than pageblock_order, RESERVE pages might be merged with buddies of different migratetype and end up on their free_list. That seems to me like a flaw in the design of reserves, but perhaps others won't think it's serious enough to fix?

Quoted 18 lines

Quoted 4 lines

So in the end this VM_DEBUG check probably cannot work anymore for MIGRATE_RESERVE, only for CMA. I'm not sure if it's worth keeping it only for CMA, what are the CMA guys' opinions on that?

I really don't want it. That was I didn't add my Acked-by at that time. For a long time, I never wanted to add more overhead hot path due to CMA unless it's really critical. It's same to this.
Although such debug patch helps to notice something goes wrong for CMA, more information would be helpful to know why CMA failed because there are another potential reasons to fail CMA allocation.

One of the idea about that is to store alloc trace into somewhere(ex, naive idea is page description like page-owner) and then we could investigate what's the owner of that page so we could know why we fail to migrate it out. With that, we would figure out how on earth such page is allocated from CMA and it would be more helpful rather just VM_BUG_ON notice.

Quoted 2 lines

The whole point is I'd like to avoid adding more overhead to hot path for rare case although it's debugging feature.

OK. I'm not that concerned with VM_DEBUG overhead as it's intended for testing, not production. But as you say the patch is not that useful
without the MIGRATE_RESERVE part, so Andrew could you please drop the
patch (mm-page_alloc-debug_vm-checks-for-free_list-placement-of-cma-and-reserve-pages.patch)?

Quoted 146 lines · Vlastimil Babka

Quoted 145 lines · Minchan Kim

Quoted 144 lines · Vlastimil Babka

Quoted 129 lines · Sasha Levin

Quoted 113 lines · Vlastimil Babka

Quoted 81 lines · Sasha Levin

Quoted 81 lines · Vlastimil Babka

Quoted 12 lines

Quoted 12 lines

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

Quoted 8 lines

Quoted 8 lines

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

Hey, that's not an issue, that means the patch works as intended :) And I believe it's not a bug introduced by PATCH 1/2.

So, according to my decodecode reading, RAX is the results of get_pageblock_migratetype() and it's MIGRATE_RESERVE. RDX is the result of get_freepage_migratetype() and it's MIGRATE_UNMOVABLE. The freepage_migratetype has just been set either by __rmqueue_smallest() or __rmqueue_fallback(), according to the free_list the page has been taken from. So this looks like a page from MIGRATE_RESERVE pageblock found on the !MIGRATE_RESERVE free_list, which is exactly what the patch intends to catch.

I think there are two possible explanations.

1) the pageblock is genuinely MIGRATE_RESERVE and it was misplaced by mistake. I think it wasn't in free_pcppages_bulk() as there's the same VM_BUG_ON which would supposedly trigger at the moment of displacing. In theory it's possible that there's a race through __free_pages_ok() -> free_one_page() where the get_pageblock_migratetype() in __free_pages_ok() would race with set_pageblock_migratetype() and result in bogus value. But nobody should be calling set_pageblock_migratetype() on a MIGRATE_RESERVE pageblock.

Quoted 14 lines

2) the pageblock was marked as MIGRATE_RESERVE due to a race between set_pageblock_migratetype() and set_pageblock_skip(). The latter is currently not serialized by zone->lock, nor it uses atomic bit set. So it may result in lost updates in a racing set_pageblock_migratetype(). I think a well-placed race when changing pageblock from MIGRATE_MOVABLE to MIGRATE_RECLAIMABLE could result in MIGRATE_RESERVE value. Similar races have been already observed to be a problem where frequent changing to/from MIGRATE_ISOLATE is involved, and I did a patch series to address this, but it was not complete and I postponed it after Mel's changes that remove the racy for-cycles completely. So it might be that his "[PATCH 08/17] mm: page_alloc: Use word-based accesses for get/set pageblock bitmaps" already solves this bug (but maybe only on certain architectures where you don't need atomic operations). You might try that patch if you can reproduce this bug frequently enough?

Quoted 1 line

I've tried that patch, but still see the same BUG_ON.

Oh damn, I've realized that my assumptions about MIGRATE_RESERVE pageblocks being created only on zone init time were wrong. setup_zone_migrate_reserve() is called also from the handler of min_free_kbytes sysctl... does trinity try to change that while running?
The function will change MOVABLE pageblocks to RESERVE and try to move all free pages to the RESERVE free_list, but of course pages on pcplists will remain MOVABLE and may trigger the VM_BUG_ON. You triggered the bug with page on MOVABLE free_list (in the first reply I said its UNMOVABLE by mistake) so this might be good explanation if trinity changes min_free_kbytes.

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Furthermore, I think there's a problem that setup_zone_migrate_reserve() operates on pageblocks, but as MAX_ODER is higher than pageblock_order, RESERVE pages might be merged with buddies of different migratetype and end up on their free_list. That seems to me like a flaw in the design of reserves, but perhaps others won't think it's serious enough to fix?

I wanna know who want MIGRATE_RESERVE. On my previous testing, one pageblock for MIGRATE_RESERVE is merged with buddies of different migratetype during boot-up and never come back again. But my system works well. :)

If it is really useful feature, we can fix the situation by aligning reserve size and pfn to MAX_ORDER_NR_PAGES.

And, IMHO, it isn't reasonable to increase, decrease and change MIGRATE_RESERVE pageblock by the handler of min_free_kbytes sysctl, because new pageblock may have no free pages to reserve. I think that it is better to prevent sysctl handler from changing MIGRATE_RESERVE pageblock, after initialization.

In conclusion, if MIGRATE_RESERVE is useful enough to maintain, to fix above problem and keep this patch is preferable to me.

Quoted 21 lines

Quoted 20 lines

Quoted 4 lines

So in the end this VM_DEBUG check probably cannot work anymore for MIGRATE_RESERVE, only for CMA. I'm not sure if it's worth keeping it only for CMA, what are the CMA guys' opinions on that?

I really don't want it. That was I didn't add my Acked-by at that time. For a long time, I never wanted to add more overhead hot path due to CMA unless it's really critical. It's same to this.
Although such debug patch helps to notice something goes wrong for CMA, more information would be helpful to know why CMA failed because there are another potential reasons to fail CMA allocation.

One of the idea about that is to store alloc trace into somewhere(ex, naive idea is page description like page-owner) and then we could investigate what's the owner of that page so we could know why we fail to migrate it out. With that, we would figure out how on earth such page is allocated from CMA and it would be more helpful rather just VM_BUG_ON notice.

The whole point is I'd like to avoid adding more overhead to hot path for rare case although it's debugging feature.

Quoted 4 lines

OK. I'm not that concerned with VM_DEBUG overhead as it's intended for testing, not production. But as you say the patch is not that useful
without the MIGRATE_RESERVE part, so Andrew could you please drop the
patch (mm-page_alloc-debug_vm-checks-for-free_list-placement-of-cma-and-reserve-pages.patch)?

I also think that VM_DEBUG overhead isn't problem because of same reason from Vlastimil.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 2 lines

I also think that VM_DEBUG overhead isn't problem because of same reason from Vlastimil.

Guys, please read this.

https://lkml.org/lkml/2013/7/17/591

If you guys really want it, we could separate it with CONFIG_DEBUG_CMA or CONFIG_DEBUG_RESERVE like stuff. Otherwise, just remain in mmotm.

Quoted 10 lines · On Thu, 8 May 2014 15:19:37 +0900 Minchan Kim <[email protected]> wrote:

Quoted 2 lines

I also think that VM_DEBUG overhead isn't problem because of same reason from Vlastimil.

Guys, please read this.

https://lkml.org/lkml/2013/7/17/591

If you guys really want it, we could separate it with CONFIG_DEBUG_CMA or CONFIG_DEBUG_RESERVE like stuff. Otherwise, just remain in mmotm.

Wise words, those.

Yes, these checks are in a pretty hot path. I'm inclined to make the patch -mm (and -next) only.

Unless there's a really good reason, such as "nobody who uses CMA is likely to be testing -next", which sounds likely :(

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Furthermore, I think there's a problem that setup_zone_migrate_reserve() operates on pageblocks, but as MAX_ODER is higher than pageblock_order, RESERVE pages might be merged with buddies of different migratetype and end up on their free_list. That seems to me like a flaw in the design of reserves, but perhaps others won't think it's serious enough to fix?

I wanna know who want MIGRATE_RESERVE. On my previous testing, one pageblock for MIGRATE_RESERVE is merged with buddies of different migratetype during boot-up and never come back again. But my system works well. :)

Quoted 146 lines · Vlastimil Babka

Quoted 145 lines · Minchan Kim

Quoted 144 lines · Vlastimil Babka

Quoted 129 lines · Sasha Levin

Quoted 113 lines · Vlastimil Babka

Quoted 81 lines · Sasha Levin

Quoted 81 lines · Vlastimil Babka

Quoted 12 lines

Quoted 12 lines

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

Quoted 8 lines

Quoted 8 lines

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

Hey, that's not an issue, that means the patch works as intended :) And I believe it's not a bug introduced by PATCH 1/2.

So, according to my decodecode reading, RAX is the results of get_pageblock_migratetype() and it's MIGRATE_RESERVE. RDX is the result of get_freepage_migratetype() and it's MIGRATE_UNMOVABLE. The freepage_migratetype has just been set either by __rmqueue_smallest() or __rmqueue_fallback(), according to the free_list the page has been taken from. So this looks like a page from MIGRATE_RESERVE pageblock found on the !MIGRATE_RESERVE free_list, which is exactly what the patch intends to catch.

I think there are two possible explanations.

1) the pageblock is genuinely MIGRATE_RESERVE and it was misplaced by mistake. I think it wasn't in free_pcppages_bulk() as there's the same VM_BUG_ON which would supposedly trigger at the moment of displacing. In theory it's possible that there's a race through __free_pages_ok() -> free_one_page() where the get_pageblock_migratetype() in __free_pages_ok() would race with set_pageblock_migratetype() and result in bogus value. But nobody should be calling set_pageblock_migratetype() on a MIGRATE_RESERVE pageblock.

Quoted 14 lines

2) the pageblock was marked as MIGRATE_RESERVE due to a race between set_pageblock_migratetype() and set_pageblock_skip(). The latter is currently not serialized by zone->lock, nor it uses atomic bit set. So it may result in lost updates in a racing set_pageblock_migratetype(). I think a well-placed race when changing pageblock from MIGRATE_MOVABLE to MIGRATE_RECLAIMABLE could result in MIGRATE_RESERVE value. Similar races have been already observed to be a problem where frequent changing to/from MIGRATE_ISOLATE is involved, and I did a patch series to address this, but it was not complete and I postponed it after Mel's changes that remove the racy for-cycles completely. So it might be that his "[PATCH 08/17] mm: page_alloc: Use word-based accesses for get/set pageblock bitmaps" already solves this bug (but maybe only on certain architectures where you don't need atomic operations). You might try that patch if you can reproduce this bug frequently enough?

Quoted 1 line

I've tried that patch, but still see the same BUG_ON.

Oh damn, I've realized that my assumptions about MIGRATE_RESERVE pageblocks being created only on zone init time were wrong. setup_zone_migrate_reserve() is called also from the handler of min_free_kbytes sysctl... does trinity try to change that while running?
The function will change MOVABLE pageblocks to RESERVE and try to move all free pages to the RESERVE free_list, but of course pages on pcplists will remain MOVABLE and may trigger the VM_BUG_ON. You triggered the bug with page on MOVABLE free_list (in the first reply I said its UNMOVABLE by mistake) so this might be good explanation if trinity changes min_free_kbytes.

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Furthermore, I think there's a problem that setup_zone_migrate_reserve() operates on pageblocks, but as MAX_ODER is higher than pageblock_order, RESERVE pages might be merged with buddies of different migratetype and end up on their free_list. That seems to me like a flaw in the design of reserves, but perhaps others won't think it's serious enough to fix?

I wanna know who want MIGRATE_RESERVE. On my previous testing, one pageblock for MIGRATE_RESERVE is merged with buddies of different migratetype during boot-up and never come back again. But my system works well. :)

If it is really useful feature, we can fix the situation by aligning reserve size and pfn to MAX_ORDER_NR_PAGES.

And, IMHO, it isn't reasonable to increase, decrease and change MIGRATE_RESERVE pageblock by the handler of min_free_kbytes sysctl, because new pageblock may have no free pages to reserve. I think that it is better to prevent sysctl handler from changing MIGRATE_RESERVE pageblock, after initialization.

In conclusion, if MIGRATE_RESERVE is useful enough to maintain, to fix above problem and keep this patch is preferable to me.

Quoted 18 lines

Quoted 18 lines

Quoted 4 lines

So in the end this VM_DEBUG check probably cannot work anymore for MIGRATE_RESERVE, only for CMA. I'm not sure if it's worth keeping it only for CMA, what are the CMA guys' opinions on that?

I really don't want it. That was I didn't add my Acked-by at that time. For a long time, I never wanted to add more overhead hot path due to CMA unless it's really critical. It's same to this.
Although such debug patch helps to notice something goes wrong for CMA, more information would be helpful to know why CMA failed because there are another potential reasons to fail CMA allocation.

One of the idea about that is to store alloc trace into somewhere(ex, naive idea is page description like page-owner) and then we could investigate what's the owner of that page so we could know why we fail to migrate it out. With that, we would figure out how on earth such page is allocated from CMA and it would be more helpful rather just VM_BUG_ON notice.

Quoted 7 lines

Quoted 2 lines

The whole point is I'd like to avoid adding more overhead to hot path for rare case although it's debugging feature.

OK. I'm not that concerned with VM_DEBUG overhead as it's intended for testing, not production. But as you say the patch is not that useful
without the MIGRATE_RESERVE part, so Andrew could you please drop the
patch (mm-page_alloc-debug_vm-checks-for-free_list-placement-of-cma-and-reserve-pages.patch)?

I also think that VM_DEBUG overhead isn't problem because of same reason from Vlastimil.

Thanks.

Quoted 154 lines · Joonsoo Kim

Quoted 146 lines · Vlastimil Babka

Quoted 145 lines · Minchan Kim

Quoted 144 lines · Vlastimil Babka

Quoted 129 lines · Sasha Levin

Quoted 113 lines · Vlastimil Babka

Quoted 81 lines · Sasha Levin

Quoted 81 lines · Vlastimil Babka

Quoted 12 lines

Quoted 12 lines

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages(). For MIGRATE_CMA, the pages also must not go to a different free_list, otherwise they could get allocated as unmovable and result in CMA failure.

This is ensured by setting the freepage_migratetype appropriately when placing pages on pcp lists, and using the information when releasing them back to free_list. It is also assumed that CMA and RESERVE pageblocks are created only in the init phase. This patch adds DEBUG_VM checks to catch any regressions introduced for this invariant.

Quoted 11 lines

Quoted 11 lines

Cc Yong-Taek Lee <[email protected]>

Cc Bartlomiej Zolnierkiewicz <[email protected]>

Cc Joonsoo Kim <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

Two issues with this patch.

First:

[ 3446.320082] kernel BUG at mm/page_alloc.c:1197!
[ 3446.320082] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[ 3446.320082] Dumping ftrace buffer:
[ 3446.320082]    (ftrace buffer empty)
[ 3446.320082] Modules linked in:
[ 3446.320082] CPU: 1 PID: 8923 Comm: trinity-c42 Not tainted 3.15.0-rc3-next-20140429-sasha-00015-g7c7e0a7-dirty #427
[ 3446.320082] task: ffff88053e208000 ti: ffff88053e246000 task.ti: ffff88053e246000
[ 3446.320082] RIP: get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.320082] RSP: 0018:ffff88053e247778  EFLAGS: 00010002
[ 3446.320082] RAX: 0000000000000003 RBX: ffffea0000f40000 RCX: 0000000000000008
[ 3446.320082] RDX: 0000000000000002 RSI: 0000000000000003 RDI: 00000000000000a0
[ 3446.320082] RBP: ffff88053e247868 R08: 0000000000000007 R09: 0000000000000000
[ 3446.320082] R10: ffff88006ffcef00 R11: 0000000000000000 R12: 0000000000000014
[ 3446.335888] R13: ffffea000115ffe0 R14: ffffea000115ffe0 R15: 0000000000000000
[ 3446.335888] FS:  00007f8c9f059700(0000) GS:ffff88006ec00000(0000) knlGS:0000000000000000
[ 3446.335888] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 3446.335888] CR2: 0000000002cbc048 CR3: 000000054cdb4000 CR4: 00000000000006a0
[ 3446.335888] DR0: 00000000006de000 DR1: 00000000006de000 DR2: 0000000000000000
[ 3446.335888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000602
[ 3446.335888] Stack:
[ 3446.335888]  ffff88053e247798 ffff88006eddc0b8 0000000000000016 0000000000000000
[ 3446.335888]  ffff88006ffd2068 ffff88006ffdb008 0000000100000000 0000000000000000
[ 3446.335888]  ffff88006ffdb000 0000000000000000 0000000000000003 0000000000000001
[ 3446.335888] Call Trace:
[ 3446.335888] __alloc_pages_nodemask (mm/page_alloc.c:2731)
[ 3446.335888] ? __this_cpu_preempt_check (lib/smp_processor_id.c:63)
[ 3446.335888] alloc_pages_vma (include/linux/mempolicy.h:76 mm/mempolicy.c:1998)
[ 3446.335888] ? shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? kvm_clock_read (arch/x86/include/asm/preempt.h:90 arch/x86/kernel/kvmclock.c:86)
[ 3446.335888] shmem_alloc_page (mm/shmem.c:881)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] ? find_get_entry (mm/filemap.c:979)
[ 3446.335888] ? find_get_entry (mm/filemap.c:940)
[ 3446.335888] ? find_lock_entry (mm/filemap.c:1024)
[ 3446.335888] shmem_getpage_gfp (mm/shmem.c:1130)
[ 3446.335888] ? sched_clock_local (kernel/sched/clock.c:214)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] shmem_fault (mm/shmem.c:1237)
[ 3446.335888] ? do_read_fault.isra.42 (mm/memory.c:3523)
[ 3446.335888] __do_fault (mm/memory.c:3344)
[ 3446.335888] ? _raw_spin_unlock (arch/x86/include/asm/preempt.h:98 include/linux/spinlock_api_smp.h:152 kernel/locking/spinlock.c:183)
[ 3446.335888] do_read_fault.isra.42 (mm/memory.c:3524)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] ? get_parent_ip (kernel/sched/core.c:2485)
[ 3446.335888] __handle_mm_fault (mm/memory.c:3662 mm/memory.c:3823 mm/memory.c:3950)
[ 3446.335888] ? __const_udelay (arch/x86/lib/delay.c:126)
[ 3446.335888] ? __rcu_read_unlock (kernel/rcu/update.c:97)
[ 3446.335888] handle_mm_fault (mm/memory.c:3973)
[ 3446.335888] __get_user_pages (mm/memory.c:1863)
[ 3446.335888] ? preempt_count_sub (kernel/sched/core.c:2541)
[ 3446.335888] __mlock_vma_pages_range (mm/mlock.c:255)

Quoted 8 lines

Quoted 8 lines

[ 3446.335888] __mm_populate (mm/mlock.c:711)
[ 3446.335888] vm_mmap_pgoff (include/linux/mm.h:1841 mm/util.c:402)
[ 3446.335888] SyS_mmap_pgoff (mm/mmap.c:1378)
[ 3446.335888] ? syscall_trace_enter (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1461)
[ 3446.335888] ia32_do_call (arch/x86/ia32/ia32entry.S:430)
[ 3446.335888] Code: 00 66 0f 1f 44 00 00 ba 02 00 00 00 31 f6 48 89 c7 e8 c1 c3 ff ff 48 8b 53 10 83 f8 03 74 08 83 f8 04 75 13 0f 1f 00 39 d0 74 0c <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 45 85 ff 75 15 49 8b 55 00
[ 3446.335888] RIP get_page_from_freelist (mm/page_alloc.c:1197 mm/page_alloc.c:1548 mm/page_alloc.c:2036)
[ 3446.335888]  RSP <ffff88053e247778>

Hey, that's not an issue, that means the patch works as intended :) And I believe it's not a bug introduced by PATCH 1/2.

So, according to my decodecode reading, RAX is the results of get_pageblock_migratetype() and it's MIGRATE_RESERVE. RDX is the result of get_freepage_migratetype() and it's MIGRATE_UNMOVABLE. The freepage_migratetype has just been set either by __rmqueue_smallest() or __rmqueue_fallback(), according to the free_list the page has been taken from. So this looks like a page from MIGRATE_RESERVE pageblock found on the !MIGRATE_RESERVE free_list, which is exactly what the patch intends to catch.

I think there are two possible explanations.

1) the pageblock is genuinely MIGRATE_RESERVE and it was misplaced by mistake. I think it wasn't in free_pcppages_bulk() as there's the same VM_BUG_ON which would supposedly trigger at the moment of displacing. In theory it's possible that there's a race through __free_pages_ok() -> free_one_page() where the get_pageblock_migratetype() in __free_pages_ok() would race with set_pageblock_migratetype() and result in bogus value. But nobody should be calling set_pageblock_migratetype() on a MIGRATE_RESERVE pageblock.

Quoted 14 lines

2) the pageblock was marked as MIGRATE_RESERVE due to a race between set_pageblock_migratetype() and set_pageblock_skip(). The latter is currently not serialized by zone->lock, nor it uses atomic bit set. So it may result in lost updates in a racing set_pageblock_migratetype(). I think a well-placed race when changing pageblock from MIGRATE_MOVABLE to MIGRATE_RECLAIMABLE could result in MIGRATE_RESERVE value. Similar races have been already observed to be a problem where frequent changing to/from MIGRATE_ISOLATE is involved, and I did a patch series to address this, but it was not complete and I postponed it after Mel's changes that remove the racy for-cycles completely. So it might be that his "[PATCH 08/17] mm: page_alloc: Use word-based accesses for get/set pageblock bitmaps" already solves this bug (but maybe only on certain architectures where you don't need atomic operations). You might try that patch if you can reproduce this bug frequently enough?

Quoted 1 line

I've tried that patch, but still see the same BUG_ON.

Oh damn, I've realized that my assumptions about MIGRATE_RESERVE pageblocks being created only on zone init time were wrong. setup_zone_migrate_reserve() is called also from the handler of min_free_kbytes sysctl... does trinity try to change that while running?
The function will change MOVABLE pageblocks to RESERVE and try to move all free pages to the RESERVE free_list, but of course pages on pcplists will remain MOVABLE and may trigger the VM_BUG_ON. You triggered the bug with page on MOVABLE free_list (in the first reply I said its UNMOVABLE by mistake) so this might be good explanation if trinity changes min_free_kbytes.

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Furthermore, I think there's a problem that setup_zone_migrate_reserve() operates on pageblocks, but as MAX_ODER is higher than pageblock_order, RESERVE pages might be merged with buddies of different migratetype and end up on their free_list. That seems to me like a flaw in the design of reserves, but perhaps others won't think it's serious enough to fix?

Quoted 7 lines

I wanna know who want MIGRATE_RESERVE. On my previous testing, one pageblock for MIGRATE_RESERVE is merged with buddies of different migratetype during boot-up and never come back again. But my system works well. :)

If it is really useful feature, we can fix the situation by aligning reserve size and pfn to MAX_ORDER_NR_PAGES.

Yes, I plan to do that.

Quoted 5 lines

And, IMHO, it isn't reasonable to increase, decrease and change MIGRATE_RESERVE pageblock by the handler of min_free_kbytes sysctl, because new pageblock may have no free pages to reserve. I think that it is better to prevent sysctl handler from changing MIGRATE_RESERVE pageblock, after initialization.

This dynamic allocation could be more aggressive by doing the same stuff as memory isolation. In any case, I think that if it cannot guarantee the pageblock to be free, there is little benefit from trying to make sure no pages from the pageblock are strayed on pcplists and later misplaced in the free_list. The only danger of misplacement is page stealing code changing MIGRATE_RESERVE pageblock to something else. This can be easily avoided directly in the page stealing code by checking the pageblock migratetype before trying to change it. This has very little extra overhead, in a path that's not hot to begin with.

Quoted 2 lines

In conclusion, if MIGRATE_RESERVE is useful enough to maintain, to fix above problem and keep this patch is preferable to me.

I think it's useful in general, but not so critical to make the min_free_kbytes handler provide 100% guarantees. Presumably one uses the handler only during early boot time when there's enough free memory?
So that leaves the debug patch only for CMA which I won't push anymore to mainline, but feel free to adapt it for mm only.

So in the end this VM_DEBUG check probably cannot work anymore for MIGRATE_RESERVE, only for CMA. I'm not sure if it's worth keeping it only for CMA, what are the CMA guys' opinions on that?

Quoted 1 line · Vlastimil Babka

So in the end this VM_DEBUG check probably cannot work anymore for MIGRATE_RESERVE, only for CMA. I'm not sure if it's worth keeping it only for CMA, what are the CMA guys' opinions on that?

The way I understood it is that this patch is wrong, but it's still alive in -mm. Should it still be there?

Quoted 5 lines · Sasha Levin

Quoted 1 line · Vlastimil Babka

So in the end this VM_DEBUG check probably cannot work anymore for MIGRATE_RESERVE, only for CMA. I'm not sure if it's worth keeping it only for CMA, what are the CMA guys' opinions on that?

The way I understood it is that this patch is wrong, but it's still alive in -mm. Should it still be there?

I agree that it should be dropped. I did not follow the discussion, but mmotm soon gives me BUG at mm/page_alloc.c:1242 under swapping load.

Hugh

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages().

Currently, it is however possible for this to happen when MIGRATE_RESERVE page is allocated on pcplist through rmqueue_bulk() as a fallback for other desired migratetype, and then later freed back through free_pcppages_bulk() without being actually used. This happens because free_pcppages_bulk() uses get_freepage_migratetype() to choose the free_list, and rmqueue_bulk() calls set_freepage_migratetype() with the *desired* migratetype and not the page's original MIGRATE_RESERVE migratetype.

This patch fixes the problem by moving the call to set_freepage_migratetype() from rmqueue_bulk() down to __rmqueue_smallest() and __rmqueue_fallback() where the actual page's migratetype (e.g. from which free_list the page is taken from) is used. Note that this migratetype might be different from the pageblock's migratetype due to freepage stealing decisions. This is OK, as page stealing never uses MIGRATE_RESERVE as a fallback, and also takes care to leave all MIGRATE_CMA pages on the correct freelist.

Therefore, as an additional benefit, the call to get_pageblock_migratetype() from rmqueue_bulk() when CMA is enabled, can be removed completely. This relies on the fact that MIGRATE_CMA pageblocks are created only during system init, and the above. The related is_migrate_isolate() check is also unnecessary, as memory isolation has other ways to move pages between freelists, and drain pcp lists containing pages that should be isolated.
The buffered_rmqueue() can also benefit from calling get_freepage_migratetype() instead of get_pageblock_migratetype().

A separate patch will add VM_BUG_ON checks for the invariant that for MIGRATE_RESERVE and MIGRATE_CMA pageblocks, freepage_migratetype must equal to pageblock_migratetype so that these pages always go to the correct free_list.

Reported-by Yong-Taek Lee <[email protected]>

Reported-by Bartlomiej Zolnierkiewicz <[email protected]>

Suggested-by Joonsoo Kim <[email protected]>

Suggested-by Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>

For the MIGRATE_RESERVE pages, it is important they do not get misplaced on free_list of other migratetype, otherwise the whole MIGRATE_RESERVE pageblock might be changed to other migratetype in try_to_steal_freepages().

Currently, it is however possible for this to happen when MIGRATE_RESERVE page is allocated on pcplist through rmqueue_bulk() as a fallback for other desired migratetype, and then later freed back through free_pcppages_bulk() without being actually used. This happens because free_pcppages_bulk() uses get_freepage_migratetype() to choose the free_list, and rmqueue_bulk() calls set_freepage_migratetype() with the *desired* migratetype and not the page's original MIGRATE_RESERVE migratetype.

This patch fixes the problem by moving the call to set_freepage_migratetype() from rmqueue_bulk() down to __rmqueue_smallest() and __rmqueue_fallback() where the actual page's migratetype (e.g. from which free_list the page is taken from) is used. Note that this migratetype might be different from the pageblock's migratetype due to freepage stealing decisions. This is OK, as page stealing never uses MIGRATE_RESERVE as a fallback, and also takes care to leave all MIGRATE_CMA pages on the correct freelist.

Therefore, as an additional benefit, the call to get_pageblock_migratetype() from rmqueue_bulk() when CMA is enabled, can be removed completely. This relies on the fact that MIGRATE_CMA pageblocks are created only during system init, and the above. The related is_migrate_isolate() check is also unnecessary, as memory isolation has other ways to move pages between freelists, and drain pcp lists containing pages that should be isolated.
The buffered_rmqueue() can also benefit from calling get_freepage_migratetype() instead of get_pageblock_migratetype().

A separate patch will add VM_BUG_ON checks for the invariant that for MIGRATE_RESERVE and MIGRATE_CMA pageblocks, freepage_migratetype must equal to pageblock_migratetype so that these pages always go to the correct free_list.

Reported-by Yong-Taek Lee <[email protected]>

Reported-by Bartlomiej Zolnierkiewicz <[email protected]>

Suggested-by Joonsoo Kim <[email protected]>

Suggested-by Mel Gorman <[email protected]>

Cc Minchan Kim <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Marek Szyprowski <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Michal Nazarewicz <[email protected]>

Signed-off-by Vlastimil Babka <[email protected]>