Skip to contentSkip to conversation

sched/numa, mm: do not promote folios to nodes not set N_MEMORY

PATCH Superseded Byungchul Park 2 participants 3 messages Feb 16, 2024

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 sched/numa, mm: do not promote folios to nodes not set N_MEMORY

Changes from v1:
1. Trim the verbose oops in the commit message. (feedbacked by Phil Auld)
2. Rewrite a comment in code. (feedbacked by Phil Auld)

--->8---
From 6830b59db267a31b78f6f09af12ae0e3132b4bca Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 16:26:23 +0900
Subject: [PATCH v2] sched/numa, mm: do not promote folios to nodes not set N_MEMORY

While running qemu with a configuration where some CPUs don't have their local memory and with a kernel numa balancing on, the following oops has been observed. It's because of null pointers of ->zone_pgdat of zones of those nodes that are not initialized properly at booting time. So should avoid nodes not set N_MEMORY from getting promoted.

Quoted 36 lines
BUG: unable to handle page fault for address: 00000000000033f3
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: 0000 [#1] PREEMPT SMP NOPTI
CPU: 2 PID: 895 Comm: masim Not tainted 6.6.0-dirty #255
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
   rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014
RIP: 0010:wakeup_kswapd (./linux/mm/vmscan.c:7812)
Code: (omitted)
RSP: 0000:ffffc90004257d58 EFLAGS: 00010286
RAX: ffffffffffffffff RBX: ffff88883fff0480 RCX: 0000000000000003
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88883fff0480
RBP: ffffffffffffffff R08: ff0003ffffffffff R09: ffffffffffffffff
R10: ffff888106c95540 R11: 0000000055555554 R12: 0000000000000003
R13: 0000000000000000 R14: 0000000000000000 R15: ffff88883fff0940
FS:  00007fc4b8124740(0000) GS:ffff888827c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000000033f3 CR3: 000000026cc08004 CR4: 0000000000770ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
 <TASK>

? __die
? page_fault_oops
? __pte_offset_map_lock
? exc_page_fault
? asm_exc_page_fault
? wakeup_kswapd
migrate_misplaced_page
__handle_mm_fault
handle_mm_fault
do_user_addr_fault
exc_page_fault
asm_exc_page_fault

Quoted 9 lines
RIP: 0033:0x55b897ba0808
Code: (omitted)
RSP: 002b:00007ffeefa821a0 EFLAGS: 00010287
RAX: 000055b89983acd0 RBX: 00007ffeefa823f8 RCX: 000055b89983acd0
RDX: 00007fc2f8122010 RSI: 0000000000020000 RDI: 000055b89983acd0
RBP: 00007ffeefa821a0 R08: 0000000000000037 R09: 0000000000000075
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000000
R13: 00007ffeefa82410 R14: 000055b897ba5dd8 R15: 00007fc4b8340000
 </TASK>
---
 kernel/sched/fair.c | 9 +++++++++
 1 file changed, 9 insertions(+)
kernel/sched/fair.c +9 -0
index d7a3c63a2171..859ef78ef72b 100644
@@ -1828,6 +1828,15 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,
 	int dst_nid = cpu_to_node(dst_cpu);
 	int last_cpupid, this_cpupid;
 
+	/*
+	 * A node of dst_nid might not have its local memory. Promoting
+	 * a folio to the node is meaningless. What's even worse, oops
+	 * can be observed by the null pointer of ->zone_pgdat in
+	 * various points of the code during migration.
+	 */
+	if (!node_state(dst_nid, N_MEMORY))
+		return false;
+
 	/*
 	 * The pages in slow memory node should be migrated according
 	 * to hot/cold instead of private/shared.
Signature · 1 line
2.17.1
#1 lore raw
Quoted 11 lines · Byungchul Park

Changes from v1:
1. Trim the verbose oops in the commit message. (feedbacked by Phil Auld)
2. Rewrite a comment in code. (feedbacked by Phil Auld)

--->8---
From 6830b59db267a31b78f6f09af12ae0e3132b4bca Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 16:26:23 +0900
Subject: [PATCH v2] sched/numa, mm: do not promote folios to nodes not set N_MEMORY

Quoted 52 lines

While running qemu with a configuration where some CPUs don't have their local memory and with a kernel numa balancing on, the following oops has been observed. It's because of null pointers of ->zone_pgdat of zones of those nodes that are not initialized properly at booting time. So should avoid nodes not set N_MEMORY from getting promoted.

Quoted 45 lines
BUG: unable to handle page fault for address: 00000000000033f3
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: 0000 [#1] PREEMPT SMP NOPTI
CPU: 2 PID: 895 Comm: masim Not tainted 6.6.0-dirty #255
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
    rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014
RIP: 0010:wakeup_kswapd (./linux/mm/vmscan.c:7812)
Code: (omitted)
RSP: 0000:ffffc90004257d58 EFLAGS: 00010286
RAX: ffffffffffffffff RBX: ffff88883fff0480 RCX: 0000000000000003
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88883fff0480
RBP: ffffffffffffffff R08: ff0003ffffffffff R09: ffffffffffffffff
R10: ffff888106c95540 R11: 0000000055555554 R12: 0000000000000003
R13: 0000000000000000 R14: 0000000000000000 R15: ffff88883fff0940
FS:  00007fc4b8124740(0000) GS:ffff888827c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000000033f3 CR3: 000000026cc08004 CR4: 0000000000770ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
  <TASK>

? __die
? page_fault_oops
? __pte_offset_map_lock
? exc_page_fault
? asm_exc_page_fault
? wakeup_kswapd
migrate_misplaced_page
__handle_mm_fault
handle_mm_fault
do_user_addr_fault
exc_page_fault
asm_exc_page_fault

RIP: 0033:0x55b897ba0808
Code: (omitted)
RSP: 002b:00007ffeefa821a0 EFLAGS: 00010287
RAX: 000055b89983acd0 RBX: 00007ffeefa823f8 RCX: 000055b89983acd0
RDX: 00007fc2f8122010 RSI: 0000000000020000 RDI: 000055b89983acd0
RBP: 00007ffeefa821a0 R08: 0000000000000037 R09: 0000000000000075
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000000
R13: 00007ffeefa82410 R14: 000055b897ba5dd8 R15: 00007fc4b8340000
  </TASK>

This sounds like we need a Fixes: and want to CC stable?

Signature · 3 lines
Cheers,

David / dhildenb
#2 lore raw
Quoted 65 lines · David Hildenbrand
Quoted 11 lines · Byungchul Park

Changes from v1:
1. Trim the verbose oops in the commit message. (feedbacked by Phil Auld)
2. Rewrite a comment in code. (feedbacked by Phil Auld)

--->8---
From 6830b59db267a31b78f6f09af12ae0e3132b4bca Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 16:26:23 +0900
Subject: [PATCH v2] sched/numa, mm: do not promote folios to nodes not set N_MEMORY

Quoted 52 lines

While running qemu with a configuration where some CPUs don't have their local memory and with a kernel numa balancing on, the following oops has been observed. It's because of null pointers of ->zone_pgdat of zones of those nodes that are not initialized properly at booting time. So should avoid nodes not set N_MEMORY from getting promoted.

Quoted 45 lines
BUG: unable to handle page fault for address: 00000000000033f3
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: 0000 [#1] PREEMPT SMP NOPTI
CPU: 2 PID: 895 Comm: masim Not tainted 6.6.0-dirty #255
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
    rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014
RIP: 0010:wakeup_kswapd (./linux/mm/vmscan.c:7812)
Code: (omitted)
RSP: 0000:ffffc90004257d58 EFLAGS: 00010286
RAX: ffffffffffffffff RBX: ffff88883fff0480 RCX: 0000000000000003
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88883fff0480
RBP: ffffffffffffffff R08: ff0003ffffffffff R09: ffffffffffffffff
R10: ffff888106c95540 R11: 0000000055555554 R12: 0000000000000003
R13: 0000000000000000 R14: 0000000000000000 R15: ffff88883fff0940
FS:  00007fc4b8124740(0000) GS:ffff888827c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000000033f3 CR3: 000000026cc08004 CR4: 0000000000770ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
  <TASK>

? __die
? page_fault_oops
? __pte_offset_map_lock
? exc_page_fault
? asm_exc_page_fault
? wakeup_kswapd
migrate_misplaced_page
__handle_mm_fault
handle_mm_fault
do_user_addr_fault
exc_page_fault
asm_exc_page_fault

RIP: 0033:0x55b897ba0808
Code: (omitted)
RSP: 002b:00007ffeefa821a0 EFLAGS: 00010287
RAX: 000055b89983acd0 RBX: 00007ffeefa823f8 RCX: 000055b89983acd0
RDX: 00007fc2f8122010 RSI: 0000000000020000 RDI: 000055b89983acd0
RBP: 00007ffeefa821a0 R08: 0000000000000037 R09: 0000000000000075
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000000
R13: 00007ffeefa82410 R14: 000055b897ba5dd8 R15: 00007fc4b8340000
  </TASK>
Quoted 1 line

This sounds like we need a Fixes: and want to CC stable?

Sure. I will CC stable at the next spin - might need to rewrite the reason.

Byungchul

Quoted 4 lines
Signature · 3 lines
Cheers,

David / dhildenb