Skip to contentSkip to conversation

riscv: cpu-hotplug: clear cpu from numa map when teardown

PATCH In review Pingfan Liu 4 participants 5 messages Jan 23, 2022 – Feb 10, 2022 RISC-V

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 riscv: cpu-hotplug: clear cpu from numa map when teardown

There is numa_add_cpu() when cpus online, accordingly, there should be numa_remove_cpu() when cpus offline.

Cc Paul Walmsley <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc Albert Ou <[email protected]>
Cc Jisheng Zhang <[email protected]>

To: [email protected]

---
 arch/riscv/kernel/cpu-hotplug.c | 1 +
 1 file changed, 1 insertion(+)
arch/riscv/kernel/cpu-hotplug.c +1 -0
index be7f05b542bb..78d156a359ef 100644
@@ -40,6 +40,7 @@ int __cpu_disable(void)
 		return ret;
 
 	remove_cpu_topology(cpu);
+	numa_remove_cpu(cpu);
 	set_cpu_online(cpu, false);
 	irq_migrate_all_off_this_cpu();
 
Signature · 7 lines
2.31.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
maintainer
#1 lore raw
Quoted 18 lines · On Sun, 23 Jan 2022 04:13:52 PST (-0800), [email protected] wrote:

There is numa_add_cpu() when cpus online, accordingly, there should be numa_remove_cpu() when cpus offline.

Cc Paul Walmsley <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc Albert Ou <[email protected]>
Cc Jisheng Zhang <[email protected]>

To: [email protected]

---
 arch/riscv/kernel/cpu-hotplug.c | 1 +
 1 file changed, 1 insertion(+)
arch/riscv/kernel/cpu-hotplug.c +0 -0
index be7f05b542bb..78d156a359ef 100644

arch/riscv/kernel/cpu-hotplug.c int __cpu_disable(void) · line 45

Quoted 6 lines
arch/riscv/kernel/cpu-hotplug.c +1 -0
@@ -40,6 +40,7 @@ int __cpu_disable(void)
 		return ret;
 
 	remove_cpu_topology(cpu);
+	numa_remove_cpu(cpu);
 	set_cpu_online(cpu, false);
 	irq_migrate_all_off_this_cpu();

Thanks, this is on fixes. I don't really have any NUMA-related test setups, but if someone has a QEMU-based config that's meaningful then I'm happy to add it.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#2 lore raw
Quoted 22 lines · Palmer Dabbelt
Quoted 20 lines · On Sun, 23 Jan 2022 04:13:52 PST (-0800), [email protected] wrote:

There is numa_add_cpu() when cpus online, accordingly, there should be numa_remove_cpu() when cpus offline.

Cc Paul Walmsley <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc Albert Ou <[email protected]>
Cc Jisheng Zhang <[email protected]>

To: [email protected]

---
 arch/riscv/kernel/cpu-hotplug.c | 1 +
 1 file changed, 1 insertion(+)
arch/riscv/kernel/cpu-hotplug.c +0 -0
index be7f05b542bb..78d156a359ef 100644
@@ -40,6 +40,7 @@ int __cpu_disable(void)
              return ret;
 
Quoted 8 lines

arch/riscv/kernel/cpu-hotplug.c int __cpu_disable(void) · line 45

Quoted 4 lines
arch/riscv/kernel/cpu-hotplug.c +1 -0
      remove_cpu_topology(cpu);
+     numa_remove_cpu(cpu);
      set_cpu_online(cpu, false);
      irq_migrate_all_off_this_cpu();

Thanks, this is on fixes. I don't really have any NUMA-related test setups, but if someone has a QEMU-based config that's meaningful then I'm happy to add it.

Sorry that I have no such things even riscv machine. I did some hotplug stuff, compared arm/arm64/riscv, and found the difference.

But since the code path should be symmetric, I think the hot-removing path should revert the hot-adding path.

Thanks,

Pingfan

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#3 lore raw
In reply to Palmer Dabbelt
Quoted 21 lines · Palmer Dabbelt
Quoted 20 lines · On Sun, 23 Jan 2022 04:13:52 PST (-0800), [email protected] wrote:

There is numa_add_cpu() when cpus online, accordingly, there should be numa_remove_cpu() when cpus offline.

Cc Paul Walmsley <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc Albert Ou <[email protected]>
Cc Jisheng Zhang <[email protected]>

To: [email protected]

---
 arch/riscv/kernel/cpu-hotplug.c | 1 +
 1 file changed, 1 insertion(+)
arch/riscv/kernel/cpu-hotplug.c +0 -0
index be7f05b542bb..78d156a359ef 100644
@@ -40,6 +40,7 @@ int __cpu_disable(void)
 		return ret;
 
Quoted 8 lines

arch/riscv/kernel/cpu-hotplug.c int __cpu_disable(void) · line 45

Quoted 4 lines
arch/riscv/kernel/cpu-hotplug.c +1 -0
 	remove_cpu_topology(cpu);
+	numa_remove_cpu(cpu);
 	set_cpu_online(cpu, false);
 	irq_migrate_all_off_this_cpu();

Thanks, this is on fixes. I don't really have any NUMA-related test setups, but if someone has a QEMU-based config that's meaningful then I'm happy to add it.

I think I accidantally pushed this to the public branch rather than my staging branch, so I missed a build issue (missing numa include in some configurations). I've force pushed a version of the patch with the fix, it's included here for reference -- I generally don't like to force push fixes, but given that it's a pretty trivial build failure I think that's the best way to go here.

commit f40fe31c01445f31253b15bef2412b33ae31093b

gpg: Signature made Thu 10 Feb 2022 09:17:00 AM PST
gpg:                using RSA key 2B3C3747446843B24A943A7A2E1319F35FBB1889
gpg:                issuer "[email protected]"
gpg: Good signature from "Palmer Dabbelt <[email protected]>" [ultimate]
gpg:                 aka "Palmer Dabbelt <[email protected]>" [ultimate]
Author: Pingfan Liu <[email protected]>
Date:   Sun Jan 23 20:13:52 2022 +0800

riscv: cpu-hotplug: clear cpu from numa map when teardown

There is numa_add_cpu() when cpus online, accordingly, there should be numa_remove_cpu() when cpus offline.

Fixes 4f0e8eef772e ("riscv: Add numa support for riscv64 platform")
[Palmer: Add missing NUMA include]
arch/riscv/kernel/cpu-hotplug.c +0 -0
index be7f05b542bb..f7a832e3a1d1 100644
@@ -12,6 +12,7 @@

#include <linux/sched/hotplug.h>
#include <asm/irq.h>
#include <asm/cpu_ops.h>
+#include <asm/numa.h>
#include <asm/sbi.h>

bool cpu_has_hotplug(unsigned int cpu)
@@ -40,6 +41,7 @@ int __cpu_disable(void)

		return ret;

	remove_cpu_topology(cpu);
+	numa_remove_cpu(cpu);
	set_cpu_online(cpu, false);
	irq_migrate_all_off_this_cpu();

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Tested-by
#4 lore raw
In reply to Palmer Dabbelt
Quoted 22 lines · Palmer Dabbelt
Quoted 20 lines · On Sun, 23 Jan 2022 04:13:52 PST (-0800), [email protected] wrote:

There is numa_add_cpu() when cpus online, accordingly, there should be numa_remove_cpu() when cpus offline.

Cc Paul Walmsley <[email protected]>
Cc Palmer Dabbelt <[email protected]>
Cc Albert Ou <[email protected]>
Cc Jisheng Zhang <[email protected]>

To: [email protected]

---
 arch/riscv/kernel/cpu-hotplug.c | 1 +
 1 file changed, 1 insertion(+)
arch/riscv/kernel/cpu-hotplug.c +0 -0
index be7f05b542bb..78d156a359ef 100644
@@ -40,6 +40,7 @@ int __cpu_disable(void)
              return ret;
 
Quoted 9 lines

arch/riscv/kernel/cpu-hotplug.c int __cpu_disable(void) · line 45

Quoted 4 lines
arch/riscv/kernel/cpu-hotplug.c +1 -0
      remove_cpu_topology(cpu);
+     numa_remove_cpu(cpu);
      set_cpu_online(cpu, false);
      irq_migrate_all_off_this_cpu();

Thanks, this is on fixes. I don't really have any NUMA-related test setups, but if someone has a QEMU-based config that's meaningful then I'm happy to add it.

lscpu/numactl already checks the online cpu list and prints correct output without the fix which explains why I did not notice this earlier.

This is the correct thing to do anyways. Thanks for the fix.

Here is my qemu config
qemu-system-riscv64 -cpu rv64 -M virt -m 1G -smp 4 \
-objectmemory-backend-ram,size=512M,policy=bind,host-nodes=0,id=ram-node0 \
-numa node,memdev=ram-node0 \
-object memory-backend-ram,size=512M,policy=bind,host-nodes=0,id=ram-node1 \
-numa node,memdev=ram-node1

Quoted 4 lines

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Signature · 7 lines
Regards,
Atish

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv