Skip to contentSkip to conversation

arch/sparc: support NR_CPUS = 4096

PATCH Superseded Jane Chu 2 participants 5 messages Jun 1, 2017 – Jun 5, 2017 SPARC + UltraSPARC (sparc/sparc64)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Jane Chu

author
#0 lore raw

PATCH 1/1 arch/sparc: support NR_CPUS = 4096

Linux SPARC64 limits NR_CPUS to 4064 because init_cpu_send_mondo_info() only allocates a single page for NR_CPUS mondo entries. Thus we cannot use all 4096 CPUs on some SPARC platforms.

To fix, allocate (2^order) pages where order is set according to the size of cpu_list for possible cpus. Since cpu_list_pa and cpu_mondo_block_pa are not used in asm code, there are no imm13 offsets from the base PA that will break because they can only reach one page.

Orabug: 25505750

---
 arch/sparc/Kconfig         |    4 ++--
 arch/sparc/kernel/irq_64.c |    8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)
arch/sparc/Kconfig +2 -2
index 58243b0..4399be7 100644
@@ -192,9 +192,9 @@ config NR_CPUS
 	int "Maximum number of CPUs"
 	depends on SMP
 	range 2 32 if SPARC32
-	range 2 1024 if SPARC64
+	range 2 4096 if SPARC64
 	default 32 if SPARC32
-	default 64 if SPARC64
+	default 4096 if SPARC64
 
 source kernel/Kconfig.hz
 
arch/sparc/kernel/irq_64.c +4 -4
index 4d0248a..5b19108 100644
@@ -1034,12 +1034,12 @@ static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb)
 {
 #ifdef CONFIG_SMP
 	unsigned long page;
+	unsigned int order;
 
-	BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > (PAGE_SIZE - 64));
-
-	page = get_zeroed_page(GFP_KERNEL);
+	order = get_order(num_possible_cpus() * sizeof(u16) + 64);
+	page = __get_free_pages(GFP_KERNEL | __GFP_ZERO, order);
 	if (!page) {
-		prom_printf("SUN4V: Error, cannot allocate cpu mondo page.\n");
+		prom_printf("SUN4V: Error, cannot allocate cpu mondo pages.\n");
 		prom_halt();
 	}
 
Signature · 1 line
1.7.1
maintainer
#1 lore raw

From: Jane Chu <[email protected]>
Date: Thu, 1 Jun 2017 15:39:13 -0600

Quoted 6 lines
arch/sparc/kernel/irq_64.c +0 -0
index 4d0248a..5b19108 100644
@@ -1034,12 +1034,12 @@ static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb)
 {

arch/sparc/kernel/irq_64.c static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb) · line 1041

Quoted 10 lines
arch/sparc/kernel/irq_64.c +3 -3
 #ifdef CONFIG_SMP
 	unsigned long page;
+	unsigned int order;
 
-	BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > (PAGE_SIZE - 64));
-
-	page = get_zeroed_page(GFP_KERNEL);
+	order = get_order(num_possible_cpus() * sizeof(u16) + 64);
+	page = __get_free_pages(GFP_KERNEL | __GFP_ZERO, order);
 	if (!page) {

The only reason we allocated these two items together was because it was convenient and it all fit into a single page.

Since it now doesn't, it makes sense to split it up.

Simply use a single page for the cpu_list_pa and kzalloc for the cpu_mondo_block.

This also allows to keep the BUILD_BUG_ON(), which I really wish you hadn't tried to remove.

static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb) {

#ifdef CONFIG_SMP
	unsigned long page;
	void *mondo;

	BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > PAGE_SIZE);

	mondo = kzalloc(64, GFP_KERNEL);
	if (!mondo) {
		prom_printf("SUN4V: Error, cannot allocate mondo block.\n");
		prom_halt();
	}
	tb->cpu_mondo_block_pa = __pa(mondo);

	page = get_zeroed_page(GFP_KERNEL);
	if (!page) {
		prom_printf("SUN4V: Error, cannot allocate cpu list page.\n");
		prom_halt();
	}
	tb->cpu_list_pa = __pa(page);

#endif
}

Thanks.

#2 lore raw

Hi, David,

Quoted 22 lines · David Miller

From: Jane Chu <[email protected]>
Date: Thu, 1 Jun 2017 15:39:13 -0600

Quoted 6 lines
arch/sparc/kernel/irq_64.c +0 -0
index 4d0248a..5b19108 100644
@@ -1034,12 +1034,12 @@ static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb)
  {

arch/sparc/kernel/irq_64.c static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb) · line 1041

Quoted 10 lines
arch/sparc/kernel/irq_64.c +3 -3
  #ifdef CONFIG_SMP
  	unsigned long page;
+	unsigned int order;
 
-	BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > (PAGE_SIZE - 64));
-
-	page = get_zeroed_page(GFP_KERNEL);
+	order = get_order(num_possible_cpus() * sizeof(u16) + 64);
+	page = __get_free_pages(GFP_KERNEL | __GFP_ZERO, order);
  	if (!page) {

The only reason we allocated these two items together was because it was convenient and it all fit into a single page.

Quoted 7 lines

Since it now doesn't, it makes sense to split it up.

Simply use a single page for the cpu_list_pa and kzalloc for the cpu_mondo_block.

This also allows to keep the BUILD_BUG_ON(), which I really wish you hadn't tried to remove.

Good point! I will just add a comment for future maintenance that the mondo trap block needs to be 64byte aligned.

Quoted 22 lines

static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb) {

#ifdef CONFIG_SMP
	unsigned long page;
	void *mondo;

	BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > PAGE_SIZE);

	mondo = kzalloc(64, GFP_KERNEL);
	if (!mondo) {
		prom_printf("SUN4V: Error, cannot allocate mondo block.\n");
		prom_halt();
	}
	tb->cpu_mondo_block_pa = __pa(mondo);

	page = get_zeroed_page(GFP_KERNEL);
	if (!page) {
		prom_printf("SUN4V: Error, cannot allocate cpu list page.\n");
		prom_halt();
	}
	tb->cpu_list_pa = __pa(page);
Quoted 2 lines

#endif
}

Thanks a lot!

-jane

Quoted 2 lines

Thanks.

#3 lore raw
In reply to David Miller

Hi,

I have a question about a checkpatch.pl warning against the

use of NR_CPUS -

$ ./scripts/checkpatch.pl diff.patch
WARNING: usage of NR_CPUS is often wrong - consider using 
cpu_possible(), num_possible_cpus(), for_each_possible_cpu(), etc
#14: FILE: arch/sparc/kernel/irq_64.c:1039:
+       BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > PAGE_SIZE);

num_possible_cpus() is the hard limit on a domain without platform reboot,

it is capped by both NR_CPUS and 'max_cpus' (a platform property in the

machine description). So it is theoretically possible to have NR_CPUS>4096

but num_possible_cpus() < 4096 -- a scenario I think we'd prefer to reject.

Is it okay to ignore the checkpatch.pl warning in this case?

thanks,

-jane

Quoted 55 lines · David Miller

From: Jane Chu <[email protected]>
Date: Thu, 1 Jun 2017 15:39:13 -0600

Quoted 6 lines
arch/sparc/kernel/irq_64.c +0 -0
index 4d0248a..5b19108 100644
@@ -1034,12 +1034,12 @@ static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb)
  {

arch/sparc/kernel/irq_64.c static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb) · line 1041

Quoted 10 lines
arch/sparc/kernel/irq_64.c +3 -3
  #ifdef CONFIG_SMP
  	unsigned long page;
+	unsigned int order;
 
-	BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > (PAGE_SIZE - 64));
-
-	page = get_zeroed_page(GFP_KERNEL);
+	order = get_order(num_possible_cpus() * sizeof(u16) + 64);
+	page = __get_free_pages(GFP_KERNEL | __GFP_ZERO, order);
  	if (!page) {

The only reason we allocated these two items together was because it was convenient and it all fit into a single page.

Since it now doesn't, it makes sense to split it up.

Simply use a single page for the cpu_list_pa and kzalloc for the cpu_mondo_block.

This also allows to keep the BUILD_BUG_ON(), which I really wish you hadn't tried to remove.

static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb) {

#ifdef CONFIG_SMP
	unsigned long page;
	void *mondo;

	BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > PAGE_SIZE);

	mondo = kzalloc(64, GFP_KERNEL);
	if (!mondo) {
		prom_printf("SUN4V: Error, cannot allocate mondo block.\n");
		prom_halt();
	}
	tb->cpu_mondo_block_pa = __pa(mondo);

	page = get_zeroed_page(GFP_KERNEL);
	if (!page) {
		prom_printf("SUN4V: Error, cannot allocate cpu list page.\n");
		prom_halt();
	}
	tb->cpu_list_pa = __pa(page);

#endif
}

Thanks.

maintainer
#4 lore raw

From: [email protected]
Date: Mon, 5 Jun 2017 14:32:10 -0700

Quoted 1 line

Is it okay to ignore the checkpatch.pl warning in this case?

I absolutely think you can ignore this warning.

Thanks.