Skip to contentSkip to conversation

migration cost tune up in sparc smp

PATCH In review Krzysztof Helt 3 participants 4 messages Jun 5, 2006 – Jun 12, 2006 SPARC + UltraSPARC (sparc/sparc64)

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 migration cost tune up in sparc smp

This patch sets the max_cache_size value required to tune up scheduler in SMP systems. Otherwise, the calculated migration_cost is too high and task scheduling may lock up.

---

This patch is for 2.6.17-rc5 kernel. This patch helps with Bob Breuer's patch. Default cache size (20MB) leads to calculation of the migration cost 20000 (20msecs). The calculated cost of migration is 4000 (4msecs) on dual SM81 with the patch.

The same trick is used with setting up max_cache_size on i386 architecture. I assume the value should be set to on sparc64 as well to correctly tune up the scheduler.

Regards,
Krzysztof

----------------------------------------------------
Nowe serie superksiążek o Czarodziejkach WITCH!
Poznaj świat bohaterek WITCH!
Will, Irma, Taranee, Cornelia i Hay-Lin zapraszają do księgarni.
http://klik.wp.pl/?adr=http%3A%2F%2Fadv.reklama.wp.pl%2Fas%2Fwitch.html&sid=781

arch/sparc/kernel/smp.c +11 -0
diff -urp linux-2.6.16/arch/sparc/kernel/smp.c linux-2.6.17/arch/sparc/kernel/smp.c
@@ -69,6 +69,17 @@ void __init smp_store_cpu_info(int id)
 						     "clock-frequency", 0);
 	cpu_data(id).prom_node = cpu_node;
 	cpu_data(id).mid = cpu_get_hwmid(cpu_node);
+
+	/* this is required to tune the scheduler correctly */
+	/* is it possible to have CPUs with different cache sizes? */
+	if (id == boot_cpu_id) {
+		int cache_line,cache_nlines;
+		cache_line = 0x20;
+		cache_line = prom_getintdefault(cpu_node, "ecache-line-size", cache_line);
+		cache_nlines = 0x8000;
+		cache_nlines = prom_getintdefault(cpu_node, "ecache-nlines", cache_nlines);
+		max_cache_size = cache_line * cache_nlines;
+	}
 	if (cpu_data(id).mid < 0)
 		panic("No MID found for CPU%d at node 0x%08d", id, cpu_node);
 }
maintainer
#1 lore raw

From: "Krzysztof Helt" <[email protected]>
Date: Mon, 05 Jun 2006 09:21:15 +0200

Quoted 4 lines

This patch sets the max_cache_size value required to tune up scheduler in SMP systems. Otherwise, the calculated migration_cost is too high and task scheduling may lock up.

Thanks for this, yes sparc64 needs this too.

But why in the world does the scheduler "lock up" just because the migration cost is too large?

It should continue to function properly even if the migration cost is very large.

#2 lore raw
In reply to Krzysztof Helt
Quoted 2 lines · David Miller

But why in the world does the scheduler "lock up" just because the migration cost is too large?

There is an address space collision between the vmalloc area and where the prom maps the framebuffer.

The vmalloc range is 0xfe600000 to 0xffc00000. I have a cg6 that gets mapped to 0xfee00000, and a cg14 with 8MB vsimm that gets mapped to 0xfe700000. These leave only 8MB and 1MB of usable vmalloc space before bad things happen.

We may want to find a different vm hole to put vmalloc into. Is there any documentation on what address spaces are reserved with the different prom versions?

Bob

maintainer
#3 lore raw
In reply to Krzysztof Helt

From: Bob Breuer <[email protected]>
Date: Sun, 11 Jun 2006 20:21:09 -0500

Quoted 6 lines
Quoted 2 lines · David Miller

But why in the world does the scheduler "lock up" just because the migration cost is too large?

There is an address space collision between the vmalloc area and where the prom maps the framebuffer.

And this is what locks up the scheduler when a bad migration cost calculation is made?

Please start a new thread if you are talking about some new topic, it gets confusing :-/

Quoted 8 lines

The vmalloc range is 0xfe600000 to 0xffc00000. I have a cg6 that gets mapped to 0xfee00000, and a cg14 with 8MB vsimm that gets mapped to 0xfe700000. These leave only 8MB and 1MB of usable vmalloc space before bad things happen.

We may want to find a different vm hole to put vmalloc into. Is there any documentation on what address spaces are reserved with the different prom versions?

I don't know, but from your report it seems that OBP takes 0xfe000000 up to 0xfff00000.

Actually, you can look at the "available" property of the virtual-memory node. Or, you can see if there is a "translations" node in the /virtual-memory node like there is on sparc64.