Skip to contentSkip to conversation

Pre-emption patch for IA-64

Linux-ia64 Mario Smarduch 6 participants 11 messages Mar 14, 2003 – Mar 28, 2003 Agpgart driver

lore
Share & tools
lore
#0 lore raw

Hi,
we're trying to build a reliable low-latency system for use
in telco call processing carrier grade environments (on rx2600 NEBS like servers) . One of the patches we're heavily relying on is the 'Kernel Preemption' patch. First of we're interested in the
(i) status of this patch on IA-64 (ii) any issues that you may have ran into.

Our understanding is that the 'Preemption Patch' offers a systematic scheduling checkpoint vs. 'low latency patch' which examines the kernel for any long running code and based on that preemption checks are inserted in the kernel proper only (kernel.org) but addon packages may introduce long latencies. While we plan to run with 1mS clock tick, low latency and the preemption paches, the preemption patch is viewed as a must to gurantee a bound on 'soft-realtime response'. Please point out if these assumption are somewhat correct or is too much emphasis being placed on the preemption patch, the soft-realtime response we're looking for is 1mS.

thanks,
Mario.

#1 lore raw

Peter Chubb has been working on adding preemption support to 2.5. I applied his patch in the 2.5.64 patch. I'm sure he would appreciate more people beating on it.

	--david
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Mario Smarduch

Mario> Hi, we're trying to build a reliable low-latency system for Mario> use in telco call processing carrier grade environments (on Mario> rx2600 NEBS like servers) . One of the patches we're heavily Mario> relying on is the 'Kernel Preemption' patch. First of we're Mario> interested in the (i) status of this patch on IA-64 (ii) any Mario> issues that you may have ran into.

Mario> Our understanding is that the 'Preemption Patch' offers a Mario> systematic scheduling checkpoint vs. 'low latency patch' Mario> which examines the kernel for any long running code and based Mario> on that preemption checks are inserted in the kernel proper Mario> only (kernel.org) but addon packages may introduce long Mario> latencies. While we plan to run with 1mS clock tick, low Mario> latency and the preemption paches, the preemption patch is Mario> viewed as a must to gurantee a bound on 'soft-realtime Mario> response'. Please point out if these assumption are somewhat Mario> correct or is too much emphasis being placed on the Mario> preemption patch, the soft-realtime response we're looking Mario> for is 1mS.

#2 lore raw
In reply to Mario Smarduch

Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok with the following exceptions:
o the AT keyboard support seems broken (characters are slow to show up on the screen and are sometimes repeated)
o agp support for 460GX causes a link failure

But by using a USB keyboard and turning off agp support, I was able to get an SMP+preempt kernel going, and it seems fine.

Jesse

Quoted 33 lines · David Mosberger

Peter Chubb has been working on adding preemption support to 2.5. I applied his patch in the 2.5.64 patch. I'm sure he would appreciate more people beating on it.

	--david
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Mario Smarduch

Mario> Hi, we're trying to build a reliable low-latency system for Mario> use in telco call processing carrier grade environments (on Mario> rx2600 NEBS like servers) . One of the patches we're heavily Mario> relying on is the 'Kernel Preemption' patch. First of we're Mario> interested in the (i) status of this patch on IA-64 (ii) any Mario> issues that you may have ran into.

Mario> Our understanding is that the 'Preemption Patch' offers a Mario> systematic scheduling checkpoint vs. 'low latency patch' Mario> which examines the kernel for any long running code and based Mario> on that preemption checks are inserted in the kernel proper Mario> only (kernel.org) but addon packages may introduce long Mario> latencies. While we plan to run with 1mS clock tick, low Mario> latency and the preemption paches, the preemption patch is Mario> viewed as a must to gurantee a bound on 'soft-realtime Mario> response'. Please point out if these assumption are somewhat Mario> correct or is too much emphasis being placed on the Mario> preemption patch, the soft-realtime response we're looking Mario> for is 1mS.

_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64

#3 lore raw
In reply to Mario Smarduch
Quoted 5 lines · Jesse Barnes

Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok with the following exceptions:
o the AT keyboard support seems broken (characters are slow to show up on the screen and are sometimes repeated)
o agp support for 460GX causes a link failure

2.4 had a problem that seems to be a compiler issue with some AGP code. I don't know whether this is the same problem, but if it is, I applied the following patch as a workaround:

drivers/char/agp/agpgart_be.c +6 -14
diff -Nru a/drivers/char/agp/agpgart_be.c b/drivers/char/agp/agpgart_be.c
@@ -341,26 +341,18 @@
 
 	temp = agp_bridge.current_size;
 
-	switch (agp_bridge.size_type) {
-	case U8_APER_SIZE:
+	if (agp_bridge.size_type = U8_APER_SIZE)
 		current_size = A_SIZE_8(temp)->size;
-		break;
-	case U16_APER_SIZE:
+	else if (agp_bridge.size_type =  U16_APER_SIZE)
 		current_size = A_SIZE_16(temp)->size;
-		break;
-	case U32_APER_SIZE:
+	else if (agp_bridge.size_type =  U32_APER_SIZE)
 		current_size = A_SIZE_32(temp)->size;
-		break;
-	case LVL2_APER_SIZE:
+	else if (agp_bridge.size_type =  LVL2_APER_SIZE)
 		current_size = A_SIZE_LVL2(temp)->size;
-		break;
-	case FIXED_APER_SIZE:
+	else if (agp_bridge.size_type =  FIXED_APER_SIZE)
 		current_size = A_SIZE_FIX(temp)->size;
-		break;
-	default:
+	else
 		current_size = 0;
-		break;
-	}
 
 	current_size -= (agp_memory_reserved / (1024*1024));
 
#4 lore raw
In reply to Mario Smarduch
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Jesse" = Jesse Barnes <[email protected]> writes:

Jesse> Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok Jesse> with the following exceptions:
Jesse> o the AT keyboard support seems broken (characters are slow to show Jesse> up on the screen and are sometimes repeated)
Jesse> o agp support for 460GX causes a link failure

AT keyboard and PS/2 mouse both!

Jesse> But by using a USB keyboard and turning off agp support, I was able to Jesse> get an SMP+preempt kernel going, and it seems fine.

On a dual McKinley prototype machine I was able to get lockups with preempt under heavy I/O and cpu load (e.g., make -j4 in glibc). I'm waiting for production model McKinleys SMP to retry, as I couldn't reproduce the problem on production-model Itanium 1, and there were no symptoms that I could debug.

Also FYI, if you're using posix threads in userland, the port of the next generation library is almost complete, and, with the kernel preemption patch, will give much lower latencies for heavily threaded code, depending on exactly what you're doing.

--
Dr Peter Chubb [email protected]
You are lost in a maze of BitKeeper repositories, all almost the same.

#5 lore raw
In reply to Mario Smarduch
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Peter Chubb

Peter> On a dual McKinley prototype machine I was able to get Peter> lockups with preempt under heavy I/O and cpu load (e.g., make Peter> -j4 in glibc). I'm waiting for production model McKinleys Peter> SMP to retry, as I couldn't reproduce the problem on Peter> production-model Itanium 1, and there were no symptoms that I Peter> could debug.

Do you know what on the zx1-based machines you can trigger INIT via the BMC? That's quite handy when the kernel locks up (you can dump the collected state at the next boot).

	--david
#6 lore raw
In reply to Mario Smarduch
Quoted 10 lines · Bjorn Helgaas
Quoted 5 lines · Jesse Barnes

Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok with the following exceptions:
o the AT keyboard support seems broken (characters are slow to show up on the screen and are sometimes repeated)
o agp support for 460GX causes a link failure

2.4 had a problem that seems to be a compiler issue with some AGP code. I don't know whether this is the same problem, but if it is, I applied the following patch as a workaround:

Thanks for the pointer. Here's a patch against 2.5.64. It builds and boots, but my XFree dri/glx/whatever drivers seem broken, so I'm not sure if it's really working like it should.

Btw Peter, thanks for doing the preemption patch. No hangs yet for me...

Jesse

drivers/char/agp/generic.c +19 -42
diff -Naur -X /usr/people/jbarnes/dontdiff linux-2.5.64-ia64-stock/drivers/char/agp/generic.c linux-2.5.64-ia64/drivers/char/agp/generic.c
@@ -34,6 +34,7 @@
 #include <linux/miscdevice.h>
 #include <linux/pm.h>
 #include <linux/agp_backend.h>
+#include <linux/vmalloc.h>
 #include "agp.h"
 
 __u32 *agp_gatt_table; 
@@ -169,26 +170,18 @@
 
 	temp = agp_bridge->current_size;
 
-	switch (agp_bridge->size_type) {
-	case U8_APER_SIZE:
+	if (agp_bridge->size_type = U8_APER_SIZE)
 		current_size = A_SIZE_8(temp)->size;
-		break;
-	case U16_APER_SIZE:
+	else if (agp_bridge->size_type = U16_APER_SIZE)
 		current_size = A_SIZE_16(temp)->size;
-		break;
-	case U32_APER_SIZE:
+	else if (agp_bridge->size_type = U32_APER_SIZE)
 		current_size = A_SIZE_32(temp)->size;
-		break;
-	case LVL2_APER_SIZE:
+	else if (agp_bridge->size_type = LVL2_APER_SIZE)
 		current_size = A_SIZE_LVL2(temp)->size;
-		break;
-	case FIXED_APER_SIZE:
+	else if (agp_bridge->size_type = FIXED_APER_SIZE)
 		current_size = A_SIZE_FIX(temp)->size;
-		break;
-	default:
+	else
 		current_size = 0;
-		break;
-	}
 
 	current_size -= (agp_memory_reserved / (1024*1024));
 	if (current_size <0)
@@ -203,26 +196,18 @@
 
 	temp = agp_bridge->current_size;
 
-	switch (agp_bridge->size_type) {
-	case U8_APER_SIZE:
+	if (agp_bridge->size_type = U8_APER_SIZE)
 		num_entries = A_SIZE_8(temp)->num_entries;
-		break;
-	case U16_APER_SIZE:
+	else if (agp_bridge->size_type = U16_APER_SIZE)
 		num_entries = A_SIZE_16(temp)->num_entries;
-		break;
-	case U32_APER_SIZE:
+	else if (agp_bridge->size_type = U32_APER_SIZE)
 		num_entries = A_SIZE_32(temp)->num_entries;
-		break;
-	case LVL2_APER_SIZE:
+	else if (agp_bridge->size_type = LVL2_APER_SIZE)
 		num_entries = A_SIZE_LVL2(temp)->num_entries;
-		break;
-	case FIXED_APER_SIZE:
+	else if (agp_bridge->size_type = FIXED_APER_SIZE)
 		num_entries = A_SIZE_FIX(temp)->num_entries;
-		break;
-	default:
+	else
 		num_entries = 0;
-		break;
-	}
 
 	num_entries -= agp_memory_reserved>>PAGE_SHIFT;
 	if (num_entries<0)
@@ -546,27 +531,19 @@
 
 	temp = agp_bridge->current_size;
 
-	switch (agp_bridge->size_type) {
-	case U8_APER_SIZE:
+	if (agp_bridge->size_type = U8_APER_SIZE)
 		page_order = A_SIZE_8(temp)->page_order;
-		break;
-	case U16_APER_SIZE:
+	else if (agp_bridge->size_type = U16_APER_SIZE)
 		page_order = A_SIZE_16(temp)->page_order;
-		break;
-	case U32_APER_SIZE:
+	else if (agp_bridge->size_type = U32_APER_SIZE)
 		page_order = A_SIZE_32(temp)->page_order;
-		break;
-	case FIXED_APER_SIZE:
+	else if (agp_bridge->size_type = FIXED_APER_SIZE)
 		page_order = A_SIZE_FIX(temp)->page_order;
-		break;
-	case LVL2_APER_SIZE:
+	else if (agp_bridge->size_type = LVL2_APER_SIZE)
 		/* The generic routines can't deal with 2 level gatt's */
 		return -EINVAL;
-		break;
-	default:
+	else
 		page_order = 0;
-		break;
-	}
 
 	/* Do not worry about freeing memory, because if this is
 	 * called, then all agp memory is deallocated and removed
#8 lore raw
In reply to Mario Smarduch

So it appears like drivers are having issues and that's to be expected if they're not reentrant. But what about the kernel proper, has anyone observed issues there or knows of any outstanding issues?

- Mario.

Quoted 32 lines · Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Jesse" = Jesse Barnes <[email protected]> writes:

Jesse> Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok Jesse> with the following exceptions:
Jesse> o the AT keyboard support seems broken (characters are slow to show Jesse> up on the screen and are sometimes repeated)
Jesse> o agp support for 460GX causes a link failure

AT keyboard and PS/2 mouse both!

Jesse> But by using a USB keyboard and turning off agp support, I was able to Jesse> get an SMP+preempt kernel going, and it seems fine.

On a dual McKinley prototype machine I was able to get lockups with preempt under heavy I/O and cpu load (e.g., make -j4 in glibc). I'm waiting for production model McKinleys SMP to retry, as I couldn't reproduce the problem on production-model Itanium 1, and there were no symptoms that I could debug.

Also FYI, if you're using posix threads in userland, the port of the next generation library is almost complete, and, with the kernel preemption patch, will give much lower latencies for heavily threaded code, depending on exactly what you're doing.

--
Dr Peter Chubb [email protected]
You are lost in a maze of BitKeeper repositories, all almost the same.

_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64

#9 lore raw
In reply to Mario Smarduch

My box is still running strong, so no kernel issues that I know of (aside from the kernel eating all my CPU time whenever someone mounts an exported nfs volume)

[jbarnes@spamtin jbarnes]$ uname -a;uptime
Linux spamtin 2.5.64 #7 SMP Fri Mar 14 14:10:36 PST 2003 ia64 unknown
15:33:01 up 13 days, 51 min, 7 users, load average: 0.70, 0.30, 0.14

Thanks,
Jesse

Quoted 42 lines · Mario Smarduch

So it appears like drivers are having issues and that's to be expected if they're not reentrant. But what about the kernel proper, has anyone observed issues there or knows of any outstanding issues?

- Mario.

Quoted 32 lines · Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Jesse" = Jesse Barnes <[email protected]> writes:

Jesse> Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok Jesse> with the following exceptions:
Jesse> o the AT keyboard support seems broken (characters are slow to show Jesse> up on the screen and are sometimes repeated)
Jesse> o agp support for 460GX causes a link failure

AT keyboard and PS/2 mouse both!

Jesse> But by using a USB keyboard and turning off agp support, I was able to Jesse> get an SMP+preempt kernel going, and it seems fine.

On a dual McKinley prototype machine I was able to get lockups with preempt under heavy I/O and cpu load (e.g., make -j4 in glibc). I'm waiting for production model McKinleys SMP to retry, as I couldn't reproduce the problem on production-model Itanium 1, and there were no symptoms that I could debug.

Also FYI, if you're using posix threads in userland, the port of the next generation library is almost complete, and, with the kernel preemption patch, will give much lower latencies for heavily threaded code, depending on exactly what you're doing.

--
Dr Peter Chubb [email protected]
You are lost in a maze of BitKeeper repositories, all almost the same.

_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64

#10 lore raw
In reply to Mario Smarduch

Mario> So it appears like drivers are having issues and that's to be Mario> expected if they're not reentrant. But what about the kernel Mario> proper, has anyone observed issues there or knows of any Mario> outstanding issues?

There are no issues I'm aware of, even in drivers, with the preeption patch. The driver issues with the PS/2 mouse and AT keyboard seem to be generic, not related to preemption support.

--
Dr Peter Chubb [email protected]
You are lost in a maze of BitKeeper repositories, all almost the same.