local apic timer ints not working with vmware: nolocala

Just get VMware 3 or 4 - they (properly) emulate APIC timer and you'll get information that host bus is running at 66.xxxx MHz. With VMware 2 you have to boot with noapic.
Petr Vandrovec

Unfortunately, I cannot currently afford to buy (outright -- there is no upgrade path from 2 to 4) a newer version of VMware. Really, other than this issue of the local APIC timer, I don't really need or care to have a newer version of VMware anyway.

If only this worked. I tried noapic, but it still tries to use the local APIC timer interrupts. noapic seems to only disable IO-APIC. That is why I was "proposing" a new kernel command line arg, "nolocalapic".

b.

Brian J. Murrell

I submitted a patch for nolapic before...

Index: linux-2.5.45-ac1/arch/i386/kernel/apic.c
===================================================================
RCS file: /build/cvsroot/linux-2.5.45-ac1/arch/i386/kernel/apic.c,v retrieving revision 1.1.1.1

diff -u -r1.1.1.1 apic.c
@@ -609,11 +609,27 @@
 
 #endif	/* CONFIG_PM */
 
+int enable_local_apic_flag __initdata = 0; /* 0=probe, 1=force, 2=disable e.g. DMI */
+
+static int __init nolapic_setup(char *str)
+{
+	enable_local_apic_flag = 2;
+	return 1;
+}
+
+static int __init lapic_setup(char *str)
+{
+	enable_local_apic_flag = 1;
+	return 1;
+}
+
+__setup("nolapic", nolapic_setup);
+__setup("lapic", lapic_setup);
+
 /*
  * Detect and enable local APICs on non-SMP boards.
  * Original code written by Keir Fraser.
  */
-int dont_enable_local_apic __initdata = 0;
 
 static int __init detect_init_APIC (void)
 {

@@ -621,11 +637,14 @@
 	extern void get_cpu_vendor(struct cpuinfo_x86*);
 
 	/* Disabled by DMI scan or kernel option? */
-	if (dont_enable_local_apic)
+	if (enable_local_apic_flag == 2)
 		return -1;
 
 	/* Workaround for us being called before identify_cpu(). */
 	get_cpu_vendor(&boot_cpu_data);
+	
+	if (enable_local_apic_flag == 1)
+		goto force_apic;
 
 	switch (boot_cpu_data.x86_vendor) {
 	case X86_VENDOR_AMD:

@@ -642,6 +661,7 @@
 		goto no_apic;
 	}
 
+force_apic:
 	if (!cpu_has_apic) {
 		/*
 		 * Some BIOSes disable the local APIC in the

Index: linux-2.5.45-ac1/arch/i386/kernel/dmi_scan.c
===================================================================
RCS file: /build/cvsroot/linux-2.5.45-ac1/arch/i386/kernel/dmi_scan.c,v retrieving revision 1.1.1.1

diff -u -r1.1.1.1 dmi_scan.c
@@ -314,9 +314,9 @@
 static int __init local_apic_kills_bios(struct dmi_blacklist *d)
 {
 #ifdef CONFIG_X86_LOCAL_APIC
-	extern int dont_enable_local_apic;
-	if (!dont_enable_local_apic) {
-		dont_enable_local_apic = 1;
+	extern int enable_local_apic_flag;
+	if (!enable_local_apic_flag) {
+		enable_local_apic_flag = 2;
 		printk(KERN_WARNING "%s with broken BIOS detected. "
 		       "Refusing to enable the local APIC.\n",
 		       d->ident);

@@ -333,9 +333,9 @@
 static int __init apm_kills_local_apic(struct dmi_blacklist *d)
 {
 #ifdef CONFIG_X86_LOCAL_APIC
-	extern int dont_enable_local_apic;
-	if (apm_info.bios.version && !dont_enable_local_apic) {
-		dont_enable_local_apic = 1;
+	extern int enable_local_apic_flag;
+	if (apm_info.bios.version && !enable_local_apic_flag) {
+		enable_local_apic_flag = 2;
 		printk(KERN_WARNING "%s with broken BIOS detected. "
 		       "Refusing to enable the local APIC.\n",
 		       d->ident);

function.linuxpower.ca

Did you get any response as to whether it was going to be accepted into the kernel or not?

The unfortunate thing is that even this sort of fix will not help my situation. The reason being (which I only discovered by accident when I set "dont_enable_local_apic = 1" rather than "dont_use_local_apic_timer" and it didn't correct the booting problem) is that it seems that even if the local apic is set disabled by setting dont_enable_local_apic = 1 in arch/i386/kernel/apic.c, setup_APIC_clocks() is still called.

So the jist is that using the local apic timer feature is not dependent on using the local apic, as per the dont_enable_local_apic and dont_use_local_apic_timer flags in arch/i386/kernel/apic.c. Maybe this is wrong, I dunno unfortunately.

I don't know anything about this APIC stuff so I don't know if that is correct or not, but it is what happens.

Thanx for the input though, much appreciated,

b.

Considering the dalay, i'll resend and give it another go, but generally it means it's not going anywhere.

How did you determine that? printks? Was this with my patch applied? I originally did this patch for the exact same problem (buggy local APIC implimentation).

(much snipped)
Linux version 2.5.70-mm1 ([email protected]) (gcc version Kernel command line: nolapic nmi_watchdog=2 ro root=/dev/hda1 profile=2 debug console=tty0 cons0
kernel profiling enabled
Initializing CPU#0
CPU0: Intel Celeron (Mendocino) stepping 05
per-CPU timeslice cutoff: 365.65 usecs.
task migration cache decay timeout: 1 msecs.
SMP motherboard not detected.
Local APIC not detected. Using dummy APIC emulation. Starting migration thread for cpu 0

You're welcome,
Zwane

function.linuxpower.ca

That sucks.

Well, originally it was a mistake in setting the wrong flag to 1 (dont_enable_local_apic rather than dont_use_local_apic_timer) and finding that it did not cure the problem, when I went back to my change to figure out why, I noticed that I had set the wrong flag.

I did not try your patch since I did pretty much the same thing with my "mistake" described above.

I know nothing about the APIC stuff, but it seems strange that even though it's disabled (dont_enable_local_apic = 1) setup_APIC_clocks() is still called. Maybe the latter is not dependent on the former, so there should not be a dependence on dont_enable_local_apic == 0 for setup_APIC_clocks() to still be used.

Yes, it's further along in the boot where my system runs into trouble, right here:

Using local APIC timer interrupts.
calibrating APIC timer ...
..... CPU clock speed is 1658.7651 MHz.
..... host bus clock speed is 0.0000 MHz.
cpu: 0, clocks: 0, slice: 0

I will take another stab at all of this tomorrow to double-verify what I am saying here regarding the use of local APIC timer interrupts even if the local apic usage flag is set to disable (dont_enable_local_apic = 1).

b.

Just to confirm now, I have modified Zwane's patch add another kernel arg, [no]locapictimer, which deals with dont_use_local_apic_timer in the same way his patch deals with the dont_enable_local_apic flag, and indeed, a kernel booted with "nolapic" does hang in the APIC timer calibration however a kernel booted with "nolocapictimer" does not.

Is it really valid to go and try to calibrate the APIC timer if it was disabled by the user, or even DMI?

b.

bah, the early boot hacks are just ugly, we do detect_init_APIC early in traps code and then just blindly go frobbing the APIC anyway.

The following patch will honour the dont_enable_local_apic flag in the necessary places. If it works for you i'll forward it.

Index: linux-2.5/arch/i386/kernel/apic.c
===================================================================
RCS file: /home/cvs/linux-2.5/arch/i386/kernel/apic.c,v retrieving revision 1.50

diff -u -p -B -r1.50 apic.c
@@ -665,6 +665,7 @@ static int __init detect_init_APIC (void
 	return 0;
 
 no_apic:
+	dont_enable_local_apic = 1;
 	printk("No local APIC present or hardware disabled\n");
 	return -1;
 }

@@ -1127,6 +1128,9 @@ asmlinkage void smp_error_interrupt(void
  */
 int __init APIC_init_uniprocessor (void)
 {
+	if (dont_enable_local_apic)
+		return -1;
+
 	if (!smp_found_config && !cpu_has_apic)
 		return -1;
 

function.linuxpower.ca

Yesserrie, it works, assuming dont_enable_local_apic gets set to "1" somewhere, somehow. In the case of broken hardware (or a broken emulation of it, like VMware 2.0.4) however, a command line arg (such as the nolapic patch you posted previously) is still needed to disable the local apic in the case where there is no DMI information that can be used to add the broken system to the blacklist.

A combination of your previous nolapic patch and this one you just posted would do the trick indeed.

Thanx much!

b.

How about clearing cpu_has_apic and smp_found_config instead? As I
understand the problem, the local APIC is useless so the kernel shouldn't pretend it's present. And the MP-table is useless without a local APIC.

+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +

I agree, but there are already the appropriate checks for wether there is an APIC enabled that should suffice.

Zwane

function.linuxpower.ca

You may have a valid SMP table and discrete local APICs (i82489DX) which
are not reported in CPU capability bits. The "nolocalapic" option should handle them, too. Otherwise it would be a surprising inconsistency.

+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +

Good point, out of interest, have you come across broken system like that? Regardless i'll update the patch.

Thanks,
Zwane

function.linuxpower.ca

So far I've met three users of i82489DX-based systems, two of whom helped
me making them work with 2.4. So at least at the beginning of 2.4.x they used to work; hopefully nothing has got broken since then (I try to monitor changes, but without real hardware to do testing something might have slipped in unnoticed). I don't know if these people have kept upgrading their kernels nor whether they still use the systems. There were no bug reports, either, which basically may mean anything.

Why do you consider the systems broken?

Great!

+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +

Not necessarily broken, just no reporting of APIC capability. Not that i should expect better from Intel (c.f. HT bit, SEP on PPro etc)

How about we only clear smp_found_config when forced.

Index: linux-2.5/arch/i386/kernel/apic.c
===================================================================
RCS file: /home/cvs/linux-2.5/arch/i386/kernel/apic.c,v retrieving revision 1.42

diff -u -p -B -r1.42 apic.c
@@ -602,6 +602,15 @@ static void apic_pm_activate(void) { }
  */
 int dont_enable_local_apic __initdata = 0;
 
+static int __init nolapic_setup(char *str)
+{
+	dont_enable_local_apic = 1;
+	smp_found_config = 0;
+	return 1;
+}
+
+__setup("nolapic", nolapic_setup);
+
 static int __init detect_init_APIC (void)
 {
 	u32 h, l, features;

@@ -609,7 +618,7 @@ static int __init detect_init_APIC (void
 
 	/* Disabled by DMI scan or kernel option? */
 	if (dont_enable_local_apic)
-		return -1;
+		goto no_apic;
 
 	/* Workaround for us being called before identify_cpu(). */
 	get_cpu_vendor(&boot_cpu_data);

@@ -665,6 +674,7 @@ static int __init detect_init_APIC (void
 	return 0;
 
 no_apic:
+	clear_bit(X86_FEATURE_APIC, boot_cpu_data.x86_capability);
 	printk("No local APIC present or hardware disabled\n");
 	return -1;
 }

Index: linux-2.5/Documentation/kernel-parameters.txt
===================================================================
RCS file: /home/cvs/linux-2.5/Documentation/kernel-parameters.txt,v retrieving revision 1.24

diff -u -p -B -r1.24 kernel-parameters.txt
@@ -625,6 +625,9 @@ running once the system is up.
 
 	nointroute	[IA-64]
 
+	nolapic		[IA-32, APIC]
+			Disable Local APIC.
+
 	nomce		[IA-32] Machine Check Exception
 
 	noresume	[SWSUSP] Disables resume and restore original swap space.

Hmm, how could e.g. an i486 report it? Remember, the i82489DX is a
discrete APIC implementation, i.e. it's a chip external to the CPU. The i82489DX is actually a complete APIC implementation, including both a local and an I/O unit in a single chip. And it's also superior to integrated APIC implementations -- it can address up to 255 units in the physical destination mode and up to 32 ones in the logical one.

These APICs were used for i486 systems as well as for Pentium ones meant
to support more than two CPUs (although the Pentium integrated local APICs can probably support more than two such units in a system, there was no suitable I/O unit available; the i82093AA chip was introduced later and the i82489DX is hardware-incompatible to later implementations, e.g. using a five-wire inter-APIC bus).

Looks OK. Probably a message could be added to report handling of the
local APIC got disabled.

+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +

Nope, could you add me to CC in future messages? It's easier for me to spot messages which need replying.

Thanks,
Zwane

function.linuxpower.ca