2.6.0-test6: APM unable to suspend (the 2.6.0-test2 saga continues)

I've just re-tested this with 2.6.0-test6, and the problem persists. Red Hat 2.4.20-20.7smp works fine.

I'm booting both 2.4 and 2.6 kernels with: init=/bin/sh apm=debug without starting any daemons, or having any PCMCIA socket drivers loaded. apm debug messages appear to be identical between 2.4 and 2.6 kernels.

If that is so, how do we go about debugging this type of problem? I've tried the obvious of disabling virtually all drivers, and this doesn't appear to make any difference what so ever. I'm guessing that something in the core i386 architecture code is causing the problem.

Russell King ([email protected])	http://www.arm.linux.org.uk/personal/
      Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
      maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                      2.6 Serial core

Alan Cox has shed some light on this problem. He mentions that the x86 GDT/LDT stuff changed around 2.5.30, which is the time when others have also reported that their APM has stopped working. I've not confirmed whether that is the case for me as well, but it seems to be highly likely.

I also asked Alan if there's the possibility of backing this out or making it configurable, but the answer seems to be a most definite no. However, maybe Ingo can say otherwise?

This effectively means that people with laptops which do not work with 2.6 APM nor ACPI can expect their machines to be stuck with 2.4 for the future, unless someone with the necessary knowledge sees this problem as important enough to solve.

It also means that I'm never going to be able to test PCMCIA suspend under real conditions with the hardware I have available to me. This in turn brings up the question of passing PCMCIA off to someone else to maintain and test for 2.6. However, I remain willing to continue the development, albiet limited by the working hardware I have available to me.

Do we have any willing volunteers?

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

A quick follow-up from a discussion I've just had with Andi Kleen... Note that I know nothing about the x86 internals of LDT/GDT/APM stuff, so I'm clutching at straws here...

Would it be possible to switch LDT/GDT to whatever the APM BIOS expects just before calling the APM BIOS to suspend/hibernate, and restore them to whatever Linux requires after the APM BIOS returns from resume?

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Possible, yes. But it would help a lot to know what's wrong with the current segments - we did leave most of them with exactly the same layout as before, and I thought we explicitly left the ones that APM cares about that way..

Linus

With thanks to Arjan, I think we've proven that this is not the change which is causing the problem by testing various Red Hat and Fedora 2.4 kernels on the machine (which have various 2.6 NPTL backports.)

I'm now back to being completely out of my depth on this issue; a 2.4 kernel booted through to init=/bin/bash suspends, but a 2.6 kernel with the same hardware support booted to the same point refuses to suspend via APM.

I think I'm going to have to resort to a binary search of the 2.5 kernel series to find out exactly what broke and when.

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Ok, the binary search led me to a changeset between 2.5.25 and 2.5.26 kernels. Further investigation led me to i8042.c, specifically this:

static void __init i8042_start_polling(void)

{
        i8042_ctr &= ~I8042_CTR_KBDDIS;
        if (i8042_aux_values.exists)
                i8042_ctr &= ~I8042_CTR_AUXDIS;

        if (i8042_command(&i8042_ctr, I8042_CMD_CTL_WCTR)) {
                printk(KERN_WARNING "i8042.c: Can't write CTR while starting polling.\n");
                return;
        }

// i8042_timer.function = i8042_timer_func;
// mod_timer(&i8042_timer, jiffies + I8042_POLL_PERIOD);
}

With the function as above, the observed behaviour is as follows:

- don't call i8042_start_polling at all - suspend works
- call i8042_start_polling - suspend fails

It seems that my BIOS is taking exception to CTR value we're writing.

My next step will be to try this with 2.6.0 and see whether this is the only issue affecting APM suspend. In the mean time, does Vojtech have any hints?

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Ok, further info with 2.6.0:

- disabling i8042.c completely by adding return 0; to the start of i8042_init() allows the suspend hotkey to work, and it works multiple times! The hibernate hotkey also works, but once the laptop has resumed from hibernate, it's no longer possible to suspend it.

- i8042_noaux=1 - this doesn't seem to make any difference, although this does appear to leave the CTR set as 0x65, which appears to be the BIOS-set value.

There seems to be more going on than just the CTR here - it seems that this APM BIOS has some dislike for our i8042 driver.

Here's the boot messages under (a failing) 2.6.0:

Linux version 2.6.0 ([email protected]) (gcc version 3.3.2 20031022 (Red Hat Linux 3.3.2-1)) #2 Sun Dec 28 21:16:27 GMT 2003 BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 0000000003fd0000 (usable)
BIOS-e820: 0000000003fd0000 - 0000000003fdf000 (ACPI data)
BIOS-e820: 0000000003fdf000 - 0000000003fe0000 (ACPI NVS)
BIOS-e820: 00000000fffe0000 - 0000000100000000 (reserved)
63MB LOWMEM available.
On node 0 totalpages: 16336
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 12240 pages, LIFO batch:2
HighMem zone: 0 pages, LIFO batch:1
DMI 2.0 present.
IBM machine detected. Enabling interrupts during APM calls.
IBM machine detected. Disabling SMBus accesses.
Building zonelist for node : 0
Kernel command line: auto BOOT_IMAGE=linux-2.6 ro root=302 BOOT_FILE=/boot/vmlinuz-2.6.0 pci=usepirqmask Initializing CPU#0
PID hash table entries: 256 (order 8: 2048 bytes)
Detected 232.122 MHz processor.
Console: colour VGA+ 80x25
Memory: 61716k/65344k available (1607k kernel code, 3172k reserved, 675k data, 96k init, 0k highmem)
Calibrating delay loop... 458.75 BogoMIPS
Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
Intel Pentium with F0 0F bug - workaround enabled.
CPU: Intel Mobile Pentium MMX stepping 01
Checking 'hlt' instruction... OK.
POSIX conformance testing by UNIFIX
NET: Registered protocol family 16
PCI: PCI BIOS revision 2.10 entry at 0xfd880, last bus=0
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
PCI: Probing PCI hardware
PCI: Probing PCI hardware (bus 00)
PCI: Using IRQ router PIIX/ICH [8086/7110] at 0000:00:06.0
PCI: Found IRQ 11 for device 0000:00:02.0
PCI: Sharing IRQ 11 with 0000:00:03.0
PCI: Found IRQ 11 for device 0000:00:02.1
apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16ac)
Installing knfsd (copyright (C) 1996 [email protected]).
udf: registering filesystem
Limiting direct PCI/PCI transfers.
pty: 256 Unix98 ptys configured
[drm] Initialized radeon 1.9.0 20020828 on minor 0
Using anticipatory io scheduler
Floppy drive(s): fd0 is 1.44M
FDC 0 is a National Semiconductor PC87306
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PIIX4: IDE controller at PCI slot 0000:00:06.1
PIIX4: chipset revision 1
PIIX4: not 100%% native mode: will probe irqs later

    ide0: BM-DMA at 0xfcf0-0xfcf7, BIOS settings: hda:DMA, hdb:DMA

hda: IBM-DTCA-24090, ATA DISK drive
hdb: SANYO CRD-S372B, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hda: max request size: 128KiB
hda: 8007552 sectors (4099 MB) w/468KiB Cache, CHS=7944/16/63, UDMA(33)
hda: hda1 hda2 hda4 < hda5 hda6 hda7 hda8 hda9 >
end_request: I/O error, dev hdb, sector 0
hdb: ATAPI 24X CD-ROM drive, 256kB Cache
Uniform CD-ROM driver Revision: 3.12
mice: PS/2 mouse device common for all mice
input: PS/2 Generic Mouse on isa0060/serio1
serio: i8042 AUX port at 0x60,0x64 irq 12
input: AT Translated Set 2 keyboard on isa0060/serio0
serio: i8042 KBD port at 0x60,0x64 irq 1
NET: Registered protocol family 2
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 4096 bind 8192)
NET: Registered protocol family 1
NET: Registered protocol family 17
BIOS EDD facility v0.10 2003-Oct-11, 1 devices found
Please report your BIOS at http://domsch.com/linux/edd30/results.html
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 96k freed

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Doesn't that leave the kbd mask the same? In particular, it still sets the "disable" bit, aka I8042_CTR_KBDDIS later on..

What happens if you just define I8042_CTR_KBDDIS to zero?

Linus

Seems to. With noaux unset, CTR is set to 0x47.

That still causes suspend to fail. I've separately tested I8042_CTR_KBDINT set to zero as well, and that still causes failure.

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

I just tried this change to i8042.c, and suspend magically started working.

@@ -814,8 +815,8 @@

        i8042_port_register(&i8042_kbd_values, &i8042_kbd_port);
  
        init_timer(&i8042_timer);
-       i8042_timer.function = i8042_timer_func;
-       mod_timer(&i8042_timer, jiffies + I8042_POLL_PERIOD);
+//     i8042_timer.function = i8042_timer_func;
+//     mod_timer(&i8042_timer, jiffies + I8042_POLL_PERIOD);

        register_reboot_notifier(&i8042_notifier);

So it looks like i8042 could do with hooking some power management to disable this timer before suspend and resume it afterwards.

Vojtech?

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Agreed. There should already be some in -mm kernels, and I'll make sure the timer is deleted before suspend. Thanks for finding this.

Vojtech Pavlik
SuSE Labs, SuSE CR

What about something like the patch below? (Now, I don't suspend my notebook so it has not been tested, just compiled.)

Dmitry

===================================================================

[email protected], 2003-12-30 20:37:33-05:00, [email protected]
Input: Add suspend methods to restore original controller state
on suspend as some BIOS don't like the state we leave it in
Also do not require extra i8042. prefix on module parameters if i8042 is compiled into the kernel

 i8042.c |   82 ++++++++++++++++++++++++++++++++++++++++++++++------------------
 1 files changed, 60 insertions(+), 22 deletions(-)

===================================================================

diff -Nru a/drivers/input/serio/i8042.c b/drivers/input/serio/i8042.c
@@ -28,6 +28,12 @@
 MODULE_DESCRIPTION("i8042 keyboard and mouse controller driver");
 MODULE_LICENSE("GPL");
 
+/*
+ * Do not add extra 'i8042.' prefix to all parameters if compiled into the kernel
+ */
+#undef MODULE_PARAM_PREFIX
+#define MODULE_PARAM_PREFIX /* empty */
+
 static unsigned int i8042_noaux;
 module_param(i8042_noaux, bool, 0);
 

@@ -746,6 +752,29 @@
 
 
 /*
+ * Reset the controller.
+ */
+void i8042_controller_reset(void)
+{
+	if (i8042_reset) {
+		unsigned char param;
+
+		if (i8042_command(&param, I8042_CMD_CTL_TEST))
+			printk(KERN_ERR "i8042.c: i8042 controller reset timeout.\n");
+	}
+
+/*
+ * Restore the original control register setting.
+ */
+
+	i8042_ctr = i8042_initial_ctr;
+
+	if (i8042_command(&i8042_ctr, I8042_CMD_CTL_WCTR))
+		printk(KERN_WARNING "i8042.c: Can't restore CTR.\n");
+}
+
+
+/*
  * Here we try to reset everything back to a state in which the BIOS will be
  * able to talk to the hardware when rebooting.
  */

@@ -770,26 +799,20 @@
 		if (i8042_mux_values[i].exists)
 			serio_cleanup(i8042_mux_port + i);
 
-/*
- * Reset the controller.
- */
-
-	if (i8042_reset) {
-		unsigned char param;
+	i8042_controller_reset();
+}
 
-		if (i8042_command(&param, I8042_CMD_CTL_TEST))
-			printk(KERN_ERR "i8042.c: i8042 controller reset timeout.\n");
-	}
 
 /*
- * Restore the original control register setting.
+ * Here we try to restore to the original BIOS settings
  */
 
-	i8042_ctr = i8042_initial_ctr;
-
-	if (i8042_command(&i8042_ctr, I8042_CMD_CTL_WCTR))
-		printk(KERN_WARNING "i8042.c: Can't restore CTR.\n");
+static int i8042_controller_suspend(void)
+{
+	del_timer_sync(&i8042_timer);
+	i8042_controller_reset();
 
+	return 0;
 }
 
 

@@ -809,7 +832,7 @@
 	if (i8042_mux_present)
 		if (i8042_enable_mux_mode(&i8042_aux_values, NULL) ||
 		    i8042_enable_mux_ports(&i8042_aux_values)) {
-			printk(KERN_WARNING "i8042: failed to resume active multiplexor, mouse won't wotk.\n");
+			printk(KERN_WARNING "i8042: failed to resume active multiplexor, mouse won't work.\n");
 		}
 
 /*

@@ -825,6 +848,10 @@
 	for (i = 0; i < 4; i++)
 		if (i8042_mux_values[i].exists && i8042_activate_port(i8042_mux_port + i) == 0)
 			serio_reconnect(i8042_mux_port + i);
+/*
+ * Restart timer (for autorepeats)
+ */ 
+	mod_timer(&i8042_timer, jiffies + I8042_POLL_PERIOD);
 
 	return 0;
 }

@@ -851,16 +878,22 @@
 };
 
 /*
- * Resume handler for the new PM scheme (driver model)
+ * Suspend/resume handlers for the new PM scheme (driver model)
  */
+static int i8042_suspend(struct sys_device *dev, u32 state)
+{
+	return i8042_controller_suspend();
+}
+
 static int i8042_resume(struct sys_device *dev)
 {
 	return i8042_controller_resume();
 }
 
 static struct sysdev_class kbc_sysclass = {
-       set_kset_name("i8042"),
-       .resume = i8042_resume,
+	set_kset_name("i8042"),
+	.suspend = i8042_suspend,
+	.resume = i8042_resume,
 };
 
 static struct sys_device device_i8042 = {

@@ -869,12 +902,17 @@
 };
 
 /*
- * Resume handler for the old PM scheme (APM)
+ * Suspend/resume handler for the old PM scheme (APM)
  */
 static int i8042_pm_callback(struct pm_dev *dev, pm_request_t request, void *dummy)
 {
-	if (request == PM_RESUME)
-		return i8042_controller_resume();
+	switch (request) {
+		case PM_SUSPEND:
+			return i8042_controller_suspend();
+
+		case PM_RESUME:
+			return i8042_controller_resume();
+	}
 
 	return 0;
 }

@@ -955,7 +993,7 @@
 		sysdev_class_unregister(&kbc_sysclass);
 	}
 
-	del_timer(&i8042_timer);
+	del_timer_sync(&i8042_timer);
 
 	i8042_controller_cleanup();
 	

I like it, see comments.

Vojtech Pavlik
SuSE Labs, SuSE CR

Yeah, unless there's a good reason, it's nicer if there's symmetry so people can guess how to pass params.

Thanks,
Rusty.

   there are those who do and those who hang on and you don't see too
   many doers quoting their contemporaries.  -- Larry McVoy

Hi!

You might want to use swap suspend instead of hibernate... at least we have full control over that one.
Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]