Help port swsusp to ppc.

Hello guys:

I'm try to porting Software Suspend to PowerPC. But I'm not family with PowerPC assemble languae. So need someone to help me write the save process state function in assemble language.

1: Download i386 suspend patch for 2.6.1 from swsusp.sf.net
you need two patch software-suspend-core-2.0-rc4-whole.bz2 software-suspend-linux-2.6.1-rev1-whole.bz2
2: first you need apply linux-2.6.1 to 2.6.1 clean kernel
then apply core-2.0-rc4 to current patched kernel
3: then apply ppc-swsusp.patch to patched kernel
4: do make menuconfig
CONFIG_SOFTWARE_SUSPEND2=y CONFIG_SOFTWARE_SUSPEND_DEBUG=y CONFIG_SOFTWARE_SUSPEND_GZIP_COMPRESSION=y CONFIG_SOFTWARE_SUSPEND_KEEP_IMAGE=y CONFIG_SOFTWARE_SUSPEND_LZF_COMPRESSION=y CONFIG_SOFTWARE_SUSPEND_SWAPWRITER=y
5: test with swsusp enable kernel.
in yaboot prompt:
/vmlinux.swsusp root=/dev/hda13 resume2=swap:/dev/hda10 init=/bin/sh

/dev/hda13 is the root device name
/dev/hda10 is swap device name

6: run suspend script to do software suspend, now the machine will power
off.

7: power on machin, in yaboot prompt:
/vmlinux.swsusp root=/dev/hda13 resume2=swap:/dev/hda10 init=/bin/sh

But for now the save and restore processor state is not finish, the resume will oops.

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

Index: include/asm-ppc/suspend.h
===================================================================
@@ -0,0 +1,6 @@
+#ifndef _PPC_SUSPEND_H
+#define _PPC_SUSPEND_H
+
+/* Nothing */
+
+#endif

Index: arch/ppc/Kconfig
===================================================================
@@ -193,6 +193,8 @@
 
 	  If in doubt, say Y here.
 
+source kernel/power/Kconfig
+
 source arch/ppc/platforms/4xx/Kconfig
 
 config PPC64BRIDGE

Index: arch/ppc/kernel/Makefile
===================================================================
@@ -34,3 +34,5 @@
 obj-$(CONFIG_8xx)		+= softemu8xx.o
 endif
 
+obj-$(CONFIG_SOFTWARE_SUSPEND2) += swsusp2-asm.o
+obj-$(CONFIG_SOFTWARE_SUSPEND2) += ppc_reg.o

Index: arch/ppc/kernel/vmlinux.lds.S
===================================================================
@@ -72,6 +72,12 @@
     CONSTRUCTORS
   }
 
+  . = ALIGN(4096);
+  __nosave_begin = .;
+  .data_nosave : { *(.data.nosave) }
+  . = ALIGN(4096);
+  __nosave_end = .;
+
   . = ALIGN(32);
   .data.cacheline_aligned : { *(.data.cacheline_aligned) }
 

Index: kernel/power/swsusp2.c
===================================================================
@@ -54,7 +54,9 @@
 
 #include <linux/suspend-common.h>
 #include <linux/suspend-version-specific.h>
+#ifdef  CONFIG_X86
 #include <asm/i387.h> /* for kernel_fpu_end */
+#endif
 #ifdef	CONFIG_KDB
 #include <linux/kdb.h>
 #endif	/* CONFIG_KDB */

@@ -327,7 +329,9 @@
 	preempt_enable();
 #endif
 
+#ifdef CONFIG_X86
 	kernel_fpu_end();
+#endif
 
 	PRINTPREEMPTCOUNT("Prior to resuming drivers.");
 

@@ -486,10 +490,10 @@
 #endif
 
 	PRINTPREEMPTCOUNT("In resume_2 after copy back.");
-
+#ifdef CONFIG_X86
 	kernel_fpu_end();
-
 	PRINTPREEMPTCOUNT("In resume_2 after fpu end.");
+#endif
 	
 #ifdef CONFIG_PREEMPT
 	preempt_enable();

Hi.

I'd love to be able to help, but unfortunately the old assembly I know is good old 6502 :> Hopefully someone else will pipe up and get you going.

Once you do get it going, I'll be happy to help keep the other parts updated, so far as I'm able.

Regards,

Nigel

My work on Software Suspend is graciously brought to you by
LinuxFund.org.

Hi !

Some comments...

Hi.

I can answer a couple of the questions:

It should contain the .S equivalent to the swsusp2.c file. It would be best if swsusp2.c could simply be compiled, but it appears that it can't at the moment on x86 (I need to learn x86 assembly so I can understand why).

That idea is to have a section that doesn't get replaced when we copy the original kernel back. Thus, small amounts of data that suspend uses or stores can be given the __nosave attribute. An example is the cpu frequency value, which should match the boot kernel, not the value at suspend time.

Regards,

Nigel

My work on Software Suspend is graciously brought to you by
LinuxFund.org.

I see no reason why this would be needed on ppc, only the last step, that is the actual CPU state save, should matter.

That's very hairy... You basically assume the boot kernel and the restore kernel are completely identical, which isn't something I would do. I didn't have time to dive into it, but I do/did intend to implement swsusp on ppc and I would eventually resume the whole environement straight from the bootloader without kernel help.

If you want to pass some infos between the "loader" kernel and the "loaded" one, I strongly suggest you define some well specified interface for doing so that is immune to kernel versions.

Also, I haven't looked in details, but when switching to the "new" kernel from the "loader" (boot) one, do you shut down all devices properly ? This switch could actually be fairly similar to a kexec pass in this regard.

Ben.

Hi.

Besides saving the CPU state, the code copies the original kernel back. It sort of defeats the purpose to remove that code :>

Well, the whole things is pretty hairy :> But I don't think there's anything wrong with assuming the boot and restored kernels are identical. After all, we're calling it suspending and resuming, not kexec. It does sound nice to do it all from the bootloader without kernel help.

It is a well defined interface: a section of memory marked nosave, with variables given the matching attribute. Not my idea, by the way. If you have a problem, you should be taking it up with Pavel or Linus. We should also note that the interface can't be too well defined - there has to be room for development over time.

Yes. we device_suspend. Regarding the similarities with kexec, I fully agree. In fact, there is also LKCD to think off. There should be a synergy here.

Regards,

Nigel

My work on Software Suspend is graciously brought to you by
LinuxFund.org.

Ok, you mean copying the memory pages back down ? That should be done with hand-made assembly or C code located specifically elsewhere then. I do not want to see any kind of this ugly C-generated assembly in arch/ppc.

I think that's a wrong assumption. I see personally _NO_ reason to assume the boot and restored kernel are the same. They absolutely don't need to do at all.

Still... That makes assumptions about how it's located and organised that plain wrong (c). Please get rid of that, at least I won't let a PPC version do that. Depending on how much time I can spare, I'll try to work on that PPC port later this week. If you need something like clock calibration data down to the loaded kernel, then either re-calculate it in the wakeup code path, or pass it via some _sane_ interface (at _least_ a versioned data structure, better, a tagged list of values like some bootloaders do).

device_suspend is, imho, hairy too. We have some semantics that need cleanup here, I'll have to talk to Patrick about them. Putting devices to an idle state is what you need and what kexec need, and doesn't mean putting them to _sleep_. Or maybe we could pass a specific state to device_suspend for that. For example, it makes little sense to spin down the hard disk at this stage, or to power off the video chip...

Ben.

Hi!

FYI, this is that "ugly C-generated assembly" we are talking about. I do not think it is so bad.

.text

/* Originally gcc generated, modified by hand */

#include <linux/linkage.h>
#include <asm/segment.h>
#include <asm/page.h>

.text

ENTRY(do_magic)
pushl %ebx
cmpl $0,8(%esp)
jne .L1450
call do_magic_suspend_1
call save_processor_state

movl %esp, saved_context_esp
movl %eax, saved_context_eax
movl %ebx, saved_context_ebx
movl %ecx, saved_context_ecx
movl %edx, saved_context_edx
movl %ebp, saved_context_ebp
movl %esi, saved_context_esi
movl %edi, saved_context_edi
pushfl ; popl saved_context_eflags

call do_magic_suspend_2
jmp .L1449
.p2align 4,,7
.L1450:

	movl $swapper_pg_dir-__PAGE_OFFSET,%ecx
	movl %ecx,%cr3

	call do_magic_resume_1
	movl $0,loop
	cmpl $0,nr_copy_pages
	je .L1453
	.p2align 4,,7
.L1455:
	movl $0,loop2
	.p2align 4,,7
.L1459:
	movl pagedir_nosave,%ecx
	movl loop,%eax
	movl loop2,%edx
	sall $4,%eax
	movl 4(%ecx,%eax),%ebx
	movl (%ecx,%eax),%eax
	movb (%edx,%eax),%al
	movb %al,(%edx,%ebx)
	movl %cr3, %eax;              
	movl %eax, %cr3;  # flush TLB 

	movl loop2,%eax
	leal 1(%eax),%edx
	movl %edx,loop2
	movl %edx,%eax
	cmpl $4095,%eax
	jbe .L1459
	movl loop,%eax
	leal 1(%eax),%edx
	movl %edx,loop
	movl %edx,%eax
	cmpl nr_copy_pages,%eax
	jb .L1455
	.p2align 4,,7

.L1453:
movl $__USER_DS,%eax

movw %ax, %ds
movw %ax, %es
movl saved_context_esp, %esp
movl saved_context_ebp, %ebp
movl saved_context_eax, %eax
movl saved_context_ebx, %ebx
movl saved_context_ecx, %ecx
movl saved_context_edx, %edx
movl saved_context_esi, %esi
movl saved_context_edi, %edi
call restore_processor_state
pushl saved_context_eflags ; popfl
call do_magic_resume_2
.L1449:
popl %ebx
ret

.section .data.nosave
loop:

       .quad 0
loop2:
       .quad 0
       .previous

FYI, there are exactly 6 variables in "nosave" section. Two loop variables you can see in above code, one spinlock, number of pages to save, pointer to directory of pages to be copied, and its length.

I could probably move spinlock and length of pgdir out of there...

Okay, I can agree that putting them into sleep is not ideal [it puzzles users, at least]. But it is quite simple and should work okay. I do not want another round of device_suspend changes in 2.6.X... [Well, perhaps if it was done in right&compatible way (tm), it would be acceptable...]
Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

The x86 version has been cleaned up and isn't _that_ bad, though it could definitely use some comments and I don't like the "Lxxx" labels, I'd rather either use number with the "nb" or "nf" GAS constructs or use real words labels. Looking at it though, I fail to see the need to get it generated by gcc in the first place :)

The PPC version that was proposed is horrible.

That's not too much, you could probably afford having a one page header to the suspend image with those informations and the page copy loop (provided by the suspended kernel so you don't have _any_ compatibility issue and can even do it from the bootloader one day...)

We already have the shutdown() callback or we can simply use device_suspend() with a D1 parameter instead of D3 or D4. That's what I'd do...

Looking at swsusp code in current 2.6, when do you do that pass ? On the shutdown pass, you call devices_suspend(4); which is fine. But I don't see where you call devices_suspend(X) on the resume path. IMHO, that should be done from the boot kernel after loading the suspend image and before starting the resume process. Your mdelay(1000) for waiting for DMA to settle down is PLAIN WRONG imho, even dangerous. (And typically, an USB controller will still be hapilly be DMA'ing all over your memory). That's what I call idling devices at this point, and that's where I'd call devices_suspend(1), and we should do that for kexec too.

Ben.

Hi.

The 2.4 version works from C, even the SMP version I've prepared. Something in 2.6 is different, and I don't understand what/why (Pavel might be able to explain). If I understood that, I could get SMP support for 2.6 going too.

I think he wanted help with cleaning it up.

Well, we do have a header at the moment with that info, (but not containing the page copy loop :>)

Hmm. Perhaps it's just my version that does it? I couldn't find it either.

Regards,

Nigel

My work on Software Suspend is graciously brought to you by
LinuxFund.org.

Hi!

Well, I had C version that was working (and it was slightly easier to code it in C), and really did not want to introduce errors at that point.

Sorry, I missed that one.

Unfortunately the copy loop has quite complicated interface. It expects tables already loaded into memory, at non-conflicting addresses etc.

We need to do something with those 1,3s and 4s. IIRC there's no such thing as D4 (or is it expected to be == D3cold?). This needs to be documented somewhere.

My fault, the patch is in my tree only. (Its here for quite a long time. I synced it to Patrick and then he disappeared.).

Yes, that's right solution. Here's the patch.

Index: linux/kernel/power/swsusp.c
===================================================================
@@ -488,33 +492,6 @@
 	printk("|\n");
 }
 
-/* Make disk drivers accept operations, again */
-static void drivers_unsuspend(void)
-{
-	device_resume();
-}
-
-/* Called from process context */
-static int drivers_suspend(void)
-{
-	return device_suspend(4);
-}
-
-#define RESUME_PHASE1 1 /* Called from interrupts disabled */
-#define RESUME_PHASE2 2 /* Called with interrupts enabled */
-#define RESUME_ALL_PHASES (RESUME_PHASE1 | RESUME_PHASE2)
-static void drivers_resume(int flags)
-{
-	if (flags & RESUME_PHASE1) {
-		device_resume();
-	}
-  	if (flags & RESUME_PHASE2) {
-#ifdef SUSPEND_CONSOLE
-		update_screen(fg_console);	/* Hmm, is this the problem? */
-#endif
-	}
-}
-
 static int suspend_prepare_image(void)
 {
 	struct sysinfo i;

@@ -569,7 +546,7 @@
 
 static void suspend_save_image(void)
 {
-	drivers_unsuspend();
+	device_resume();
 
 	lock_swapdevices();
 	write_suspend_image();

@@ -615,6 +592,7 @@
 	mb();
 	spin_lock_irq(&suspend_pagedir_lock);	/* Done to disable interrupts */ 
 
+	device_power_down(4);
 	PRINTK( "Waiting for DMAs to settle down...\n");
 	mdelay(1000);	/* We do not want some readahead with DMA to corrupt our memory, right?
 			   Do it with disabled interrupts for best effect. That way, if some

@@ -630,8 +608,10 @@
 
 	PRINTK( "Freeing prev allocated pagedir\n" );
 	free_suspend_pagedir((unsigned long) pagedir_save);
+	device_power_up();
 	spin_unlock_irq(&suspend_pagedir_lock);
-	drivers_resume(RESUME_ALL_PHASES);
+	device_resume();
+	update_screen(fg_console);	/* Hmm, is this the problem? */
 
 	PRINTK( "Fixing swap signatures... " );
 	mark_swapfiles(((swp_entry_t) {0}), MARK_SWAP_RESUME);

@@ -672,7 +652,9 @@
 {
 	int is_problem;
 	read_swapfiles();
+	device_power_down(4);
 	is_problem = suspend_prepare_image();
+	device_power_up();
 	spin_unlock_irq(&suspend_pagedir_lock);
 	if (!is_problem) {
 		kernel_fpu_end();	/* save_processor_state() does kernel_fpu_begin, and we need to revert it in order to pass in_atomic() checks */

@@ -716,7 +709,7 @@
 		blk_run_queues();
 
 		/* Save state of all device drivers, and stop them. */		   
-		if(drivers_suspend()==0)
+		if ((res = device_suspend(4))==0)
 			/* If stopping device drivers worked, we proceed basically into
 			 * suspend_save_image.
 			 *

@@ -1091,6 +1072,7 @@
 	printk( "resuming from %s\n", resume_file);
 	if (read_suspend_image(resume_file, 0))
 		goto read_failure;
+	device_suspend(4);
 	do_magic(1);
 	panic("This never returns");
 

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Attached file is current version of port swsusp to ppc, STILL can not works, Benjamin, gave me some comments.

I has add one files swsusp2-asm.S. The save/restore processor state base on pmac_sleep.S. The copybackup is copy from gcc generate assmeble.

Now the suspend has no problem, resume can not works, strange.

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

Index: linux-2.6.1-suspend/arch/ppc/Kconfig
===================================================================
@@ -193,6 +193,8 @@
 
 	  If in doubt, say Y here.
 
+source kernel/power/Kconfig
+
 source arch/ppc/platforms/4xx/Kconfig
 
 config PPC64BRIDGE

@@ -681,6 +683,7 @@
 
 config NR_CPUS
 	int "Maximum number of CPUs (2-32)"
+	range 2 32
 	depends on SMP
 	default "4"
 

Index: linux-2.6.1-suspend/arch/ppc/kernel/Makefile
===================================================================
@@ -34,3 +34,4 @@
 obj-$(CONFIG_8xx)		+= softemu8xx.o
 endif
 
+obj-$(CONFIG_SOFTWARE_SUSPEND2) += swsusp2-asm.o

Index: linux-2.6.1-suspend/arch/ppc/kernel/vmlinux.lds.S
===================================================================
@@ -72,6 +72,12 @@
     CONSTRUCTORS
   }
 
+  . = ALIGN(4096);
+  __nosave_begin = .;
+  .data_nosave : { *(.data.nosave) }
+  . = ALIGN(4096);
+  __nosave_end = .;
+
   . = ALIGN(32);
   .data.cacheline_aligned : { *(.data.cacheline_aligned) }
 

Index: linux-2.6.1-suspend/arch/ppc/kernel/signal.c
===================================================================
@@ -28,6 +28,7 @@
 #include <linux/elf.h>
 #include <linux/tty.h>
 #include <linux/binfmts.h>
+#include <linux/suspend.h>
 #include <asm/ucontext.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>

@@ -552,6 +553,11 @@
 	unsigned long frame, newsp;
 	int signr, ret;
 
+	if (current->flags & PF_FREEZE) {
+		refrigerator(PF_FREEZE);
+		return 0;
+	}
+
 	if (!oldset)
 		oldset = &current->blocked;
 

Index: linux-2.6.1-suspend/include/asm-ppc/suspend.h
===================================================================
@@ -0,0 +1,14 @@
+#ifndef _PPC_SUSPEND_H
+#define _PPC_SUSPEND_H
+
+static inline void flush_tlb_all(void)
+{
+	/* Flush all TLBs */
+	__asm__ __volatile__("lis 4, 0x1000");
+	__asm__ __volatile__("1: addic. 4,4,-0x1000");
+	__asm__ __volatile__("tlbie 4");
+	__asm__ __volatile__("blt 1b");
+	__asm__ __volatile__("sync");
+}
+
+#endif

I know nothing about the PPC. Is it a uniprocessor?

Regards,

Nigel

My work on Software Suspend is graciously brought to you by
LinuxFund.org.

There is a document abourt ppc, that can download from Moto, It help me to understand what is ppc, cool.

http://soulinfo.com/~hugang/tmp/MPCFPE_AD_R1.pdf

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

There is at least one reason I think your code cannot work: When resuming, you are basically blowing up the MMU hash table and kernel page tables when copying the pages. I'm hacking on an implementation of pmdisk at the moment that switches the MMU off during the page copy to avoid that problem. This isn't the best way though.

I'll keep you informed of my progress

Ben.

I has hacking on swsusp2 to ppc for whole day. Here is the update swsusp2-asm.S.

First I can true suspend is ok, the data and CPU context are full write to disk. Then in resume, the copyback function works.

How I know it?
I'm adding printk before do_swsusp2_lowleve finished, I has printf
the r1, That is the pointer to cpu context, it must right on resume, when resume it'll triger xmon, I d pm_sleep_storage, and check the pointer, I see that same things, So I true the copyback is ok, at least the cpu context memory has backed.

very tire day.

The current problem that is, In resume, It stop with all register (r0-r32) is zero. I don't known where I am. :)

I'm too. :)

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

Ok, I hammered that for a day and got pmdisk (patrick's version) suspending and resuming on a pismo G3 (with XFree etc.. running). Lots of rough edges still (via-pmu sleep need to be improved, ADB need porting to the new driver model to be properly suspended/resumed, a sysdev for RTC is needed too for time, the asm code should be fixed for G5, etc...)

I had to fix some issues in the core pmdisk code though. One big one is that lots of drivers expect suspend to disk to be state 4 while the current code used state 3 for that (and suspend to RAM to be state 3 btw). I hacked that in include/linux/suspend.h, but we shall probably just get rid of those stupid numbers and properly define each constant indstead.

We should also use a different state for the suspend calls done before saving the image, and the ones done before resuming the image, some driver may be optimized for these cases.

The patch is against my tree currently, and the arch/ppc/kernel/pmdisk.S file is appended as-is (not in patch form). I don't plan to release that right now, I may hack a bit on it in the "background" (I want to get HIGHMEM working some day). Feel free to improve, but then keep me informed please.

Ah, also: The "Freeing memory" phase takes forever. That should really be fixed.

Ben.

First the patch:

===== arch/ppc/Kconfig 1.60 vs edited =====

@@ -913,6 +913,8 @@
 
 source "drivers/zorro/Kconfig"
 
+source kernel/power/Kconfig
+
 endmenu
 
 menu "Bus options"

===== arch/ppc/kernel/Makefile 1.61 vs edited =====

@@ -15,6 +15,7 @@
 extra-$(CONFIG_8xx)		:= head_8xx.o
 extra-$(CONFIG_6xx)		+= idle_6xx.o
 extra-$(CONFIG_POWER4)		+= idle_power4.o
+
 extra-y				+= vmlinux.lds.s
 
 obj-y				:= entry.o traps.o irq.o idle.o time.o misc.o \

@@ -22,6 +23,7 @@
 					semaphore.o syscalls.o setup.o \
 					cputable.o ppc_htab.o
 obj-$(CONFIG_6xx)		+= l2cr.o cpu_setup_6xx.o
+obj-$(CONFIG_PM_DISK)		+= pmdisk.o
 obj-$(CONFIG_POWER4)		+= cpu_setup_power4.o
 obj-$(CONFIG_MODULES)		+= module.o ppc_ksyms.o
 obj-$(CONFIG_PCI)		+= pci.o

===== arch/ppc/kernel/signal.c 1.37 vs edited =====

@@ -28,6 +28,7 @@
 #include <linux/elf.h>
 #include <linux/tty.h>
 #include <linux/binfmts.h>
+#include <linux/suspend.h>
 #include <asm/ucontext.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>

@@ -563,6 +564,11 @@
 	struct k_sigaction *ka;
 	unsigned long frame, newsp;
 	int signr, ret;
+
+	if (current->flags & PF_FREEZE) {
+		refrigerator(0);
+		return 0;
+	}
 
 	if (!oldset)
 		oldset = &current->blocked;

===== arch/ppc/kernel/vmlinux.lds.S 1.39 vs edited =====

@@ -72,6 +72,12 @@
     CONSTRUCTORS
   }
 
+  . = ALIGN(4096);
+  __nosave_begin = .;
+  .data_nosave : { *(.data.nosave) }
+  . = ALIGN(4096);
+  __nosave_end = .;
+
   . = ALIGN(32);
   .data.cacheline_aligned : { *(.data.cacheline_aligned) }
 

===== arch/ppc/platforms/pmac_setup.c 1.48 vs edited =====

===== drivers/ide/ppc/pmac.c 1.47 vs edited =====

@@ -1366,7 +1366,7 @@
 	ide_hwif_t	*hwif = (ide_hwif_t *)dev_get_drvdata(&mdev->ofdev.dev);
 	int		rc = 0;
 
-	if (state != mdev->ofdev.dev.power_state && state >= 2) {
+	if (state != mdev->ofdev.dev.power_state && state >= 2 && state != 4) {
 		rc = pmac_ide_do_suspend(hwif);
 		if (rc == 0)
 			mdev->ofdev.dev.power_state = state;

@@ -1469,7 +1469,7 @@
 	ide_hwif_t	*hwif = (ide_hwif_t *)pci_get_drvdata(pdev);
 	int		rc = 0;
 	
-	if (state != pdev->dev.power_state && state >= 2) {
+	if (state != pdev->dev.power_state && state >= 2  && state != 4) {
 		rc = pmac_ide_do_suspend(hwif);
 		if (rc == 0)
 			pdev->dev.power_state = state;

===== drivers/macintosh/Kconfig 1.1 vs edited =====

@@ -29,7 +29,7 @@
 
 config PMAC_PBOOK
 	bool "Power management support for PowerBooks"
-	depends on ADB_PMU
+	depends on PM && ADB_PMU
 	---help---
 	  This provides support for putting a PowerBook to sleep; it also
 	  enables media bay support.  Power management works on the

@@ -46,10 +46,10 @@
 	  have it autoloaded. The act of removing the module shuts down the
 	  sound hardware for more power savings.
 
-config PM
-	bool
-	depends on PPC_PMAC && ADB_PMU && PMAC_PBOOK
-	default y
+#config PM
+#	bool
+#	depends on PPC_PMAC && ADB_PMU && PMAC_PBOOK
+#	default y
 
 config PMAC_APM_EMU
 	tristate "APM emulation"

===== drivers/macintosh/mediabay.c 1.23 vs edited =====

@@ -703,7 +703,7 @@
 {
 	struct media_bay_info	*bay = macio_get_drvdata(mdev);
 
-	if (state != mdev->ofdev.dev.power_state && state >= 2) {
+	if (state != mdev->ofdev.dev.power_state && state >= 2 && state != 4) {
 		down(&bay->lock);
 		bay->sleeping = 1;
 		set_mb_power(bay, 0);

===== drivers/macintosh/via-pmu.c 1.54 vs edited =====

===== drivers/video/aty/aty128fb.c 1.34 vs edited =====

@@ -2251,13 +2251,16 @@
 	 * can properly take care of D3 ? Also, with swsusp, we
 	 * know we'll be rebooted, ...
 	 */
+	if (state != 2 && state != 3)
+		return 0;
+
 #ifdef CONFIG_PPC_PMAC
 	/* HACK ALERT ! Once I find a proper way to say to each driver
 	 * individually what will happen with it's PCI slot, I'll change
 	 * that. On laptops, the AGP slot is just unclocked, so D2 is
 	 * expected, while on desktops, the card is powered off
 	 */
-	if (state >= 3)
+	if (state == 3)
 		state = 2;
 #endif /* CONFIG_PPC_PMAC */
 	 

===== drivers/video/aty/radeon_pm.c 1.4 vs edited =====

@@ -845,6 +845,8 @@
 	 */
 
 	printk(KERN_DEBUG "radeonfb: suspending to state: %d...\n", state);
+	if (state != 2 && state != 3)
+		return 0;
 	
 	acquire_console_sem();
 

===== include/linux/pm.h 1.9 vs edited =====

@@ -195,10 +195,10 @@
 extern void (*pm_power_off)(void);
 
 enum {
-	PM_SUSPEND_ON,
-	PM_SUSPEND_STANDBY,
-	PM_SUSPEND_MEM,
-	PM_SUSPEND_DISK,
+	PM_SUSPEND_ON = 0,
+	PM_SUSPEND_STANDBY = 1,
+	PM_SUSPEND_MEM = 3,
+	PM_SUSPEND_DISK = 4,
 	PM_SUSPEND_MAX,
 };
 

===== include/linux/reboot.h 1.5 vs edited =====

@@ -40,6 +40,8 @@
 extern int register_reboot_notifier(struct notifier_block *);
 extern int unregister_reboot_notifier(struct notifier_block *);
 
+/* For use by swsusp only */
+extern struct notifier_block *reboot_notifier_list;
 
 /*
  * Architecture-specific implementations of sys_reboot commands.

===== include/linux/suspend.h 1.22 vs edited =====

@@ -1,9 +1,9 @@
 #ifndef _LINUX_SWSUSP_H
 #define _LINUX_SWSUSP_H
 
-#ifdef CONFIG_X86
+//#ifdef CONFIG_X86
 #include <asm/suspend.h>
-#endif
+//#endif
 #include <linux/swap.h>
 #include <linux/notifier.h>
 #include <linux/config.h>

===== kernel/sys.c 1.68 vs edited =====

@@ -84,7 +84,7 @@
  *	and the like. 
  */
 
-static struct notifier_block *reboot_notifier_list;
+struct notifier_block *reboot_notifier_list;
 rwlock_t notifier_lock = RW_LOCK_UNLOCKED;
 
 /**

===== kernel/power/disk.c 1.6 vs edited =====

@@ -16,6 +16,7 @@
 #include <linux/string.h>
 #include <linux/delay.h>
 #include <linux/fs.h>
+#include <linux/reboot.h>
 #include "power.h"
 
 

@@ -46,23 +47,26 @@
 	unsigned long flags;
 	int error = 0;
 
-	local_irq_save(flags);
-	device_power_down(PM_SUSPEND_DISK);
 	switch(mode) {
 	case PM_DISK_PLATFORM:
+		local_irq_save(flags);
 		error = pm_ops->enter(PM_SUSPEND_DISK);
+		local_irq_restore(flags);
 		break;
 	case PM_DISK_SHUTDOWN:
 		printk("Powering off system\n");
+		notifier_call_chain(&reboot_notifier_list, SYS_POWER_OFF, NULL);
+		device_shutdown();
 		machine_power_off();
 		break;
 	case PM_DISK_REBOOT:
+		printk("Rebooting system\n");
+		notifier_call_chain(&reboot_notifier_list, SYS_RESTART, NULL);
+		device_shutdown();
 		machine_restart(NULL);
 		break;
 	}
 	machine_halt();
-	device_power_up();
-	local_irq_restore(flags);
 	return 0;
 }
 

@@ -163,8 +167,10 @@
 
 	pr_debug("PM: snapshotting memory.\n");
 	in_suspend = 1;
-	if ((error = pmdisk_save()))
+	if ((error = pmdisk_save())) {
+		pr_debug("PM: snapshot memory failed !\n");
 		goto Done;
+	}
 
 	if (in_suspend) {
 		pr_debug("PM: writing image.\n");

@@ -225,9 +231,6 @@
 	 * Do it with disabled interrupts for best effect. That way, if some
 	 * driver scheduled DMA, we have good chance for DMA to finish ;-).
 	 */
-	pr_debug("PM: Waiting for DMAs to settle down.\n");
-	mdelay(1000);
-
 	pr_debug("PM: Restoring saved image.\n");
 	pmdisk_restore();
 	pr_debug("PM: Restore failed, recovering.n");

===== kernel/power/main.c 1.16 vs edited =====

@@ -120,6 +120,7 @@
 
 char * pm_states[] = {
 	[PM_SUSPEND_STANDBY]	= "standby",
+	[2]			= "",
 	[PM_SUSPEND_MEM]	= "mem",
 	[PM_SUSPEND_DISK]	= "disk",
 	NULL,

===== kernel/power/pmdisk.c 1.80 vs edited =====

@@ -18,7 +18,7 @@
  *
  */
 
-#undef DEBUG
+#define DEBUG
 
 #include <linux/mm.h>
 #include <linux/bio.h>

@@ -28,6 +28,7 @@
 #include <linux/device.h>
 #include <linux/swapops.h>
 #include <linux/bootmem.h>
+#include <linux/utsname.h>
 
 #include <asm/mmu_context.h>
 

@@ -624,8 +625,10 @@
 {
 	int error = 0;
 
-	if ((error = read_swapfiles()))
+	if ((error = read_swapfiles())) {
+		printk("Can't read swapfiles\n");
 		return error;
+	}
 
 	drain_local_pages();
 

@@ -702,6 +705,7 @@
  * Magic happens here
  */
 
+#if 0
 int pmdisk_resume(void)
 {
 	BUG_ON (nr_copy_pages_check != pmdisk_pages);

@@ -711,6 +715,7 @@
 	__flush_tlb_global();
 	return 0;
 }
+#endif
 
 /* pmdisk_arch_suspend() is implemented in arch/?/power/pmdisk.S,
    and basically does:

@@ -1082,9 +1087,11 @@
 	if ((error = arch_prepare_suspend()))
 		return error;
 	local_irq_disable();
+	device_power_down(PM_SUSPEND_DISK);
 	save_processor_state();
 	error = pmdisk_arch_suspend(0);
 	restore_processor_state();
+	device_power_up();
 	local_irq_enable();
 	return error;
 }

@@ -1143,10 +1150,13 @@
 int __init pmdisk_restore(void)
 {
 	int error;
+
 	local_irq_disable();
+	device_power_down(PM_SUSPEND_DISK);
 	save_processor_state();
 	error = pmdisk_arch_suspend(1);
 	restore_processor_state();
+	device_power_up();
 	local_irq_enable();
 	return error;
 }

Then the arch/ppc/kernel/pmdisk.S file:

#include <linux/config.h>
#include <linux/threads.h>
#include <asm/processor.h>
#include <asm/page.h>
#include <asm/cputable.h>
#include <asm/thread_info.h>
#include <asm/ppc_asm.h>
#include <asm/offsets.h>

/*
* Structure for storing CPU registers on the save area. */
#define SL_SP 0
#define SL_PC 4
#define SL_MSR 8
#define SL_SDR1 0xc
#define SL_SPRG0 0x10 /* 4 sprg's */
#define SL_DBAT0 0x20
#define SL_IBAT0 0x28
#define SL_DBAT1 0x30
#define SL_IBAT1 0x38
#define SL_DBAT2 0x40
#define SL_IBAT2 0x48
#define SL_DBAT3 0x50
#define SL_IBAT3 0x58
#define SL_TB 0x60
#define SL_R2 0x68
#define SL_CR 0x6c
#define SL_LR 0x70
#define SL_R12 0x74 /* r12 to r31 */
#define SL_SIZE (SL_R12 + 80)

	.section .data
	.align	5

_GLOBAL(pmdisk_save_area)

	.space	SL_SIZE


	.section .text
	.align	5

_GLOBAL(pmdisk_arch_suspend)

	cmpi	0,r3,0
	bne	do_resume

	lis	r11,pmdisk_save_area@h
	ori	r11,r11,pmdisk_save_area@l

	mflr	r0
	stw	r0,SL_LR(r11)
	mfcr	r0
	stw	r0,SL_CR(r11)
	stw	r1,SL_SP(r11)
	stw	r2,SL_R2(r11)
	stmw	r12,SL_R12(r11)

	/* Save MSR & SDR1 */
	mfmsr	r4
	stw	r4,SL_MSR(r11)
	mfsdr1	r4
	stw	r4,SL_SDR1(r11)

	/* Get a stable timebase and save it */
1:	mftbu	r4
	stw	r4,SL_TB(r11)
	mftb	r5
	stw	r5,SL_TB+4(r11)
	mftbu	r3
	cmpw	r3,r4
	bne	1b

	/* Save SPRGs */
	mfsprg	r4,0
	stw	r4,SL_SPRG0(r11)
	mfsprg	r4,1
	stw	r4,SL_SPRG0+4(r11)
	mfsprg	r4,2
	stw	r4,SL_SPRG0+8(r11)
	mfsprg	r4,3
	stw	r4,SL_SPRG0+12(r11)

	/* Save BATs */
	mfdbatu	r4,0
	stw	r4,SL_DBAT0(r11)
	mfdbatl	r4,0
	stw	r4,SL_DBAT0+4(r11)
	mfdbatu	r4,1
	stw	r4,SL_DBAT1(r11)
	mfdbatl	r4,1
	stw	r4,SL_DBAT1+4(r11)
	mfdbatu	r4,2
	stw	r4,SL_DBAT2(r11)
	mfdbatl	r4,2
	stw	r4,SL_DBAT2+4(r11)
	mfdbatu	r4,3
	stw	r4,SL_DBAT3(r11)
	mfdbatl	r4,3
	stw	r4,SL_DBAT3+4(r11)
	mfibatu	r4,0
	stw	r4,SL_IBAT0(r11)
	mfibatl	r4,0
	stw	r4,SL_IBAT0+4(r11)
	mfibatu	r4,1
	stw	r4,SL_IBAT1(r11)
	mfibatl	r4,1
	stw	r4,SL_IBAT1+4(r11)
	mfibatu	r4,2
	stw	r4,SL_IBAT2(r11)
	mfibatl	r4,2
	stw	r4,SL_IBAT2+4(r11)
	mfibatu	r4,3
	stw	r4,SL_IBAT3(r11)
	mfibatl	r4,3
	stw	r4,SL_IBAT3+4(r11)

#if  0
	/* Backup various CPU config stuffs */
	bl	__save_cpu_setup
#endif
	/* Call the low level suspend stuff (we should probably have made
	 * a stackframe...
	 */
	bl	pmdisk_suspend
	
	/* Restore LR from the save area */
	lis	r11,pmdisk_save_area@h
	ori	r11,r11,pmdisk_save_area@l
	lwz	r0,SL_LR(r11)
	mtlr	r0

	blr

/* Resume code */
do_resume:

	/* Stop pending alitvec streams and memory accesses */

BEGIN_FTR_SECTION
DSSALL
END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
sync

/* Disable MSR:DR to make sure we don't take a TLB or
* hash miss during the copy, as our hash table will
* for a while be unuseable. For .text, we assume we are
* covered by a BAT. This works only for non-G5 at this
* point. G5 will need a better approach, possibly using
* a small temporary hash table filled with large mappings,
* disabling the MMU completely isn't a good option for
* performance reasons.
* (Note that 750's may have the same performance issue as
* the G5 in this case, we should investigate using moving
* BATs for these CPUs)
*/
mfmsr r0
sync
rlwinm r0,r0,0,28,26 /* clear MSR_DR */
mtmsr r0
sync
isync

/* Load ptr the list of pages to copy in r3 */
lis r11,(pm_pagedir_nosave - KERNELBASE)@h
ori r11,r11,pm_pagedir_nosave@l
lwz r10,0(r11)
tophys(r3,r10)

/* Load the count of pages to copy in r4 */
lis r11,(pmdisk_pages - KERNELBASE)@h
ori r11,r11,pmdisk_pages@l
lwz r4,0(r11)

/* Copy the pages. This is a very basic implementation, to
* be replaced by something more cache efficient */
1:

	li	r0,256
	mtctr	r0
	lwz	r11,0(r3)	/* source */
	tophys(r5,r11)
	lwz	r10,4(r3)	/* destination */
	tophys(r6,r10)
2:
	lwz	r8,0(r5)
	lwz	r9,4(r5)
	lwz	r10,8(r5)
	lwz	r11,12(r5)
	addi	r5,r5,16
	stw	r8,0(r6)
	stw	r9,4(r6)
	stw	r10,8(r6)
	stw	r11,12(r6)
	addi	r6,r6,16
	bdnz	2b
	addi	r3,r3,16
	subi	r4,r4,1
	cmpwi	0,r4,0
	bne	1b
	
	/* Do a very simple cache flush/inval of the L1 to ensure
	 * coherency of the icache
	 */
	lis	r3,0x0002
	mtctr	r3
	li	r3, 0
1:
	lwz	r0,0(r3)
	addi	r3,r3,0x0020
	bdnz	1b
	isync
	sync

	/* Now flush those cache lines */
	lis	r3,0x0002
	mtctr	r3
	li	r3, 0
1:
	dcbf	0,r3
	addi	r3,r3,0x0020
	bdnz	1b
	sync

	/* Ok, we are now running with the kernel data of the old
	 * kernel fully restored. We can get to the save area
	 * easily now. As for the rest of the code, it assumes the
	 * loader kernel and the booted one are exactly identical
	 */
	lis	r11,pmdisk_save_area@h
	ori	r11,r11,pmdisk_save_area@l
	tophys(r11,r11)

#if 0
	/* Restore various CPU config stuffs */
	bl	__restore_cpu_setup
#endif
	/* Restore the BATs, and SDR1.  Then we can turn on the MMU. 
	 * This is a bit hairy as we are running out of those BATs,
	 * but first, our code is probably in the icache, and we are
	 * writing the same value to the BAT, so that should be fine,
	 * though a better solution will have to be found long-term
	 */
	lwz	r4,SL_SDR1(r11)
	mtsdr1	r4
	lwz	r4,SL_SPRG0(r11)
	mtsprg	0,r4
	lwz	r4,SL_SPRG0+4(r11)
	mtsprg	1,r4
	lwz	r4,SL_SPRG0+8(r11)
	mtsprg	2,r4
	lwz	r4,SL_SPRG0+12(r11)
	mtsprg	3,r4

#if 0
	lwz	r4,SL_DBAT0(r11)
	mtdbatu	0,r4
	lwz	r4,SL_DBAT0+4(r11)
	mtdbatl	0,r4
	lwz	r4,SL_DBAT1(r11)
	mtdbatu	1,r4
	lwz	r4,SL_DBAT1+4(r11)
	mtdbatl	1,r4
	lwz	r4,SL_DBAT2(r11)
	mtdbatu	2,r4
	lwz	r4,SL_DBAT2+4(r11)
	mtdbatl	2,r4
	lwz	r4,SL_DBAT3(r11)
	mtdbatu	3,r4
	lwz	r4,SL_DBAT3+4(r11)
	mtdbatl	3,r4
	lwz	r4,SL_IBAT0(r11)
	mtibatu	0,r4
	lwz	r4,SL_IBAT0+4(r11)
	mtibatl	0,r4
	lwz	r4,SL_IBAT1(r11)
	mtibatu	1,r4
	lwz	r4,SL_IBAT1+4(r11)
	mtibatl	1,r4
	lwz	r4,SL_IBAT2(r11)
	mtibatu	2,r4
	lwz	r4,SL_IBAT2+4(r11)
	mtibatl	2,r4
	lwz	r4,SL_IBAT3(r11)
	mtibatu	3,r4
	lwz	r4,SL_IBAT3+4(r11)
	mtibatl	3,r4

#endif

BEGIN_FTR_SECTION

	li	r4,0
	mtspr	SPRN_DBAT4U,r4
	mtspr	SPRN_DBAT4L,r4
	mtspr	SPRN_DBAT5U,r4
	mtspr	SPRN_DBAT5L,r4
	mtspr	SPRN_DBAT6U,r4
	mtspr	SPRN_DBAT6L,r4
	mtspr	SPRN_DBAT7U,r4
	mtspr	SPRN_DBAT7L,r4
	mtspr	SPRN_IBAT4U,r4
	mtspr	SPRN_IBAT4L,r4
	mtspr	SPRN_IBAT5U,r4
	mtspr	SPRN_IBAT5L,r4
	mtspr	SPRN_IBAT6U,r4
	mtspr	SPRN_IBAT6L,r4
	mtspr	SPRN_IBAT7U,r4
	mtspr	SPRN_IBAT7L,r4

END_FTR_SECTION_IFSET(CPU_FTR_HAS_HIGH_BATS)

	/* Flush all TLBs */
	lis	r4,0x1000
1:	addic.	r4,r4,-0x1000
	tlbie	r4
	blt	1b
	sync

	/* restore the MSR and turn on the MMU */
	lwz	r3,SL_MSR(r11)
	bl	turn_on_mmu
	tovirt(r11,r11)

	/* Restore TB */
	li	r3,0
	mttbl	r3
	lwz	r3,SL_TB(r11)
	lwz	r4,SL_TB+4(r11)
	mttbu	r3
	mttbl	r4

	/* Kick decrementer */
	li	r0,1
	mtdec	r0

	/* Restore the callee-saved registers and return */
	lwz	r0,SL_CR(r11)
	mtcr	r0
	lwz	r2,SL_R2(r11)
	lmw	r12,SL_R12(r11)
	lwz	r1,SL_SP(r11)
	lwz	r0,SL_LR(r11)
	mtlr	r0

	// XXX Note: we don't really need to call pmdisk_resume

	li	r3,0
	blr

/* FIXME:This construct is actually not useful since we don't shut
* down the instruction MMU, we could just flip back MSR-DR on. */
turn_on_mmu:

	mflr	r4
	mtsrr0	r4
	mtsrr1	r3
	sync
	isync
	rfi

Yes, It works in my laptop too. Cool.

thanks for greate work.

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

On 24 Jan 2004 at 13h01, Benjamin Herrenschmidt wrote:

Hi,

Didn't you forget to include include/asm-ppc/suspend.h ? ;-)

Colin

Yes, but you could have re-created it easily:

static inline int arch_prepare_suspend(void)

{
	return 0;
}

static inline void save_processor_state(void)
{
}

static inline void restore_processor_state(void)
{
}

On 25 Jan 2004 at 10h01, Benjamin Herrenschmidt wrote:

Hi,

Thanks - I wasn't sure about it.
The kernel now builds. However, after doing
echo disk > /sys/power/state
or "hda14" or "/dev/hda14" (which is my swap partition) instead of "disk", nothing happens (and nothing gets logged).
The only things different from your patch is that I added PMAC_MB_CAN_SLEEP to my iBook's entry in pmac_feature.c, and added

	return -EBUSY; 
after your 
        if (state != 2 && state != 3)
                return 0;

in radeon_pm.c (to avoid pmud putting the iBook to sleep by mistake).

relevant .config part:
# CONFIG_SOFTWARE_SUSPEND is not set
CONFIG_PM_DISK=y
CONFIG_PM_DISK_PARTITION="/dev/hda14"

Any pointer ? (did I miss something obvious?)

Thanks,

Colin

Hrm... It tends to do that when it's not happy with something, but I did get it working... Ah yes, do

echo -n "disk" instead :) It doesn't like the trailing \n

Ben.

On 26 Jan 2004 at 11h01, Benjamin Herrenschmidt wrote:

Hi,

Thanks, worked better. Got an oops with ohci-hcd or ehci-hcd, though. shutdown worked after rmmoding these.
However resume did not - got:

dn: cn= pmdisk: CPUs: 544501616
pmdisk: Image: 168453230 Pages
pmdisk: Pagedir: 975201134 Pages
pmdisk: Resume mismatch: kernel version
pmdisk: Error -1 resuming
PM: Resume from disk failed.

of course, this was the same kernel. Don't really know what to try next, but it doesn't matter that much to me, don't lose your time with it if you lack time :)

Colin

pmdisk will save to the first (or the last, I'm not sure) installed swap partition, not what your say on the command line. It will _resume_ from what you say on the command line.

ben.

Works fine on a

processor : 0
cpu : 7455, altivec supported
clock : 867MHz
revision : 3.3 (pvr 8001 0303)
bogomips : 863.00
machine : PowerBook6,1
motherboard : PowerBook6,1 MacRISC3 Power Macintosh
board revision : 00000001
detected as : 287 (PowerBook G4 12")
pmac flags : 0000000b
L2 cache : 256K unified
memory : 256MB
pmac-generation : NewWorld

Cheers,
-- Guido

Haven't looked at it yet. Several crash reports so far, mostly lockups right after printing the number of pages to save. I wonder if we have something broken in there. It dies for me once too at this point.

Also, at least on pmac laptops, the HD is usually so fast, that I suspect spending 10 seconds freeing things is less efficient than spending this 10 seconds writing 200Mb of data to disk :) Also, one wakup, it's quite painful to see everything be swapped in again. It may make sense to be less agressive on the memory freeing, though finding a good balance isn't easy.

Hi.

Yes. That's why suspend2 doesn't free any memory at all by default, but gives the user the option of setting a maximum image size.

Regards,

Nigel

My work on Software Suspend is graciously brought to you by
LinuxFund.org.

HaHa, Here is swsusp2 patch for ppc. Now it WORKS fine.

The code base for testing, any comments and testing are welcome.

-: Howto let it work.
1: download clean 2.6.1 kernel.
2: apply swusp2 2.6.1 patch.
3: apply swsusp2 core patch.
4: apply ppc_swsusp2_1.diff
5: do make menuconfig
CONFIG_SOFTWARE_SUSPEND2=y
CONFIG_SOFTWARE_SUSPEND_DEBUG=y CONFIG_SOFTWARE_SUSPEND_GZIP_COMPRESSION=y CONFIG_SOFTWARE_SUSPEND_LZF_COMPRESSION=y CONFIG_SOFTWARE_SUSPEND_SWAPWRITER=y
6:add "resume2=swap:/dev/hda10" into yaboot.conf.
"/dev/hda10" is your swap partition.
7:do suspend.
I using a sciprt to do it. or use suspend script from swsusp2.sf.net .
8:you system will backed.

-: Plans.
I has created arch/ppc/power directory like i386 arch, and put power
manger code in it.
1: clean code.
2: merge pmac sleep into.

thanks.

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

Index: linux-2.6.1-suspend/kernel/power/swsusp2.c
===================================================================
@@ -54,7 +54,9 @@
 
 #include <linux/suspend-common.h>
 #include <linux/suspend-version-specific.h>
+#ifdef  CONFIG_X86
 #include <asm/i387.h> /* for kernel_fpu_end */
+#endif
 #ifdef	CONFIG_KDB
 #include <linux/kdb.h>
 #endif	/* CONFIG_KDB */

@@ -327,7 +329,9 @@
 	preempt_enable();
 #endif
 
+#ifdef CONFIG_X86
 	kernel_fpu_end();
+#endif
 
 	PRINTPREEMPTCOUNT("Prior to resuming drivers.");
 

@@ -486,10 +490,10 @@
 #endif
 
 	PRINTPREEMPTCOUNT("In resume_2 after copy back.");
-
+#ifdef CONFIG_X86
 	kernel_fpu_end();
-
 	PRINTPREEMPTCOUNT("In resume_2 after fpu end.");
+#endif
 	
 #ifdef CONFIG_PREEMPT
 	preempt_enable();

Index: linux-2.6.1-suspend/include/asm-ppc/suspend.h
===================================================================
@@ -0,0 +1,14 @@
+#ifndef _PPC_SUSPEND_H
+#define _PPC_SUSPEND_H
+
+static inline void flush_tlb_all(void)
+{
+	/* Flush all TLBs */
+	__asm__ __volatile__("lis 4, 0x1000");
+	__asm__ __volatile__("1: addic. 4,4,-0x1000");
+	__asm__ __volatile__("tlbie 4");
+	__asm__ __volatile__("blt 1b");
+	__asm__ __volatile__("sync");
+}
+
+#endif

Index: linux-2.6.1-suspend/arch/ppc/kernel/vmlinux.lds.S
===================================================================
@@ -72,6 +72,12 @@
     CONSTRUCTORS
   }
 
+  . = ALIGN(4096);
+  __nosave_begin = .;
+  .data_nosave : { *(.data.nosave) }
+  . = ALIGN(4096);
+  __nosave_end = .;
+
   . = ALIGN(32);
   .data.cacheline_aligned : { *(.data.cacheline_aligned) }
 

Index: linux-2.6.1-suspend/arch/ppc/kernel/signal.c
===================================================================
@@ -28,6 +28,7 @@
 #include <linux/elf.h>
 #include <linux/tty.h>
 #include <linux/binfmts.h>
+#include <linux/suspend.h>
 #include <asm/ucontext.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>

@@ -552,6 +553,11 @@
 	unsigned long frame, newsp;
 	int signr, ret;
 
+	if (current->flags & PF_FREEZE) {
+		refrigerator(PF_FREEZE);
+		return 0;
+	}
+
 	if (!oldset)
 		oldset = &current->blocked;
 

Index: linux-2.6.1-suspend/arch/ppc/Kconfig
===================================================================
@@ -193,6 +193,8 @@
 
 	  If in doubt, say Y here.
 
+source kernel/power/Kconfig
+
 source arch/ppc/platforms/4xx/Kconfig
 
 config PPC64BRIDGE

Index: linux-2.6.1-suspend/arch/ppc/power/Makefile
===================================================================
@@ -0,0 +1,2 @@
+obj-$(CONFIG_PM) += cpu.o
+obj-$(CONFIG_SOFTWARE_SUSPEND2) += swsusp2-asm.o

previous patch, I has forgot a file, here is it.

Index: arch/ppc/Makefile
===================================================================
@@ -44,6 +44,7 @@
 drivers-$(CONFIG_4xx)      += arch/ppc/4xx_io/
 drivers-$(CONFIG_8260)     += arch/ppc/8260_io/
 drivers-$(CONFIG_OCP)      += arch/ppc/ocp/
+drivers-$(CONFIG_PM)       += arch/ppc/power/
 
 BOOT_TARGETS = zImage zImage.initrd znetboot znetboot.initrd vmlinux.sm
 

Attached file that is diff for the lastest swsusp2.
for clean 2.6.1 + 2.6.1-swsusp2-rc3 + swsusp2-core-rc5

swsusp2 for ppc like very stable, I has been do 5 times, has no problems.

Nice day.

- SWSUSP core : 2.0-rc4
- Kernel Version : 2.6.1
- Version spec. : 2.0.0
- Compiler vers. : 2.95
ide_cd cdrom nfs lockd sunrpc af_packet ohci_hcd snd_pcm_oss snd_mixer_oss i2c_keywest snd_powermac snd_pcm snd_page_alloc snd_timer snd soundcore i2c_core ohci1394 ieee1394 hid sungem sungem_phy reiserfs dm_mod uninorth_agp agpgart usbcore - Attempt number : 5

- Pageset sizes  : 7201 and 56927 (56927 low).
- Parameters     : 0 1 1 0 0 32
- Calculations   : Image size: 2. Ram to suspend: 631.
- Limits         : 65536 pages RAM. Initial boot: 63232.

- Overall expected compression percentage: 2.
- Swapwriter active.
Attempting to automatically swapon: .
Swap available for image: 110002.
- GZIP compressor enabled.
Compressed 129732608 bytes into 103573980.
Image compressed by 20 percent.
- LZF Compressor enabled.
Compressed 262668288 bytes into 129727869.
Image compressed by 50 percent.
- Debugging compiled in.
- Max ranges used: 11352

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

Index: kernel/power/swsusp2.c
===================================================================
@@ -54,7 +54,9 @@
 
 #include <linux/suspend-common.h>
 #include <linux/suspend-version-specific.h>
+#ifdef  CONFIG_X86
 #include <asm/i387.h> /* for kernel_fpu_end */
+#endif
 #ifdef	CONFIG_KDB
 #include <linux/kdb.h>
 #endif	/* CONFIG_KDB */

@@ -343,7 +345,9 @@
 	preempt_enable();
 #endif
 
+#ifdef CONFIG_X86
 	kernel_fpu_end();
+#endif
 
 	PRINTPREEMPTCOUNT("Prior to resuming drivers.");
 

Index: include/asm-ppc/suspend.h
===================================================================
@@ -0,0 +1,14 @@
+#ifndef _PPC_SUSPEND_H
+#define _PPC_SUSPEND_H
+
+static inline void flush_tlb_all(void)
+{
+	/* Flush all TLBs */
+	__asm__ __volatile__("lis 4, 0x1000");
+	__asm__ __volatile__("1: addic. 4,4,-0x1000");
+	__asm__ __volatile__("tlbie 4");
+	__asm__ __volatile__("blt 1b");
+	__asm__ __volatile__("sync");
+}
+
+#endif

Index: arch/ppc/Kconfig
===================================================================
@@ -193,6 +193,8 @@
 
 	  If in doubt, say Y here.
 
+source kernel/power/Kconfig
+
 source arch/ppc/platforms/4xx/Kconfig
 
 config PPC64BRIDGE

Index: arch/ppc/kernel/vmlinux.lds.S
===================================================================
@@ -72,6 +72,12 @@
     CONSTRUCTORS
   }
 
+  . = ALIGN(4096);
+  __nosave_begin = .;
+  .data_nosave : { *(.data.nosave) }
+  . = ALIGN(4096);
+  __nosave_end = .;
+
   . = ALIGN(32);
   .data.cacheline_aligned : { *(.data.cacheline_aligned) }
 

Index: arch/ppc/kernel/signal.c
===================================================================
@@ -28,6 +28,7 @@
 #include <linux/elf.h>
 #include <linux/tty.h>
 #include <linux/binfmts.h>
+#include <linux/suspend.h>
 #include <asm/ucontext.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>

@@ -552,6 +553,11 @@
 	unsigned long frame, newsp;
 	int signr, ret;
 
+	if (current->flags & PF_FREEZE) {
+		refrigerator(PF_FREEZE);
+		return 0;
+	}
+
 	if (!oldset)
 		oldset = &current->blocked;
 

Index: arch/ppc/power/Makefile
===================================================================
@@ -0,0 +1,2 @@
+obj-$(CONFIG_PM) += cpu.o
+obj-$(CONFIG_SOFTWARE_SUSPEND2) += swsusp2-asm.o

Index: arch/ppc/Makefile
===================================================================
@@ -44,6 +44,7 @@
 drivers-$(CONFIG_4xx)		+= arch/ppc/4xx_io/
 drivers-$(CONFIG_8260)		+= arch/ppc/8260_io/
 drivers-$(CONFIG_OCP)		+= arch/ppc/ocp/
+drivers-$(CONFIG_PM)		+= arch/ppc/power/
 
 BOOT_TARGETS = zImage zImage.initrd znetboot znetboot.initrd vmlinux.sm
 

Congratulations on getting it going!

I'll add it to the patch, but mark it as experimental for the moment.

Regards,

Nigel

My work on Software Suspend is graciously brought to you by
LinuxFund.org.

Not easy for me. But do it improve my kernel understand.

thanks.

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

Ok, had a quick look. I _HATE_ those horrible macros you did. Why not just call asm functions or just inline the code ?

Ben.

Good idea, But will try inline first. I can't sure change to call asm function can works. But I'll try.

Thanks for look.

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

As long as you make sure you save the LR in case you need it, you can call asm functions. macros are _evil_ :)

Also, you can remove the code playing with BATs for now, they don't really need to be saved. If the boot kernel sets them up any differently than the saved kernel, we are in trouble anyway. And the G5 has no BATs.

Ben.

Hi!

You may want to try this one.. Maybe it helps?
Pavel

Index: linux/kernel/power/process.c
===================================================================
@@ -49,10 +49,11 @@
 	pr_debug("%s entered refrigerator\n", current->comm);
 	printk("=");
 	current->flags &= ~PF_FREEZE;
-	if (flag)
-		flush_signals(current); /* We have signaled a kernel thread, which isn't normal behaviour
-					   and that may lead to 100%CPU sucking because those threads
-					   just don't manage signals. */
+
+	spin_lock_irq(&current->sighand->siglock);
+	recalc_sigpending(); /* We sent fake signal, clean it up */
+	spin_unlock_irq(&current->sighand->siglock);
+
 	current->flags |= PF_FROZEN;
 	while (current->flags & PF_FROZEN)
 		schedule();

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

I has check the kernel, I using swsusp2 patch, the process freeze like this, but still need the patch I lastest send.

           if (flag) {
                swsusp_spin_lock_irqsave(PROCESS_SIG_MASK(current),

flags); RECALC_SIGPENDING;
swsusp_spin_unlock_irqrestore(PROCESS_SIG_MASK(current),
flags); }

Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

I'll have a look tomorrow, that code was a quick hack indeed.

Ben.

Hi!

Notice that swsusp needs half of physical memory free by design. That means that we need _some_ freeing. Nigel's swsusp2 works around that at cost of more complicated implementation.
Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Yes, my method is a bit more complicated.

Yours doesn't always need some freeing though - you only need to free memory until that 1/2 limitation is met. Last time I looked at it, it freed memory until it could free no more. Is that still true?

Nigel

My work on Software Suspend is graciously brought to you by
LinuxFund.org.

Hi!

Yes. [It could be changed, but at that point I'd have to figure out how much memory I really do need for write buffers, bio allocation, etc.]

Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

The PMU locking up with interrupts disabled, for me (Lombard, 400 MHz). Seems to happen more often on slow machines.
Otherwise, it works (saving to the last swap partition if you try to specify which one to use; patch sent to Ben FWIW).

I don't mind the wait for memory being freed. Copying more pages with interrupts disabled would mean a higher chance for PMU lockup, won't it?

Michael

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

It's also possible that there is a bug and it just crashes :)

Also, I do suspend devices before the page copy, and I quickly hacked a PMU sysdev that should suspend ADB interrupts (at least) so that should be fine, except if you have a brightness request in flight maybe ?

The whole PMU suspend thing must be reworked I suppose. (And the battery monitoring thing "paused" as well, we must wait for async PMU requests, that is at least battery requests and backlight ones to stop)

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

I haven't. I've added another pmu_suspend/pmu_resume around the bulk of pmdisk_save, that seems to have improved things (but not completely). It's executed outside your sysdev calls, so it shouldn't really improve things.

Battery requests might have been underway; pmud is running. If I read pmu_suspend() right, pending battery requests should result in pmu_suspend waiting for them to finish?

Michael

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Normally yes... pmud isn't running at this point (all processes are frozen)

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

OK. I put a mdelay(100) between the PMU_SYSTEM_READY request and the pmu_suspend() call in pmu_sys_resume and it seems to work so far. Side
note: if you are saving a large system and have multiple swap files, play with swap priorities to make sure the pages are swapped out to the file _not_ used for suspend.

Michael

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Can you try with swsusp2 ppc port, if you have time.

:)

--
Hu Gang / Steve
Linux Registered User 204016
GPG Public Key: http://soulinfo.com/~hugang/HuGang.asc

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

It would be really nice if you could provide a unique diff, would be easier to test ;-)

--
Colin, lazy

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Seconded. And I'd like to have it all in one single diff - I started losing track of what patch to apply on top of the first one already.

Michael

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

I'll have a look at it. Unless you take care of PMU timeouts some way, I'd dare predict it won't work, though.

Michael

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Well, I though you would do it :) The basis is the same, only the page copy routine in the assembly has to change afaik...

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Hrm... Remove the PMU_SYSTEM_READY, it's a bug to call it on a 101... And it's not needed anyway.

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

I'm trying a if (pmu_kind != 3) around that call - should it rather be <= 3 ?

Michael

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Just remove it for you tests. Only KEYLARGO based PMUs have this call

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Ah ? What's up ? suspend trashes the swap ? That's very bad...

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Hi!

No, it does not.

But swap space is shared between suspend and swap. And if that place is full (unlikely), you've got a problem.
Pavel
--
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Nope; freeing memory puts everything in the highest priority swap (happens to be the first one), and suspend doesn't find enough room in that swap area. The calculation of free swap done by pmdisk is bogus, it calculates total free swap, not free swap in the area that pmdisk is going to use for the snapshot.

Michael

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Hi!

What is "PMU locking"? Is there some part of powermac harware that kills machine if you sleep too long with interrupts disabled?

Pavel
--
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

The PMU is the "Power Management Unit". It's a microcontroller that does a bunch of things, some PM related, but also the ADB emulation for the keyboard & trackpad on laptops, the real time clock, battery management, etc...

We communicate with it via an SPI, it's asynchronous (interrupt driven) protocol (banging things to a shit register basically).

The kernel is continuously talking to the PMU. If for some reason communication stops in the middle of a message exchange, the PMU can time out and shuts the box down.

"PMU suspend" is a hook I added to the driver that completes pending requests and stops answering to "event" interrupts (those interrupts are sent by the PMU when we need to read it's incoming event messages, like keystrokes, environement events, etc...).

When we get such an interrupt, we send a command to read the event and get the data in the reply. The PMU can timeout in the middle of that exchange, but won't timeout if we don't ask for the data at all...

On suspend-to-RAM, since the PMU driver drives the whole suspend process, there is no real problem, it does things the right way.

But with suspend-to-disk, it's just yet-another device, and so we need to make sure we properly stop pending communication. The quick hack I did (added a sysdev with suspend/resume hooks for it) may not be perfect yet :)

Ben.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

(RESENT, sorry if you got it already, something apparently went wrong on the SMTP here)

Ok, I hammered that for a day and got pmdisk (patrick's version) suspending and resuming on a pismo G3 (with XFree etc.. running). Lots of rough edges still (via-pmu sleep need to be improved, ADB need porting to the new driver model to be properly suspended/resumed, a sysdev for RTC is needed too for time, the asm code should be fixed for G5, etc...)

I had to fix some issues in the core pmdisk code though. One big one is that lots of drivers expect suspend to disk to be state 4 while the current code used state 3 for that (and suspend to RAM to be state 3 btw). I hacked that in include/linux/suspend.h, but we shall probably just get rid of those stupid numbers and properly define each constant indstead.

We should also use a different state for the suspend calls done before saving the image, and the ones done before resuming the image, some driver may be optimized for these cases.

The patch is against my tree currently, and the arch/ppc/kernel/pmdisk.S file is appended as-is (not in patch form). I don't plan to release that right now, I may hack a bit on it in the "background" (I want to get HIGHMEM working some day). Feel free to improve, but then keep me informed please.

Ah, also: The "Freeing memory" phase takes forever. That should really be fixed.

Ben.

First the patch:

===== arch/ppc/Kconfig 1.60 vs edited =====

@@ -913,6 +913,8 @@
 
 source "drivers/zorro/Kconfig"
 
+source kernel/power/Kconfig
+
 endmenu
 
 menu "Bus options"

===== arch/ppc/kernel/Makefile 1.61 vs edited =====

@@ -15,6 +15,7 @@
 extra-$(CONFIG_8xx)		:= head_8xx.o
 extra-$(CONFIG_6xx)		+= idle_6xx.o
 extra-$(CONFIG_POWER4)		+= idle_power4.o
+
 extra-y				+= vmlinux.lds.s
 
 obj-y				:= entry.o traps.o irq.o idle.o time.o misc.o \

@@ -22,6 +23,7 @@
 					semaphore.o syscalls.o setup.o \
 					cputable.o ppc_htab.o
 obj-$(CONFIG_6xx)		+= l2cr.o cpu_setup_6xx.o
+obj-$(CONFIG_PM_DISK)		+= pmdisk.o
 obj-$(CONFIG_POWER4)		+= cpu_setup_power4.o
 obj-$(CONFIG_MODULES)		+= module.o ppc_ksyms.o
 obj-$(CONFIG_PCI)		+= pci.o

===== arch/ppc/kernel/signal.c 1.37 vs edited =====

@@ -28,6 +28,7 @@
 #include <linux/elf.h>
 #include <linux/tty.h>
 #include <linux/binfmts.h>
+#include <linux/suspend.h>
 #include <asm/ucontext.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>

@@ -563,6 +564,11 @@
 	struct k_sigaction *ka;
 	unsigned long frame, newsp;
 	int signr, ret;
+
+	if (current->flags & PF_FREEZE) {
+		refrigerator(0);
+		return 0;
+	}
 
 	if (!oldset)
 		oldset = &current->blocked;

===== arch/ppc/kernel/vmlinux.lds.S 1.39 vs edited =====

@@ -72,6 +72,12 @@
     CONSTRUCTORS
   }
 
+  . = ALIGN(4096);
+  __nosave_begin = .;
+  .data_nosave : { *(.data.nosave) }
+  . = ALIGN(4096);
+  __nosave_end = .;
+
   . = ALIGN(32);
   .data.cacheline_aligned : { *(.data.cacheline_aligned) }
 

===== arch/ppc/platforms/pmac_setup.c 1.48 vs edited =====

===== drivers/ide/ppc/pmac.c 1.47 vs edited =====

@@ -1366,7 +1366,7 @@
 	ide_hwif_t	*hwif = (ide_hwif_t *)dev_get_drvdata(&mdev->ofdev.dev);
 	int		rc = 0;
 
-	if (state != mdev->ofdev.dev.power_state && state >= 2) {
+	if (state != mdev->ofdev.dev.power_state && state >= 2 && state != 4) {
 		rc = pmac_ide_do_suspend(hwif);
 		if (rc == 0)
 			mdev->ofdev.dev.power_state = state;

@@ -1469,7 +1469,7 @@
 	ide_hwif_t	*hwif = (ide_hwif_t *)pci_get_drvdata(pdev);
 	int		rc = 0;
 	
-	if (state != pdev->dev.power_state && state >= 2) {
+	if (state != pdev->dev.power_state && state >= 2  && state != 4) {
 		rc = pmac_ide_do_suspend(hwif);
 		if (rc == 0)
 			pdev->dev.power_state = state;

===== drivers/macintosh/Kconfig 1.1 vs edited =====

@@ -29,7 +29,7 @@
 
 config PMAC_PBOOK
 	bool "Power management support for PowerBooks"
-	depends on ADB_PMU
+	depends on PM && ADB_PMU
 	---help---
 	  This provides support for putting a PowerBook to sleep; it also
 	  enables media bay support.  Power management works on the

@@ -46,10 +46,10 @@
 	  have it autoloaded. The act of removing the module shuts down the
 	  sound hardware for more power savings.
 
-config PM
-	bool
-	depends on PPC_PMAC && ADB_PMU && PMAC_PBOOK
-	default y
+#config PM
+#	bool
+#	depends on PPC_PMAC && ADB_PMU && PMAC_PBOOK
+#	default y
 
 config PMAC_APM_EMU
 	tristate "APM emulation"

===== drivers/macintosh/mediabay.c 1.23 vs edited =====

@@ -703,7 +703,7 @@
 {
 	struct media_bay_info	*bay = macio_get_drvdata(mdev);
 
-	if (state != mdev->ofdev.dev.power_state && state >= 2) {
+	if (state != mdev->ofdev.dev.power_state && state >= 2 && state != 4) {
 		down(&bay->lock);
 		bay->sleeping = 1;
 		set_mb_power(bay, 0);

===== drivers/macintosh/via-pmu.c 1.54 vs edited =====

===== drivers/video/aty/aty128fb.c 1.34 vs edited =====

@@ -2251,13 +2251,16 @@
 	 * can properly take care of D3 ? Also, with swsusp, we
 	 * know we'll be rebooted, ...
 	 */
+	if (state != 2 && state != 3)
+		return 0;
+
 #ifdef CONFIG_PPC_PMAC
 	/* HACK ALERT ! Once I find a proper way to say to each driver
 	 * individually what will happen with it's PCI slot, I'll change
 	 * that. On laptops, the AGP slot is just unclocked, so D2 is
 	 * expected, while on desktops, the card is powered off
 	 */
-	if (state >= 3)
+	if (state == 3)
 		state = 2;
 #endif /* CONFIG_PPC_PMAC */
 	 

===== drivers/video/aty/radeon_pm.c 1.4 vs edited =====

@@ -845,6 +845,8 @@
 	 */
 
 	printk(KERN_DEBUG "radeonfb: suspending to state: %d...\n", state);
+	if (state != 2 && state != 3)
+		return 0;
 	
 	acquire_console_sem();
 

===== include/linux/pm.h 1.9 vs edited =====

@@ -195,10 +195,10 @@
 extern void (*pm_power_off)(void);
 
 enum {
-	PM_SUSPEND_ON,
-	PM_SUSPEND_STANDBY,
-	PM_SUSPEND_MEM,
-	PM_SUSPEND_DISK,
+	PM_SUSPEND_ON = 0,
+	PM_SUSPEND_STANDBY = 1,
+	PM_SUSPEND_MEM = 3,
+	PM_SUSPEND_DISK = 4,
 	PM_SUSPEND_MAX,
 };
 

===== include/linux/reboot.h 1.5 vs edited =====

@@ -40,6 +40,8 @@
 extern int register_reboot_notifier(struct notifier_block *);
 extern int unregister_reboot_notifier(struct notifier_block *);
 
+/* For use by swsusp only */
+extern struct notifier_block *reboot_notifier_list;
 
 /*
  * Architecture-specific implementations of sys_reboot commands.

===== include/linux/suspend.h 1.22 vs edited =====

@@ -1,9 +1,9 @@
 #ifndef _LINUX_SWSUSP_H
 #define _LINUX_SWSUSP_H
 
-#ifdef CONFIG_X86
+//#ifdef CONFIG_X86
 #include <asm/suspend.h>
-#endif
+//#endif
 #include <linux/swap.h>
 #include <linux/notifier.h>
 #include <linux/config.h>

===== kernel/sys.c 1.68 vs edited =====

@@ -84,7 +84,7 @@
  *	and the like. 
  */
 
-static struct notifier_block *reboot_notifier_list;
+struct notifier_block *reboot_notifier_list;
 rwlock_t notifier_lock = RW_LOCK_UNLOCKED;
 
 /**

===== kernel/power/disk.c 1.6 vs edited =====

@@ -16,6 +16,7 @@
 #include <linux/string.h>
 #include <linux/delay.h>
 #include <linux/fs.h>
+#include <linux/reboot.h>
 #include "power.h"
 
 

@@ -46,23 +47,26 @@
 	unsigned long flags;
 	int error = 0;
 
-	local_irq_save(flags);
-	device_power_down(PM_SUSPEND_DISK);
 	switch(mode) {
 	case PM_DISK_PLATFORM:
+		local_irq_save(flags);
 		error = pm_ops->enter(PM_SUSPEND_DISK);
+		local_irq_restore(flags);
 		break;
 	case PM_DISK_SHUTDOWN:
 		printk("Powering off system\n");
+		notifier_call_chain(&reboot_notifier_list, SYS_POWER_OFF, NULL);
+		device_shutdown();
 		machine_power_off();
 		break;
 	case PM_DISK_REBOOT:
+		printk("Rebooting system\n");
+		notifier_call_chain(&reboot_notifier_list, SYS_RESTART, NULL);
+		device_shutdown();
 		machine_restart(NULL);
 		break;
 	}
 	machine_halt();
-	device_power_up();
-	local_irq_restore(flags);
 	return 0;
 }
 

@@ -163,8 +167,10 @@
 
 	pr_debug("PM: snapshotting memory.\n");
 	in_suspend = 1;
-	if ((error = pmdisk_save()))
+	if ((error = pmdisk_save())) {
+		pr_debug("PM: snapshot memory failed !\n");
 		goto Done;
+	}
 
 	if (in_suspend) {
 		pr_debug("PM: writing image.\n");

@@ -225,9 +231,6 @@
 	 * Do it with disabled interrupts for best effect. That way, if some
 	 * driver scheduled DMA, we have good chance for DMA to finish ;-).
 	 */
-	pr_debug("PM: Waiting for DMAs to settle down.\n");
-	mdelay(1000);
-
 	pr_debug("PM: Restoring saved image.\n");
 	pmdisk_restore();
 	pr_debug("PM: Restore failed, recovering.n");

===== kernel/power/main.c 1.16 vs edited =====

@@ -120,6 +120,7 @@
 
 char * pm_states[] = {
 	[PM_SUSPEND_STANDBY]	= "standby",
+	[2]			= "",
 	[PM_SUSPEND_MEM]	= "mem",
 	[PM_SUSPEND_DISK]	= "disk",
 	NULL,

===== kernel/power/pmdisk.c 1.80 vs edited =====

@@ -18,7 +18,7 @@
  *
  */
 
-#undef DEBUG
+#define DEBUG
 
 #include <linux/mm.h>
 #include <linux/bio.h>

@@ -28,6 +28,7 @@
 #include <linux/device.h>
 #include <linux/swapops.h>
 #include <linux/bootmem.h>
+#include <linux/utsname.h>
 
 #include <asm/mmu_context.h>
 

@@ -624,8 +625,10 @@
 {
 	int error = 0;
 
-	if ((error = read_swapfiles()))
+	if ((error = read_swapfiles())) {
+		printk("Can't read swapfiles\n");
 		return error;
+	}
 
 	drain_local_pages();
 

@@ -702,6 +705,7 @@
  * Magic happens here
  */
 
+#if 0
 int pmdisk_resume(void)
 {
 	BUG_ON (nr_copy_pages_check != pmdisk_pages);

@@ -711,6 +715,7 @@
 	__flush_tlb_global();
 	return 0;
 }
+#endif
 
 /* pmdisk_arch_suspend() is implemented in arch/?/power/pmdisk.S,
    and basically does:

@@ -1082,9 +1087,11 @@
 	if ((error = arch_prepare_suspend()))
 		return error;
 	local_irq_disable();
+	device_power_down(PM_SUSPEND_DISK);
 	save_processor_state();
 	error = pmdisk_arch_suspend(0);
 	restore_processor_state();
+	device_power_up();
 	local_irq_enable();
 	return error;
 }

@@ -1143,10 +1150,13 @@
 int __init pmdisk_restore(void)
 {
 	int error;
+
 	local_irq_disable();
+	device_power_down(PM_SUSPEND_DISK);
 	save_processor_state();
 	error = pmdisk_arch_suspend(1);
 	restore_processor_state();
+	device_power_up();
 	local_irq_enable();
 	return error;
 }

Then the arch/ppc/kernel/pmdisk.S file:

#include <linux/config.h>
#include <linux/threads.h>
#include <asm/processor.h>
#include <asm/page.h>
#include <asm/cputable.h>
#include <asm/thread_info.h>
#include <asm/ppc_asm.h>
#include <asm/offsets.h>

/*
* Structure for storing CPU registers on the save area. */
#define SL_SP 0
#define SL_PC 4
#define SL_MSR 8
#define SL_SDR1 0xc
#define SL_SPRG0 0x10 /* 4 sprg's */
#define SL_DBAT0 0x20
#define SL_IBAT0 0x28
#define SL_DBAT1 0x30
#define SL_IBAT1 0x38
#define SL_DBAT2 0x40
#define SL_IBAT2 0x48
#define SL_DBAT3 0x50
#define SL_IBAT3 0x58
#define SL_TB 0x60
#define SL_R2 0x68
#define SL_CR 0x6c
#define SL_LR 0x70
#define SL_R12 0x74 /* r12 to r31 */
#define SL_SIZE (SL_R12 + 80)

	.section .data
	.align	5

_GLOBAL(pmdisk_save_area)

	.space	SL_SIZE


	.section .text
	.align	5

_GLOBAL(pmdisk_arch_suspend)

	cmpi	0,r3,0
	bne	do_resume

	lis	r11,pmdisk_save_area@h
	ori	r11,r11,pmdisk_save_area@l

	mflr	r0
	stw	r0,SL_LR(r11)
	mfcr	r0
	stw	r0,SL_CR(r11)
	stw	r1,SL_SP(r11)
	stw	r2,SL_R2(r11)
	stmw	r12,SL_R12(r11)

	/* Save MSR & SDR1 */
	mfmsr	r4
	stw	r4,SL_MSR(r11)
	mfsdr1	r4
	stw	r4,SL_SDR1(r11)

	/* Get a stable timebase and save it */
1:	mftbu	r4
	stw	r4,SL_TB(r11)
	mftb	r5
	stw	r5,SL_TB+4(r11)
	mftbu	r3
	cmpw	r3,r4
	bne	1b

	/* Save SPRGs */
	mfsprg	r4,0
	stw	r4,SL_SPRG0(r11)
	mfsprg	r4,1
	stw	r4,SL_SPRG0+4(r11)
	mfsprg	r4,2
	stw	r4,SL_SPRG0+8(r11)
	mfsprg	r4,3
	stw	r4,SL_SPRG0+12(r11)

	/* Save BATs */
	mfdbatu	r4,0
	stw	r4,SL_DBAT0(r11)
	mfdbatl	r4,0
	stw	r4,SL_DBAT0+4(r11)
	mfdbatu	r4,1
	stw	r4,SL_DBAT1(r11)
	mfdbatl	r4,1
	stw	r4,SL_DBAT1+4(r11)
	mfdbatu	r4,2
	stw	r4,SL_DBAT2(r11)
	mfdbatl	r4,2
	stw	r4,SL_DBAT2+4(r11)
	mfdbatu	r4,3
	stw	r4,SL_DBAT3(r11)
	mfdbatl	r4,3
	stw	r4,SL_DBAT3+4(r11)
	mfibatu	r4,0
	stw	r4,SL_IBAT0(r11)
	mfibatl	r4,0
	stw	r4,SL_IBAT0+4(r11)
	mfibatu	r4,1
	stw	r4,SL_IBAT1(r11)
	mfibatl	r4,1
	stw	r4,SL_IBAT1+4(r11)
	mfibatu	r4,2
	stw	r4,SL_IBAT2(r11)
	mfibatl	r4,2
	stw	r4,SL_IBAT2+4(r11)
	mfibatu	r4,3
	stw	r4,SL_IBAT3(r11)
	mfibatl	r4,3
	stw	r4,SL_IBAT3+4(r11)

#if  0
	/* Backup various CPU config stuffs */
	bl	__save_cpu_setup
#endif
	/* Call the low level suspend stuff (we should probably have made
	 * a stackframe...
	 */
	bl	pmdisk_suspend
	
	/* Restore LR from the save area */
	lis	r11,pmdisk_save_area@h
	ori	r11,r11,pmdisk_save_area@l
	lwz	r0,SL_LR(r11)
	mtlr	r0

	blr

/* Resume code */
do_resume:

	/* Stop pending alitvec streams and memory accesses */

BEGIN_FTR_SECTION
DSSALL
END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
sync

/* Disable MSR:DR to make sure we don't take a TLB or
* hash miss during the copy, as our hash table will
* for a while be unuseable. For .text, we assume we are
* covered by a BAT. This works only for non-G5 at this
* point. G5 will need a better approach, possibly using
* a small temporary hash table filled with large mappings,
* disabling the MMU completely isn't a good option for
* performance reasons.
* (Note that 750's may have the same performance issue as
* the G5 in this case, we should investigate using moving
* BATs for these CPUs)
*/
mfmsr r0
sync
rlwinm r0,r0,0,28,26 /* clear MSR_DR */
mtmsr r0
sync
isync

/* Load ptr the list of pages to copy in r3 */
lis r11,(pm_pagedir_nosave - KERNELBASE)@h
ori r11,r11,pm_pagedir_nosave@l
lwz r10,0(r11)
tophys(r3,r10)

/* Load the count of pages to copy in r4 */
lis r11,(pmdisk_pages - KERNELBASE)@h
ori r11,r11,pmdisk_pages@l
lwz r4,0(r11)

/* Copy the pages. This is a very basic implementation, to
* be replaced by something more cache efficient */
1:

	li	r0,256
	mtctr	r0
	lwz	r11,0(r3)	/* source */
	tophys(r5,r11)
	lwz	r10,4(r3)	/* destination */
	tophys(r6,r10)
2:
	lwz	r8,0(r5)
	lwz	r9,4(r5)
	lwz	r10,8(r5)
	lwz	r11,12(r5)
	addi	r5,r5,16
	stw	r8,0(r6)
	stw	r9,4(r6)
	stw	r10,8(r6)
	stw	r11,12(r6)
	addi	r6,r6,16
	bdnz	2b
	addi	r3,r3,16
	subi	r4,r4,1
	cmpwi	0,r4,0
	bne	1b
	
	/* Do a very simple cache flush/inval of the L1 to ensure
	 * coherency of the icache
	 */
	lis	r3,0x0002
	mtctr	r3
	li	r3, 0
1:
	lwz	r0,0(r3)
	addi	r3,r3,0x0020
	bdnz	1b
	isync
	sync

	/* Now flush those cache lines */
	lis	r3,0x0002
	mtctr	r3
	li	r3, 0
1:
	dcbf	0,r3
	addi	r3,r3,0x0020
	bdnz	1b
	sync

	/* Ok, we are now running with the kernel data of the old
	 * kernel fully restored. We can get to the save area
	 * easily now. As for the rest of the code, it assumes the
	 * loader kernel and the booted one are exactly identical
	 */
	lis	r11,pmdisk_save_area@h
	ori	r11,r11,pmdisk_save_area@l
	tophys(r11,r11)

#if 0
	/* Restore various CPU config stuffs */
	bl	__restore_cpu_setup
#endif
	/* Restore the BATs, and SDR1.  Then we can turn on the MMU. 
	 * This is a bit hairy as we are running out of those BATs,
	 * but first, our code is probably in the icache, and we are
	 * writing the same value to the BAT, so that should be fine,
	 * though a better solution will have to be found long-term
	 */
	lwz	r4,SL_SDR1(r11)
	mtsdr1	r4
	lwz	r4,SL_SPRG0(r11)
	mtsprg	0,r4
	lwz	r4,SL_SPRG0+4(r11)
	mtsprg	1,r4
	lwz	r4,SL_SPRG0+8(r11)
	mtsprg	2,r4
	lwz	r4,SL_SPRG0+12(r11)
	mtsprg	3,r4

#if 0
	lwz	r4,SL_DBAT0(r11)
	mtdbatu	0,r4
	lwz	r4,SL_DBAT0+4(r11)
	mtdbatl	0,r4
	lwz	r4,SL_DBAT1(r11)
	mtdbatu	1,r4
	lwz	r4,SL_DBAT1+4(r11)
	mtdbatl	1,r4
	lwz	r4,SL_DBAT2(r11)
	mtdbatu	2,r4
	lwz	r4,SL_DBAT2+4(r11)
	mtdbatl	2,r4
	lwz	r4,SL_DBAT3(r11)
	mtdbatu	3,r4
	lwz	r4,SL_DBAT3+4(r11)
	mtdbatl	3,r4
	lwz	r4,SL_IBAT0(r11)
	mtibatu	0,r4
	lwz	r4,SL_IBAT0+4(r11)
	mtibatl	0,r4
	lwz	r4,SL_IBAT1(r11)
	mtibatu	1,r4
	lwz	r4,SL_IBAT1+4(r11)
	mtibatl	1,r4
	lwz	r4,SL_IBAT2(r11)
	mtibatu	2,r4
	lwz	r4,SL_IBAT2+4(r11)
	mtibatl	2,r4
	lwz	r4,SL_IBAT3(r11)
	mtibatu	3,r4
	lwz	r4,SL_IBAT3+4(r11)
	mtibatl	3,r4

#endif

BEGIN_FTR_SECTION

	li	r4,0
	mtspr	SPRN_DBAT4U,r4
	mtspr	SPRN_DBAT4L,r4
	mtspr	SPRN_DBAT5U,r4
	mtspr	SPRN_DBAT5L,r4
	mtspr	SPRN_DBAT6U,r4
	mtspr	SPRN_DBAT6L,r4
	mtspr	SPRN_DBAT7U,r4
	mtspr	SPRN_DBAT7L,r4
	mtspr	SPRN_IBAT4U,r4
	mtspr	SPRN_IBAT4L,r4
	mtspr	SPRN_IBAT5U,r4
	mtspr	SPRN_IBAT5L,r4
	mtspr	SPRN_IBAT6U,r4
	mtspr	SPRN_IBAT6L,r4
	mtspr	SPRN_IBAT7U,r4
	mtspr	SPRN_IBAT7L,r4

END_FTR_SECTION_IFSET(CPU_FTR_HAS_HIGH_BATS)

	/* Flush all TLBs */
	lis	r4,0x1000
1:	addic.	r4,r4,-0x1000
	tlbie	r4
	blt	1b
	sync

	/* restore the MSR and turn on the MMU */
	lwz	r3,SL_MSR(r11)
	bl	turn_on_mmu
	tovirt(r11,r11)

	/* Restore TB */
	li	r3,0
	mttbl	r3
	lwz	r3,SL_TB(r11)
	lwz	r4,SL_TB+4(r11)
	mttbu	r3
	mttbl	r4

	/* Kick decrementer */
	li	r0,1
	mtdec	r0

	/* Restore the callee-saved registers and return */
	lwz	r0,SL_CR(r11)
	mtcr	r0
	lwz	r2,SL_R2(r11)
	lmw	r12,SL_R12(r11)
	lwz	r1,SL_SP(r11)
	lwz	r0,SL_LR(r11)
	mtlr	r0

	// XXX Note: we don't really need to call pmdisk_resume

	li	r3,0
	blr

/* FIXME:This construct is actually not useful since we don't shut
* down the instruction MMU, we could just flip back MSR-DR on. */
turn_on_mmu:

	mflr	r4
	mtsrr0	r4
	mtsrr1	r3
	sync
	isync
	rfi

Hi!

Congratulations!
Pavel
[Now we'll have to do something with pmdisk vs. swsusp...]

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

pmdisk is the only one that ever was close to working on my laptop, do don't kill that one. Some driver is (was?) preventing it from resuming properly. Is there some list somewhere of drivers known to be pmdisk safe?

Måns Rullgård
[email protected]

Hi!

Well, then what you do is not swsusp.

swsusp does assume same kernel during suspend and resume. Doing resume within bootloader (and thus avoiding this) would be completely different design.

I hope we do shut devices down. In my tree at least.
Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Wait... what the hell in swsusp requires this assumption ? It seems to me like a completely unnecessary design limitation.

Ben.

Hi!

(1) There's routine during resume that copies pages to their old locations. If you (would want to) have different kernel during resume, how do you guarantee that that "kernel being resumed" does not use memory ocupied by copying routine?

(2) Plus number of problems with devices grows with number of versions squared. To guarantee it works properly you'd have to test all combinations of "suspend kernel" and "resume kernel".

[(1) Could be solved by reserving 4KB somewhere for copy routine, and making sure copy routine is never bigger than 4KB etc. But I'd like to keep it simple and really don't want to deal with (2).]
Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

By having the copy routine sit elsewhere. You can have the copy routine be in a known location of the kernel beeing resumed (that is it uses its own copy routine) that is aligned on a page boundary and knows how to copy itself. Fairly trivial.

Why ? You aren't passing any device/driver information from the boot kernel and the resumed one... do you ?

Then you don't wnat to do things properly...

Ben.

Hi!

I said it is possible to solve. But "copy routine" becomes even more nasty than it is already.

I'm not passing device information, but devices *do* have internal state. I quiesce them before booting new kernel, but there's probably more than one way to quiesce devices...

No, I really do not want to make things more complicated in 2.6. And you should not want to complicate it, too.
Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Not that many. You don't quite know in what state they are when the BIOS calls you neither in most cases :) Nor when waking up from BIOS-managed state.... It's usually safe if they just don't bust master and are idle.

I will not impose that limitation on a ppc implementation. I don't even want to load the resume image from the boot kernel, it's much more easier to load it from the bootloader for me anyway. And the copy routine is just a tricky bit of asm, not even _that_ tricky I'd say (well, I don't do x86 asm, but I've certainly had to deal with more tricky stuffs on ppc so far).

Ben.

Hi!

Well, if you have just one common bootloader on ppc, no problem, go ahead at load it from bootloader. You'll not have to worry about page tables etc. I could not do that on i386 because there are many bootloaders in use here.

You are going to have completely different resume phase, through. [And if someone wants suspend-to-disk on ppc, *now*, it might still be easiest to write those two screens of assembly to port swsusp to ppc.]

Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Swsusp saves the data structures from the suspended kernel, so they have to match the data structures of the resumed kernel, right?

I't s a bit like trying insmod -f on a module compiled for a completely different kernel version... *bang*

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Hi!

Well, *all* the data pages are saved, so that would be okay (even if they changed, as I'm replacing all the data pages, that should work), but I'm not saving kernel text for example.
Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Ahh... that's an interesting point. You aren't saving kernel text. I'm not sure how that could be a problem for me. I think i'll just save it along with the image though. I think there is much risk screwing up because an uncompatible boot/load kernel pair here than because devices aren't fully idle.

Ben.

Hi!

Also pay attetion to page tables. I know that page tables "copy" routine is running from are same between suspend and resume kernel.

Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

I plan to run everything provided by the suspended kernel actually. My idea is to keep a handle to a page of the suspended kernel that contains that code and just kick into it. Copying pages to their final location without overriding the source pages is a bit of a funky job, but I had to do it already with BootX so ... I'll work on that during one of the upcoming few weeks hopefully, I'm a bit swamped with 3 different things at the moment.

ben.

Hi!

You actually *know* that half of memory is free during resume, so you can load kernel-to-be-resumed data into that free memory. That's how swsusp works. It makes it quite simple...
Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

I'm really interested to see how you are going to initiate and execute the suspend.

Perhaps I'm just ignorant, but I can't see how you can do it without resorting to the same tricks we use now with regards to CPU context. I think you're going to find yourself reinventing the wheel.

Regards,

Nigel

My work on Software Suspend is graciously brought to you by
LinuxFund.org.

CPU context ? I don't think swsusp needs anything different than suspend-to-RAM in this regard (at least the pmac version of suspend-to -RAM that wakes up on a CPU reset). The CPU setup will be saved as part of the kernel data and will be restored after the pages are back in place the same way it is restored when waking up from RAM. The code is already there (and actually, that ppc port of swsusp borrowed it).

Ben.

No, swsusp saves the whole memory image, including the old kernel. The "boot" kernel is only used as a loader until all pages are put back in place and control can be given back to the old kernel. Well, that might not exactly be what x86 does but that's definitely waht I'll do on PPC :)

Ben.

Hi!

You need to check resulting assembly for stack accesses. So yes, you can compile it from .c file, _but you have to read it_.
Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Hrm... That's awful and terribly fragile. You should either write it entirely in assembly (thus readable & commented) or write it in C with a temporary stack or whatever that makes it safe. I'll certainly not let something like that sneak into arch/ppc anyway.

Ben.

Hi!

Take a look before commenting.

Long time ago I took .c version, compiled it into assembly, handcleaned (etc) and now its used.

But if someone wants to do ppc, taking .c version, compiling it into assembly, then checking/fixing is probably easiest way...

Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]