Skip to contentSkip to conversation

swsusp: Remove arch-specific references from generic code

PATCH Pavel Machek 4 participants 11 messages Mar 16, 2005 – Mar 22, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi!

This is fix for "swsusp_restore crap"-: we had some i386-specific code referenced from generic code. This fixes it by inlining tlb_flush_all into assembly.

Please apply,
Pavel

From: Rafael J. Wysocki <[email protected]>

arch/i386/power/swsusp.S +10 -1
diff -Nrup linux-2.6.11-bk10-a/arch/i386/power/swsusp.S linux-2.6.11-bk10-b/arch/i386/power/swsusp.S
@@ -51,6 +51,15 @@ copy_loop:
 	.p2align 4,,7
 
 done:
+	/* Flush TLB, including "global" things (vmalloc) */
+	movl	mmu_cr4_features, %eax
+	movl	%eax, %edx
+	andl	$~(1<<7), %edx;  # PGE
+	movl	%edx, %cr4;  # turn off PGE
+	movl	%cr3, %ecx;  # flush TLB
+	movl	%ecx, %cr3
+	movl	%eax, %cr4;  # turn PGE back on
+
 	movl saved_context_esp, %esp
 	movl saved_context_ebp, %ebp
 	movl saved_context_ebx, %ebx
@@ -58,5 +67,5 @@ done:
 	movl saved_context_edi, %edi
 
 	pushl saved_context_eflags ; popfl
-	call swsusp_restore
+
 	ret
arch/x86_64/kernel/suspend_asm.S +10 -1
diff -Nrup linux-2.6.11-bk10-a/arch/x86_64/kernel/suspend_asm.S linux-2.6.11-bk10-b/arch/x86_64/kernel/suspend_asm.S
@@ -69,6 +69,15 @@ loop:
 	movq	pbe_next(%rdx), %rdx
 	jmp	loop
 done:
+	/* Flush TLB, including "global" things (vmalloc) */
+	movq	mmu_cr4_features(%rip), %rax
+	movq	%rax, %rdx
+	andq	$~(1<<7), %rdx;  # PGE
+	movq	%rdx, %cr4;  # turn off PGE
+	movq	%cr3, %rcx;  # flush TLB
+	movq	%rcx, %cr3
+	movq	%rax, %cr4;  # turn PGE back on
+
 	movl	$24, %eax
 	movl	%eax, %ds
 
@@ -89,5 +98,5 @@ done:
 	movq saved_context_r14(%rip), %r14
 	movq saved_context_r15(%rip), %r15
 	pushq saved_context_eflags(%rip) ; popfq
-	call	swsusp_restore
+
 	ret
kernel/power/swsusp.c +1 -10
diff -Nrup linux-2.6.11-bk10-a/kernel/power/swsusp.c linux-2.6.11-bk10-b/kernel/power/swsusp.c
@@ -900,22 +900,13 @@ int swsusp_suspend(void)
 	error = swsusp_arch_suspend();
 	/* Restore control flow magically appears here */
 	restore_processor_state();
+	BUG_ON (nr_copy_pages_check != nr_copy_pages);
 	restore_highmem();
 	device_power_up();
 	local_irq_enable();
 	return error;
 }
 
-
-asmlinkage int swsusp_restore(void)
-{
-	BUG_ON (nr_copy_pages_check != nr_copy_pages);
-	
-	/* Even mappings of "global" things (vmalloc) need to be fixed */
-	__flush_tlb_global();
-	return 0;
-}
-
 int swsusp_resume(void)
 {
 	int error;
Signature · 9 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"

----- End forwarded message -----

-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!
#1 lore raw

Hi,

Quoted 7 lines · Pavel Machek

Hi!

This is fix for "swsusp_restore crap"-: we had some i386-specific code referenced from generic code. This fixes it by inlining tlb_flush_all into assembly.

Please apply,

Unfortunately, this patch requires the following fix. Without it, swsusp will leak lots of memory on every resume. Sorry for this bug, it was really dumb.

Rafael

arch/i386/power/swsusp.S +2 -0
diff -Nrup linux-2.6.12-rc1-a/arch/i386/power/swsusp.S linux-2.6.12-rc1-b/arch/i386/power/swsusp.S
@@ -68,4 +68,6 @@ done:
 
 	pushl saved_context_eflags ; popfl
 
+	xorl	%eax, %eax
+
 	ret
arch/x86_64/kernel/suspend_asm.S +3 -1
diff -Nrup linux-2.6.12-rc1-a/arch/x86_64/kernel/suspend_asm.S linux-2.6.12-rc1-b/arch/x86_64/kernel/suspend_asm.S
@@ -83,7 +83,7 @@ done:
 
 	movq saved_context_esp(%rip), %rsp
 	movq saved_context_ebp(%rip), %rbp
-	movq saved_context_eax(%rip), %rax
+	/* Don't restore %rax, it must be 0 */
 	movq saved_context_ebx(%rip), %rbx
 	movq saved_context_ecx(%rip), %rcx
 	movq saved_context_edx(%rip), %rdx
@@ -99,4 +99,6 @@ done:
 	movq saved_context_r15(%rip), %r15
 	pushq saved_context_eflags(%rip) ; popfq
 
+	xorq	%rax, %rax
+
 	ret
Signature · 3 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"
#2 lore raw
Quoted 4 lines · Rafael J. Wysocki
Quoted 2 lines · Pavel Machek

Hi!

Quoted 8 lines
Quoted 5 lines

This is fix for "swsusp_restore crap"-: we had some i386-specific code referenced from generic code. This fixes it by inlining tlb_flush_all into assembly.

Please apply,

Unfortunately, this patch requires the following fix. Without it, swsusp will leak lots of memory on every resume. Sorry for this bug, it was really dumb.

Just fyi, the only swsusp patches I currently have queued are

swsusp-add-missing-refrigerator-calls.patch swsusp-suspend_pd_pages-fix.patch suspend-to-ram-update-videotxt-with-more-systems.patch

I've been ducking all the "swsusp_restore crap" patches. Pavel, could you please aggregate, test and resend everything when the dust has settled?

#3 lore raw

Hi!

Quoted 1 line

swsusp-suspend_pd_pages-fix.patch

Could you drop this one? It is "fixing" unused macro, we don't want it going anywhere.

Quoted 4 lines

suspend-to-ram-update-videotxt-with-more-systems.patch

I've been ducking all the "swsusp_restore crap" patches. Pavel, could you please aggregate, test and resend everything when the dust has settled?

Ok.

Pavel

Signature · 2 lines
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!
#4 lore raw
In reply to Andrew Morton

Hi!

Do you think you could just send me diff between 2.6.12-rc1 and your tree? I'll merge it here.
Pavel

Signature · 2 lines
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!
#5 lore raw

Hi,

Quoted 4 lines · Pavel Machek

Hi!

Do you think you could just send me diff between 2.6.12-rc1 and your tree? I'll merge it here.

Sure, no problem, the diff follows. :-) It contains the following changes:

- remove swsusp_restore() (with the fix to return 0 from swsusp_arch_resume() on x86*)
- drop SUSPEND_PD_PAGES and pagedir_order
- fix possible memory leaks in swsusp_suspend()

The original patches are also attached in case you need them (they all apply to 2.6.12-rc1).

Please let me know if that's ok.

Greets,
Rafael

arch/i386/power/swsusp.S +12 -1
diff -Nrup linux-2.6.12-rc1/arch/i386/power/swsusp.S linux-2.6.12-rc1-new/arch/i386/power/swsusp.S
@@ -51,6 +51,15 @@ copy_loop:
 	.p2align 4,,7
 
 done:
+	/* Flush TLB, including "global" things (vmalloc) */
+	movl	mmu_cr4_features, %eax
+	movl	%eax, %edx
+	andl	$~(1<<7), %edx;  # PGE
+	movl	%edx, %cr4;  # turn off PGE
+	movl	%cr3, %ecx;  # flush TLB
+	movl	%ecx, %cr3
+	movl	%eax, %cr4;  # turn PGE back on
+
 	movl saved_context_esp, %esp
 	movl saved_context_ebp, %ebp
 	movl saved_context_ebx, %ebx
@@ -58,5 +67,7 @@ done:
 	movl saved_context_edi, %edi
 
 	pushl saved_context_eflags ; popfl
-	call swsusp_restore
+
+	xorl	%eax, %eax
+
 	ret
arch/x86_64/kernel/suspend_asm.S +13 -2
diff -Nrup linux-2.6.12-rc1/arch/x86_64/kernel/suspend_asm.S linux-2.6.12-rc1-new/arch/x86_64/kernel/suspend_asm.S
@@ -69,12 +69,21 @@ loop:
 	movq	pbe_next(%rdx), %rdx
 	jmp	loop
 done:
+	/* Flush TLB, including "global" things (vmalloc) */
+	movq	mmu_cr4_features(%rip), %rax
+	movq	%rax, %rdx
+	andq	$~(1<<7), %rdx;  # PGE
+	movq	%rdx, %cr4;  # turn off PGE
+	movq	%cr3, %rcx;  # flush TLB
+	movq	%rcx, %cr3
+	movq	%rax, %cr4;  # turn PGE back on
+
 	movl	$24, %eax
 	movl	%eax, %ds
 
 	movq saved_context_esp(%rip), %rsp
 	movq saved_context_ebp(%rip), %rbp
-	movq saved_context_eax(%rip), %rax
+	/* Don't restore %rax, it must be 0 anyway */
 	movq saved_context_ebx(%rip), %rbx
 	movq saved_context_ecx(%rip), %rcx
 	movq saved_context_edx(%rip), %rdx
@@ -89,5 +98,7 @@ done:
 	movq saved_context_r14(%rip), %r14
 	movq saved_context_r15(%rip), %r15
 	pushq saved_context_eflags(%rip) ; popfq
-	call	swsusp_restore
+
+	xorq	%rax, %rax
+
 	ret
include/linux/suspend.h +0 -2
diff -Nrup linux-2.6.12-rc1/include/linux/suspend.h linux-2.6.12-rc1-new/include/linux/suspend.h
@@ -34,8 +34,6 @@ typedef struct pbe {
 #define SWAP_FILENAME_MAXLENGTH	32
 
 
-#define SUSPEND_PD_PAGES(x)     (((x)*sizeof(struct pbe))/PAGE_SIZE+1)
-
 extern dev_t swsusp_resume_device;
    
 /* mm/vmscan.c */
kernel/power/swsusp.c +4 -13
diff -Nrup linux-2.6.12-rc1/kernel/power/swsusp.c linux-2.6.12-rc1-new/kernel/power/swsusp.c
@@ -98,7 +98,6 @@ unsigned int nr_copy_pages __nosavedata 
  */
 suspend_pagedir_t *pagedir_nosave __nosavedata = NULL;
 static suspend_pagedir_t *pagedir_save;
-static int pagedir_order __nosavedata = 0;
 
 #define SWSUSP_SIG	"S1SUSPEND"
 
@@ -894,28 +893,21 @@ int swsusp_suspend(void)
 	 */
 	if ((error = device_power_down(PMSG_FREEZE))) {
 		local_irq_enable();
+		swsusp_free();
 		return error;
 	}
 	save_processor_state();
-	error = swsusp_arch_suspend();
+	if ((error = swsusp_arch_suspend()))
+		swsusp_free();
 	/* Restore control flow magically appears here */
 	restore_processor_state();
+	BUG_ON (nr_copy_pages_check != nr_copy_pages);
 	restore_highmem();
 	device_power_up();
 	local_irq_enable();
 	return error;
 }
 
-
-asmlinkage int swsusp_restore(void)
-{
-	BUG_ON (nr_copy_pages_check != nr_copy_pages);
-	
-	/* Even mappings of "global" things (vmalloc) need to be fixed */
-	__flush_tlb_global();
-	return 0;
-}
-
 int swsusp_resume(void)
 {
 	int error;
@@ -1219,7 +1211,6 @@ static int check_header(void)
 		return -EPERM;
 	}
 	nr_copy_pages = swsusp_info.image_pages;
-	pagedir_order = get_bitmask_order(SUSPEND_PD_PAGES(nr_copy_pages));
 	return error;
 }
 
Signature · 3 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"
arch/i386/power/swsusp.S +12 -1
diff -Nrup linux-2.6.12-rc1/arch/i386/power/swsusp.S linux-2.6.12-rc1-new/arch/i386/power/swsusp.S
@@ -51,6 +51,15 @@ copy_loop:
 	.p2align 4,,7
 
 done:
+	/* Flush TLB, including "global" things (vmalloc) */
+	movl	mmu_cr4_features, %eax
+	movl	%eax, %edx
+	andl	$~(1<<7), %edx;  # PGE
+	movl	%edx, %cr4;  # turn off PGE
+	movl	%cr3, %ecx;  # flush TLB
+	movl	%ecx, %cr3
+	movl	%eax, %cr4;  # turn PGE back on
+
 	movl saved_context_esp, %esp
 	movl saved_context_ebp, %ebp
 	movl saved_context_ebx, %ebx
@@ -58,5 +67,7 @@ done:
 	movl saved_context_edi, %edi
 
 	pushl saved_context_eflags ; popfl
-	call swsusp_restore
+
+	xorl	%eax, %eax
+
 	ret
arch/x86_64/kernel/suspend_asm.S +13 -2
diff -Nrup linux-2.6.12-rc1/arch/x86_64/kernel/suspend_asm.S linux-2.6.12-rc1-new/arch/x86_64/kernel/suspend_asm.S
@@ -69,12 +69,21 @@ loop:
 	movq	pbe_next(%rdx), %rdx
 	jmp	loop
 done:
+	/* Flush TLB, including "global" things (vmalloc) */
+	movq	mmu_cr4_features(%rip), %rax
+	movq	%rax, %rdx
+	andq	$~(1<<7), %rdx;  # PGE
+	movq	%rdx, %cr4;  # turn off PGE
+	movq	%cr3, %rcx;  # flush TLB
+	movq	%rcx, %cr3
+	movq	%rax, %cr4;  # turn PGE back on
+
 	movl	$24, %eax
 	movl	%eax, %ds
 
 	movq saved_context_esp(%rip), %rsp
 	movq saved_context_ebp(%rip), %rbp
-	movq saved_context_eax(%rip), %rax
+	/* Don't restore %rax, it must be 0 anyway */
 	movq saved_context_ebx(%rip), %rbx
 	movq saved_context_ecx(%rip), %rcx
 	movq saved_context_edx(%rip), %rdx
@@ -89,5 +98,7 @@ done:
 	movq saved_context_r14(%rip), %r14
 	movq saved_context_r15(%rip), %r15
 	pushq saved_context_eflags(%rip) ; popfq
-	call	swsusp_restore
+
+	xorq	%rax, %rax
+
 	ret
kernel/power/swsusp.c +1 -10
diff -Nrup linux-2.6.12-rc1/kernel/power/swsusp.c linux-2.6.12-rc1-new/kernel/power/swsusp.c
@@ -900,22 +900,13 @@ int swsusp_suspend(void)
 	error = swsusp_arch_suspend();
 	/* Restore control flow magically appears here */
 	restore_processor_state();
+	BUG_ON (nr_copy_pages_check != nr_copy_pages);
 	restore_highmem();
 	device_power_up();
 	local_irq_enable();
 	return error;
 }
 
-
-asmlinkage int swsusp_restore(void)
-{
-	BUG_ON (nr_copy_pages_check != nr_copy_pages);
-	
-	/* Even mappings of "global" things (vmalloc) need to be fixed */
-	__flush_tlb_global();
-	return 0;
-}
-
 int swsusp_resume(void)
 {
 	int error;
include/linux/suspend.h +0 -2
diff -Nrup linux-2.6.12-rc1/include/linux/suspend.h linux-2.6.12-rc1-a/include/linux/suspend.h
@@ -34,8 +34,6 @@ typedef struct pbe {
 #define SWAP_FILENAME_MAXLENGTH	32
 
 
-#define SUSPEND_PD_PAGES(x)     (((x)*sizeof(struct pbe))/PAGE_SIZE+1)
-
 extern dev_t swsusp_resume_device;
    
 /* mm/vmscan.c */
kernel/power/swsusp.c +0 -2
diff -Nrup linux-2.6.12-rc1/kernel/power/swsusp.c linux-2.6.12-rc1-a/kernel/power/swsusp.c
@@ -98,7 +98,6 @@ unsigned int nr_copy_pages __nosavedata 
  */
 suspend_pagedir_t *pagedir_nosave __nosavedata = NULL;
 static suspend_pagedir_t *pagedir_save;
-static int pagedir_order __nosavedata = 0;
 
 #define SWSUSP_SIG	"S1SUSPEND"
 
@@ -1219,7 +1218,6 @@ static int check_header(void)
 		return -EPERM;
 	}
 	nr_copy_pages = swsusp_info.image_pages;
-	pagedir_order = get_bitmask_order(SUSPEND_PD_PAGES(nr_copy_pages));
 	return error;
 }
 
kernel/power/swsusp.c +3 -1
@@ -894,10 +894,12 @@ int swsusp_suspend(void)
 	 */
 	if ((error = device_power_down(PMSG_FREEZE))) {
 		local_irq_enable();
+		swsusp_free();
 		return error;
 	}
 	save_processor_state();
-	error = swsusp_arch_suspend();
+	if ((error = swsusp_arch_suspend()))
+		swsusp_free();
 	/* Restore control flow magically appears here */
 	restore_processor_state();
 	BUG_ON (nr_copy_pages_check != nr_copy_pages);
#6 lore raw

swsusp smp problems... [was Re: swsusp: Remove arch-specific references from generic code]

Hi!

At least part of them is caused by CONFIG_MTRR. I had to disable it on i386 to make it work...
Pavel

Signature · 2 lines
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!
#7 lore raw

Re: swsusp smp problems... [was Re: swsusp: Remove arch-specific references from generic code]

Hi,

Quoted 4 lines · Pavel Machek

Hi!

At least part of them is caused by CONFIG_MTRR. I had to disable it on i386 to make it work...

Later today I'll check if that helps on x86-64.

Anyway in the meantime I have played a bit with the CPU hotplug code. It needs some work, but looks promising. I've changed disable_nonboot_cpus() to use the CPU hotplug code and it seems to work. Well, almost, because some traces of the second CPU remain in the kernel, as some things do not work properly (eg flush_tlb_others() is called with a mask that triggers a BUG() in it etc.). This should not be difficult to get fixed, however. Strangely enough, the processes still fail to freeze after the second CPU has been disabled (specifically one of them, which is "syslogd"). I'm going to investigate this more thoroughly.

Turning the second CPU back on does not work for me, but in fact I haven't looked at it so far.

Greets,
Rafael

Signature · 3 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"
#8 lore raw

Re: swsusp smp problems... [was Re: swsusp: Remove arch-specific references from generic code]

Hi!

Quoted 15 lines
Quoted 2 lines

At least part of them is caused by CONFIG_MTRR. I had to disable it on i386 to make it work...

Later today I'll check if that helps on x86-64.

Anyway in the meantime I have played a bit with the CPU hotplug code. It needs some work, but looks promising. I've changed disable_nonboot_cpus() to use the CPU hotplug code and it seems to work. Well, almost, because some traces of the second CPU remain in the kernel, as some things do not work properly (eg flush_tlb_others() is called with a mask that triggers a BUG() in it etc.). This should not be difficult to get fixed, however. Strangely enough, the processes still fail to freeze after the second CPU has been disabled (specifically one of them, which is "syslogd"). I'm going to investigate this more thoroughly.

Quoted 2 lines

Turning the second CPU back on does not work for me, but in fact I haven't looked at it so far.

Can youm mail me (and probably l-k) the latest diffs? I started playing with it, too... (remember that scrap-metal machine?).

Pavel

Signature · 2 lines
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!
#9 lore raw

Re: swsusp smp problems... [was Re: swsusp: Remove arch-specific references from generic code]

Hi,

Sorry for the delay, I had an urgent work to do yestarday ...

Quoted 6 lines · Pavel Machek

Hi!

Quoted 4 lines
Quoted 2 lines

At least part of them is caused by CONFIG_MTRR. I had to disable it on i386 to make it work...

Later today I'll check if that helps on x86-64.

On vanilla 2.6.11-mm4 processes freeze successfully when CONFIG_MTRR is disabled, but it doesn't help if the CPU hotplug code is used before freezing the processes.

Quoted 10 lines
Quoted 10 lines

Anyway in the meantime I have played a bit with the CPU hotplug code. It needs some work, but looks promising. I've changed disable_nonboot_cpus() to use the CPU hotplug code and it seems to work. Well, almost, because some traces of the second CPU remain in the kernel, as some things do not work properly (eg flush_tlb_others() is called with a mask that triggers a BUG() in it etc.). This should not be difficult to get fixed, however. Strangely enough, the processes still fail to freeze after the second CPU has been disabled (specifically one of them, which is "syslogd"). I'm going to investigate this more thoroughly.

Quoted 5 lines
Quoted 2 lines

Turning the second CPU back on does not work for me, but in fact I haven't looked at it so far.

Can youm mail me (and probably l-k) the latest diffs? I started playing with it, too... (remember that scrap-metal machine?).

All right, the current diff between 2.6.11-mm4 and my development tree on the SMP box is attached. It's quite big, as it contains the CPU hotplug code that I've dragged to the x86-64 tree. There's a lot of debug stuff (probably bugs too) in it and it doesn't let the box actually suspend. Also, it doesn't enable the non-boot CPUs after they've been disabled.

Greets,
Rafael

Signature · 3 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"
arch/x86_64/Kconfig +10 -0
diff -Nrup linux-2.6.11-mm4/arch/x86_64/Kconfig linux-2.6.11-mm4-new/arch/x86_64/Kconfig
@@ -451,6 +451,16 @@ config UNORDERED_IO
 
 source "drivers/pci/Kconfig"
 
+config HOTPLUG_CPU
+	bool "Support for hot-pluggable CPUs (EXPERIMENTAL)"
+	depends on SMP && HOTPLUG && EXPERIMENTAL
+	---help---
+	  Say Y here to experiment with turning CPUs off and on
+	  or if you want to use software suspend (swsusp) with SMP.  CPUs
+	  can be controlled through /sys/devices/system/cpu.
+
+	  Say N.
+
 source "drivers/pcmcia/Kconfig"
 
 source "drivers/pci/hotplug/Kconfig"
arch/x86_64/kernel/apic.c +1 -1
diff -Nrup linux-2.6.11-mm4/arch/x86_64/kernel/apic.c linux-2.6.11-mm4-new/arch/x86_64/kernel/apic.c
@@ -816,7 +816,7 @@ void __init setup_secondary_APIC_clock(v
 	local_irq_enable();
 }
 
-void __init disable_APIC_timer(void)
+void disable_APIC_timer(void)
 {
 	if (using_apic_timer) {
 		unsigned long v;
arch/x86_64/kernel/irq.c +41 -0

50 lines not on this page · raw message

arch/x86_64/kernel/process.c +33 -0
diff -Nrup linux-2.6.11-mm4/arch/x86_64/kernel/process.c linux-2.6.11-mm4-new/arch/x86_64/kernel/process.c
@@ -49,6 +49,7 @@
 #include <asm/desc.h>
 #include <asm/proto.h>
 #include <asm/ia32.h>
+#include <asm/cpu.h>
 
 asmlinkage extern void ret_from_fork(void);
 
@@ -154,6 +155,34 @@ void cpu_idle_wait(void)
 }
 EXPORT_SYMBOL_GPL(cpu_idle_wait);
 
+#ifdef CONFIG_HOTPLUG_CPU
+#include <asm/nmi.h>
+/* We don't actually take CPU down, just spin without interrupts. */
+static inline void play_dead(void)
+{
+	/* Ack it */
+	__get_cpu_var(cpu_state) = CPU_DEAD;
+
+	/* We shouldn't have to disable interrupts while dead, but
+	* some interrupts just don't seem to go away, and this makes
+	* it "work" for testing purposes. */
+	/* Death loop */
+	while (__get_cpu_var(cpu_state) != CPU_UP_PREPARE)
+		cpu_relax();
+
+	local_irq_disable();
+	__flush_tlb_all();
+	cpu_set(smp_processor_id(), cpu_online_map);
+	enable_APIC_timer();
+	local_irq_enable();
+}
+#else
+static inline void play_dead(void)
+{
+	BUG();
+}
+#endif /* CONFIG_HOTPLUG_CPU */
+
 /*
  * The idle thread. There's no useful work to be
  * done, so just try to conserve power and have a
@@ -174,6 +203,10 @@ void cpu_idle (void)
 			idle = pm_idle;
 			if (!idle)
 				idle = default_idle;
+
+			if (cpu_is_offline(smp_processor_id()))
+				play_dead();
+
 			idle();
 		}
 
arch/x86_64/kernel/smpboot.c +87 -0

106 lines not on this page · raw message

arch/x86_64/kernel/traps.c +6 -0
diff -Nrup linux-2.6.11-mm4/arch/x86_64/kernel/traps.c linux-2.6.11-mm4-new/arch/x86_64/kernel/traps.c
@@ -590,6 +590,12 @@ asmlinkage void default_do_nmi(struct pt
 {
 	unsigned char reason = 0;
 
+#ifdef CONFIG_HOTPLUG_CPU
+	/* Ignore offline CPUs */
+	if (!cpu_online(smp_processor_id()))
+		return;
+#endif
+
 	/* Only the BSP gets external NMIs from the system.  */
 	if (!smp_processor_id())
 		reason = get_nmi_reason();
drivers/serial/serial_core.c +6 -0
diff -Nrup linux-2.6.11-mm4/drivers/serial/serial_core.c linux-2.6.11-mm4-new/drivers/serial/serial_core.c
@@ -1831,6 +1831,9 @@ int uart_suspend_port(struct uart_driver
 {
 	struct uart_state *state = drv->state + port->line;
 
+	if (uart_console(port))
+		return 0;
+
 	down(&state->sem);
 
 	if (state->info && state->info->flags & UIF_INITIALIZED) {
@@ -1869,6 +1872,9 @@ int uart_resume_port(struct uart_driver 
 {
 	struct uart_state *state = drv->state + port->line;
 
+	if (uart_console(port))
+		return 0;
+
 	down(&state->sem);
 
 	uart_change_pm(state, 0);
include/asm-x86_64/irq.h +7 -0
diff -Nrup linux-2.6.11-mm4/include/asm-x86_64/irq.h linux-2.6.11-mm4-new/include/asm-x86_64/irq.h
@@ -10,6 +10,9 @@
  *	<[email protected]>
  */
 
+#include <linux/config.h>
+#include <linux/sched.h>
+
 #define TIMER_IRQ 0
 
 /*
@@ -52,4 +55,8 @@ struct irqaction;
 struct pt_regs;
 int handle_IRQ_event(unsigned int, struct pt_regs *, struct irqaction *);
 
+#ifdef CONFIG_HOTPLUG_CPU
+extern void fixup_irqs(cpumask_t map);
+#endif
+
 #endif /* _ASM_IRQ_H */
include/asm-x86_64/smp.h +3 -0
diff -Nrup linux-2.6.11-mm4/include/asm-x86_64/smp.h linux-2.6.11-mm4-new/include/asm-x86_64/smp.h
@@ -145,6 +145,9 @@ static __inline int logical_smp_processo
 	/* we don't want to mark this access volatile - bad code generation */
 	return GET_APIC_LOGICAL_ID(*(unsigned long *)(APIC_BASE+APIC_LDR));
 }
+
+extern int __cpu_disable(void);
+extern void __cpu_die(unsigned int cpu);
 #endif
 
 #endif
include/linux/suspend.h +2 -2
diff -Nrup linux-2.6.11-mm4/include/linux/suspend.h linux-2.6.11-mm4-new/include/linux/suspend.h
@@ -61,10 +61,10 @@ static inline int software_suspend(void)
 #endif
 
 #ifdef CONFIG_SMP
-extern void disable_nonboot_cpus(void);
+extern int disable_nonboot_cpus(void);
 extern void enable_nonboot_cpus(void);
 #else
-static inline void disable_nonboot_cpus(void) {}
+static inline int disable_nonboot_cpus(void) { return 0; }
 static inline void enable_nonboot_cpus(void) {}
 #endif
 
kernel/power/disk.c +15 -10
diff -Nrup linux-2.6.11-mm4/kernel/power/disk.c linux-2.6.11-mm4-new/kernel/power/disk.c
@@ -117,8 +117,8 @@ static void finish(void)
 {
 	device_resume();
 	platform_finish();
-	enable_nonboot_cpus();
 	thaw_processes();
+	enable_nonboot_cpus();
 	pm_restore_console();
 }
 
@@ -127,8 +127,6 @@ static int prepare_processes(void)
 {
 	int error;
 
-	pm_prepare_console();
-
 	sys_sync();
 
 	if (freeze_processes()) {
@@ -151,8 +149,8 @@ static int prepare_processes(void)
 
 static void unprepare_processes(void)
 {
-	enable_nonboot_cpus();
 	thaw_processes();
+	enable_nonboot_cpus();
 	pm_restore_console();
 }
 
@@ -160,11 +158,9 @@ static int prepare_devices(void)
 {
 	int error;
 
-	disable_nonboot_cpus();
 	if ((error = device_suspend(PMSG_FREEZE))) {
 		printk("Some devices failed to suspend\n");
 		platform_finish();
-		enable_nonboot_cpus();
 		return error;
 	}
 
@@ -184,13 +180,22 @@ int pm_suspend_disk(void)
 {
 	int error;
 
+	pm_prepare_console();
+
+	if ((error = disable_nonboot_cpus())) {
+		/*enable_nonboot_cpus();*/
+		pm_restore_console();
+		return error;
+	}
 	error = prepare_processes();
-	if (!error) {
+	error = -EFAULT;
+	ssleep(5);
+	if (!error)
 		error = prepare_devices();
-	}
-
 	if (error) {
-		unprepare_processes();
+		thaw_processes();
+		/*enable_nonboot_cpus();*/
+		pm_restore_console();
 		return error;
 	}
 
kernel/power/Kconfig +1 -1
diff -Nrup linux-2.6.11-mm4/kernel/power/Kconfig linux-2.6.11-mm4-new/kernel/power/Kconfig
@@ -28,7 +28,7 @@ config PM_DEBUG
 
 config SOFTWARE_SUSPEND
 	bool "Software Suspend (EXPERIMENTAL)"
-	depends on EXPERIMENTAL && PM && SWAP
+	depends on EXPERIMENTAL && PM && SWAP && (HOTPLUG_CPU || !SMP)
 	---help---
 	  Enable the possibility of suspending the machine.
 	  It doesn't need APM.
kernel/power/smp.c +32 -59

111 lines not on this page · raw message

kernel/stop_machine.c +3 -0
diff -Nrup linux-2.6.11-mm4/kernel/stop_machine.c linux-2.6.11-mm4-new/kernel/stop_machine.c
@@ -175,10 +175,13 @@ struct task_struct *__stop_machine_run(i
 
 	down(&stopmachine_mutex);
 
+	printk("__stop_machine_run(): cpu = %d\n", cpu);
 	/* If they don't care which CPU fn runs on, bind to any online one. */
 	if (cpu == NR_CPUS)
 		cpu = _smp_processor_id();
 
+	printk("__stop_machine_run(): Running on CPU %d\n", _smp_processor_id()); /*RJW*/
+	printk("__stop_machine_run(): Destination CPU %d\n", cpu); /*RJW*/
 	p = kthread_create(do_stop, &smdata, "kstopmachine");
 	if (!IS_ERR(p)) {
 		kthread_bind(p, cpu);
author Applied
#10 lore raw
In reply to Rafael J. Wysocki

Hi!

Quoted 5 lines
Quoted 2 lines

Do you think you could just send me diff between 2.6.12-rc1 and your tree? I'll merge it here.

Sure, no problem, the diff follows. :-) It contains the following changes:

Quoted 8 lines

- remove swsusp_restore() (with the fix to return 0 from swsusp_arch_resume() on x86*)
- drop SUSPEND_PD_PAGES and pagedir_order
- fix possible memory leaks in swsusp_suspend()

The original patches are also attached in case you need them (they all apply to 2.6.12-rc1).

Please let me know if that's ok.

Thanks, applied to my tree. (Actually I applied 3 attachments, but that should be okay).
Pavel

Signature · 2 lines
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!