Skip to contentSkip to conversation

amd64-agp vs. swsusp

DISCUSSION Michal Schmidt 6 participants 20 messages Jul 19, 2005 – Aug 7, 2005

lore
Share & tools
lore
#0 lore raw

Hello,

Does resuming from swsuspend work for anyone with amd64-agp loaded?

On my system when I suspend with amd64-agp loaded, I get a spontaneous reboot on resume. It reboots immediately after reading the saved image from disk. This is 100% reproducible.

Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.

Regards,
Michal

#1 lore raw
Quoted 10 lines · Michal Schmidt

Hello,

Does resuming from swsuspend work for anyone with amd64-agp loaded?

On my system when I suspend with amd64-agp loaded, I get a spontaneous reboot on resume. It reboots immediately after reading the saved image from disk.
This is 100% reproducible.

Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.

AMD Athlon(tm) 64 Processor 3000+, Acer Aspire

Linux gringo 2.6.13-rc3-gringo #36 Sun Jul 17 15:57:17 CEST 2005 x86_64 unknown unknown GNU/Linux

CONFIG_AGP=y
CONFIG_AGP_AMD64=y

swsusp works for me. Could it be mm, agp as a module or some speciality of your hardware?

Signature · 1 line
Andreas Steinmetz                       SPAMmers use [email protected]
#2 lore raw
Quoted 13 lines · Andreas Steinmetz
Quoted 8 lines · Michal Schmidt

Does resuming from swsuspend work for anyone with amd64-agp loaded?

On my system when I suspend with amd64-agp loaded, I get a spontaneous reboot on resume. It reboots immediately after reading the saved image from disk.
This is 100% reproducible.

Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.

AMD Athlon(tm) 64 Processor 3000+, Acer Aspire

Quoted 7 lines

Linux gringo 2.6.13-rc3-gringo #36 Sun Jul 17 15:57:17 CEST 2005 x86_64 unknown unknown GNU/Linux

CONFIG_AGP=y
CONFIG_AGP_AMD64=y

swsusp works for me. Could it be mm, agp as a module or some speciality

^^^^^^^^^^^^^^^
That seems to be the problem!

Quoted 1 line

of your hardware?

I have rebuilt agpgart and amd64-agp into the kernel and now it has resumed successfully for the first time. Thank you for the hint!

But I still wonder, why that makes a difference.

Michal

#3 lore raw

Hi,

Quoted 21 lines · Michal Schmidt
Quoted 13 lines · Andreas Steinmetz
Quoted 8 lines · Michal Schmidt

Does resuming from swsuspend work for anyone with amd64-agp loaded?

On my system when I suspend with amd64-agp loaded, I get a spontaneous reboot on resume. It reboots immediately after reading the saved image from disk.
This is 100% reproducible.

Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.

AMD Athlon(tm) 64 Processor 3000+, Acer Aspire

Quoted 7 lines

Linux gringo 2.6.13-rc3-gringo #36 Sun Jul 17 15:57:17 CEST 2005 x86_64 unknown unknown GNU/Linux

CONFIG_AGP=y
CONFIG_AGP_AMD64=y

swsusp works for me. Could it be mm, agp as a module or some speciality

Quoted 8 lines

^^^^^^^^^^^^^^^
That seems to be the problem!

Quoted 1 line

of your hardware?

I have rebuilt agpgart and amd64-agp into the kernel and now it has resumed successfully for the first time. Thank you for the hint!

But I still wonder, why that makes a difference.

Before resume the module is not present. When it gets loaded from the image it probably runs with the assumption that the hardware was initialized which is not correct.

Greets,
Rafael

Signature · 3 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"
#4 lore raw
Quoted 9 lines · Rafael J. Wysocki
Quoted 3 lines · Michal Schmidt

I have rebuilt agpgart and amd64-agp into the kernel and now it has resumed successfully for the first time. Thank you for the hint!

But I still wonder, why that makes a difference.

Before resume the module is not present. When it gets loaded from the image it probably runs with the assumption that the hardware was initialized which is not correct.

It seems that the module doesn't even get a chance to run after resume. I've put some printks and udelays into kernel/power/swsusp.c and other places and I've found that the spontaneous reset occurs already in swsusp_arch_resume(), ie. before the drivers get their resume methods called. This is what I have in swsusp_suspend() now:

	...
        save_processor_state();
        if ((error = swsusp_arch_suspend()))
                printk(KERN_ERR "Error %d suspending\n", error);
        /* Restore control flow magically appears here */
        restore_processor_state();
        printk(KERN_INFO "processor state restored!\n");/*I added this*/
        BUG_ON (nr_copy_pages_check != nr_copy_pages);
        restore_highmem();
        device_power_up();
	...

I'm recording the screen during resuming with a digital camera to see if the added printk is displayed before the reset and I am now sure that the reset occurs before that. The last thing I see is:

Stopping tasks: --|
Freeing memory... done (0 pages freed)
swsusp: Need to copy 8121 pages

Then on the next frame of the recorded MPEG, the display is already beginning to dim as the computer is resetting.

I also tried putting a printk before restore_processor_state(), but I'm not sure if it is safe to use printk there.
So I tried putting a loop of 5000 x udelay(1000) there to see if the reset would be delayed by 5s. It was not delayed, so I think that the reset occurs before restore_processor_state().

Michal

#5 lore raw
Quoted 29 lines · Michal Schmidt
Quoted 10 lines · Rafael J. Wysocki
Quoted 4 lines · Michal Schmidt

I have rebuilt agpgart and amd64-agp into the kernel and now it has resumed successfully for the first time. Thank you for the hint!

But I still wonder, why that makes a difference.

Before resume the module is not present. When it gets loaded from the image it probably runs with the assumption that the hardware was initialized which is not correct.

It seems that the module doesn't even get a chance to run after resume. I've put some printks and udelays into kernel/power/swsusp.c and other places and I've found that the spontaneous reset occurs already in swsusp_arch_resume(), ie. before the drivers get their resume methods called. This is what I have in swsusp_suspend() now:

	...
         save_processor_state();
         if ((error = swsusp_arch_suspend()))
                 printk(KERN_ERR "Error %d suspending\n", error);
         /* Restore control flow magically appears here */
         restore_processor_state();
         printk(KERN_INFO "processor state restored!\n");/*I added this*/
         BUG_ON (nr_copy_pages_check != nr_copy_pages);
         restore_highmem();
         device_power_up();
	...
Quoted 7 lines

I'm recording the screen during resuming with a digital camera to see if the added printk is displayed before the reset and I am now sure that the reset occurs before that. The last thing I see is:

Stopping tasks: --|
Freeing memory... done (0 pages freed)
swsusp: Need to copy 8121 pages

Please note that printk() only places the string in a buffer and it does not get actually printed before it can be displayed which is probably after your screen is set up (including the AGP resume, I'd guess). :-)

You may be able to get more from eg. the serial console, but this requires some tampering with the serial code, AFAIR.

Quoted 5 lines

Then on the next frame of the recorded MPEG, the display is already beginning to dim as the computer is resetting.

I also tried putting a printk before restore_processor_state(), but I'm not sure if it is safe to use printk there.

Yes, it is, but you may be unable to see the message if the box reboots before it can be displayed.

Greets,
Rafael

Signature · 3 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"
#6 lore raw
Quoted 6 lines · Rafael J. Wysocki
Quoted 1 line · Michal Schmidt

I also tried putting a printk before restore_processor_state(), but I'm not sure if it is safe to use printk there.

Yes, it is, but you may be unable to see the message if the box reboots before it can be displayed.

OK, but then I also tried putting a 5s long busy wait there and the reset was not delayed. Therefore, the reset must be occurring before restore_processor_state(). Or is there a reason why

	for(i=0; i<5000; i++)
		udelay(1000);

wouldn't work as expected?

Michal

#8 lore raw
In reply to Michal Schmidt

Hi!

Quoted 36 lines
Quoted 10 lines
Quoted 4 lines

I have rebuilt agpgart and amd64-agp into the kernel and now it has resumed successfully for the first time. Thank you for the hint!

But I still wonder, why that makes a difference.

Before resume the module is not present. When it gets loaded from the image it probably runs with the assumption that the hardware was initialized
which is not correct.

It seems that the module doesn't even get a chance to run after resume. I've put some printks and udelays into kernel/power/swsusp.c and other places and I've found that the spontaneous reset occurs already in swsusp_arch_resume(), ie. before the drivers get their resume methods called. This is what I have in swsusp_suspend() now:

	...
        save_processor_state();
        if ((error = swsusp_arch_suspend()))
                printk(KERN_ERR "Error %d suspending\n", error);
        /* Restore control flow magically appears here */
        restore_processor_state();
        printk(KERN_INFO "processor state restored!\n");/*I added this*/
        BUG_ON (nr_copy_pages_check != nr_copy_pages);
        restore_highmem();
        device_power_up();
	...

I'm recording the screen during resuming with a digital camera to see if the added printk is displayed before the reset and I am now sure that the reset occurs before that. The last thing I see is:

Stopping tasks: --|
Freeing memory... done (0 pages freed)
swsusp: Need to copy 8121 pages

Quoted 8 lines

Then on the next frame of the recorded MPEG, the display is already beginning to dim as the computer is resetting.

I also tried putting a printk before restore_processor_state(), but I'm not sure if it is safe to use printk there.
So I tried putting a loop of 5000 x udelay(1000) there to see if the reset would be delayed by 5s. It was not delayed, so I think that the reset occurs before restore_processor_state().

Long time ago there were i386 problems because we assumed that kernel is mapped in one big mapping and agp broke that assumption. Copying pages backwards "fixed" it (and then we done proper fix). It should not be, but it seems similar to this problem....

Pavel

Signature · 1 line
Boycott Kodak -- for their patent abuse against Java.
#7 lore raw
In reply to Michal Schmidt
Quoted 9 lines · Michal Schmidt
Quoted 7 lines · Rafael J. Wysocki
Quoted 2 lines · Michal Schmidt

I also tried putting a printk before restore_processor_state(), but I'm not sure if it is safe to use printk there.

Yes, it is, but you may be unable to see the message if the box reboots before it can be displayed.

Quoted 7 lines

OK, but then I also tried putting a 5s long busy wait there and the reset was not delayed. Therefore, the reset must be occurring before restore_processor_state().
Or is there a reason why

	for(i=0; i<5000; i++)
		udelay(1000);

wouldn't work as expected?

No, it should work, AFAIK. Then it looks like something gets copied into a wrong place or in a wrong order while restoring the image.

Greets,
Rafael

Signature · 3 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"
#9 lore raw
In reply to Pavel Machek
Quoted 4 lines · Pavel Machek

Long time ago there were i386 problems because we assumed that kernel is mapped in one big mapping and agp broke that assumption. Copying pages backwards "fixed" it (and then we done proper fix). It should not be, but it seems similar to this problem....

Do you mean this patch of yours?:
http://www.ussg.iu.edu/hypermail/linux/kernel/0404.3/0640.html

I'm trying to do something similar for x86_64. See the attached patch.
Unfortunately, it doesn't help. The behaviour seems unchanged (resume still works iff amd64-agp wasn't loaded before suspend).

Michal

arch/x86_64/kernel/suspend_asm.S +1 -1
diff -Nurp -X dontdiff.new linux-mm/arch/x86_64/kernel/suspend_asm.S linux-mm.mich/arch/x86_64/kernel/suspend_asm.S
@@ -41,7 +41,7 @@ ENTRY(swsusp_arch_suspend)
 
 ENTRY(swsusp_arch_resume)
 	/* set up cr3 */	
-	leaq	init_level4_pgt(%rip),%rax
+	leaq	swsusp_level4_pgt(%rip),%rax
 	subq	$__START_KERNEL_map,%rax
 	movq	%rax,%cr3
 
arch/x86_64/mm/init.c +23 -1
diff -Nurp -X dontdiff.new linux-mm/arch/x86_64/mm/init.c linux-mm.mich/arch/x86_64/mm/init.c
@@ -310,10 +310,32 @@ void __init init_memory_mapping(unsigned
 
 extern struct x8664_pda cpu_pda[NR_CPUS];
 
+#ifdef CONFIG_SOFTWARE_SUSPEND
+/*
+ * Swap suspend & friends need this for resume because things like the intel-agp
+ * driver might have split up a kernel 4MB mapping.
+ */
+char __nosavedata swsusp_level4_pgt[PAGE_SIZE]
+	__attribute__ ((aligned (PAGE_SIZE)));
+
+static inline void save_pg_dir(void)
+{
+	memcpy(swsusp_level4_pgt, init_level4_pgt, PAGE_SIZE);
+}
+#else
+static inline void save_pg_dir(void)
+{
+}
+#endif
+
 /* Assumes all CPUs still execute in init_mm */
 void zap_low_mappings(void)
 {
-	pgd_t *pgd = pgd_offset_k(0UL);
+	pgd_t *pgd;
+
+	save_pg_dir();
+
+	pgd = pgd_offset_k(0UL);
 	pgd_clear(pgd);
 	flush_tlb_all();
 }
#10 lore raw

Hi!

Quoted 7 lines
Quoted 4 lines

Long time ago there were i386 problems because we assumed that kernel is mapped in one big mapping and agp broke that assumption. Copying pages backwards "fixed" it (and then we done proper fix). It should not be, but it seems similar to this problem....

Do you mean this patch of yours?:
http://www.ussg.iu.edu/hypermail/linux/kernel/0404.3/0640.html

Yes.

Quoted 3 lines

I'm trying to do something similar for x86_64. See the attached patch. Unfortunately, it doesn't help. The behaviour seems unchanged (resume still works iff amd64-agp wasn't loaded before suspend).

Are you sure problem is on level4_pgt? We probably use constant level4_pgt but split pages at some deeper level. You may want try saving 3rd-level table, instead.
Pavel

Signature · 1 line
Boycott Kodak -- for their patent abuse against Java.
#11 lore raw
Quoted 7 lines · Pavel Machek
Quoted 2 lines

I'm trying to do something similar for x86_64. See the attached patch.
Unfortunately, it doesn't help. The behaviour seems unchanged (resume still works iff amd64-agp wasn't loaded before suspend).

Are you sure problem is on level4_pgt? We probably use constant level4_pgt but split pages at some deeper level. You may want try saving 3rd-level table, instead.

I'm not sure about that at all. That was just my attempt of cargocult programming :-)
OK, I'll try saving the 3rd-level table. It'll take me some time to figure out how to do that, however :-)

Michal

#12 lore raw
Quoted 6 lines · Michal Schmidt
Quoted 5 lines · Pavel Machek
Quoted 3 lines

I'm trying to do something similar for x86_64. See the attached patch. Unfortunately, it doesn't help. The behaviour seems unchanged (resume still works iff amd64-agp wasn't loaded before suspend).

Quoted 8 lines
Quoted 3 lines

Are you sure problem is on level4_pgt? We probably use constant level4_pgt but split pages at some deeper level. You may want try saving 3rd-level table, instead.

I'm not sure about that at all. That was just my attempt of cargocult programming :-)
OK, I'll try saving the 3rd-level table. It'll take me some time to figure out how to do that, however :-)

I think the amd64-agp is the problem here. There are some memory mappings that seem to require the hardware to be initialized before they can be used safely (at least as far as I understand it).

Greets,
Rafael

Signature · 3 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"
#13 lore raw
In reply to Michal Schmidt
Quoted 4 lines · Michal Schmidt

Hello,

Does resuming from swsuspend work for anyone with amd64-agp loaded?

It would seem not ;)

Quoted 6 lines

On my system when I suspend with amd64-agp loaded, I get a spontaneous reboot on resume. It reboots immediately after reading the saved image from disk.
This is 100% reproducible.

Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.

If this is reproducible in the soon-to-be-released 2.6.13-rc6 then could you please raise an entry at bugzilla.kernel.org and we'll plug away at it as the suspend stuff continues to get sorted out, thanks.

#14 lore raw
Quoted 5 lines · Andrew Morton
Quoted 2 lines · Michal Schmidt

Does resuming from swsuspend work for anyone with amd64-agp loaded?

It would seem not ;)

Must have missed the OP. Yes I can resume fine from swsusp with amd64-agp.

System is an Averatec 3270 (Mobile Sempron(K8)).

Not that this helps at all other than confirming it is possible ;)

-cp

Signature · 2 lines
"There are three kinds of lies: lies, damned lies, and statistics."
                     -- Benjamin Disraeli
#18 lore raw
In reply to Andrew Morton

Hi!

Quoted 9 lines
Quoted 1 line

Does resuming from swsuspend work for anyone with amd64-agp loaded?

It would seem not ;)

Quoted 5 lines

On my system when I suspend with amd64-agp loaded, I get a spontaneous reboot on resume. It reboots immediately after reading the saved image from disk.
This is 100% reproducible.

Quoted 5 lines
Quoted 1 line

Athlon 64 FX-53, Asus A8V Deluxe, Linux 2.6.13-rc3-mm1.

If this is reproducible in the soon-to-be-released 2.6.13-rc6 then could you please raise an entry at bugzilla.kernel.org and we'll plug away at it as the suspend stuff continues to get sorted out, thanks.

I assume it is in -rc6, too; it is long-standing bug and I am not aware of any attempts at fixing it. Please file bug report, assign to me.

OTOH it is not regression; AFAIK it never worked. Please do not let it block 2.6.13.

Pavel

Signature · 1 line
if you have sharp zaurus hardware you don't need... you know my address
#15 lore raw
In reply to Cal Peake
Quoted 10 lines · Cal Peake
Quoted 5 lines · Andrew Morton
Quoted 1 line · Michal Schmidt

Does resuming from swsuspend work for anyone with amd64-agp loaded?

It would seem not ;)

Quoted 8 lines

Must have missed the OP. Yes I can resume fine from swsusp with amd64-agp.

System is an Averatec 3270 (Mobile Sempron(K8)).

Not that this helps at all other than confirming it is possible ;)

-cp

AFAIK it works when agp is built into the kernel. You will get problems when it is built as a module. In the module case you may want to try if loading the module before resuming helps.

Signature · 1 line
Andreas Steinmetz                       SPAMmers use [email protected]
#16 lore raw
Quoted 3 lines · Andreas Steinmetz

AFAIK it works when agp is built into the kernel. You will get problems when it is built as a module. In the module case you may want to try if loading the module before resuming helps.

Strange. I've had it built as a module from day one and never had a problem resuming.

-cp

Signature · 2 lines
"There are three kinds of lies: lies, damned lies, and statistics."
                     -- Benjamin Disraeli
#17 lore raw
Quoted 10 lines · Cal Peake
Quoted 3 lines · Andreas Steinmetz

AFAIK it works when agp is built into the kernel. You will get problems when it is built as a module. In the module case you may want to try if loading the module before resuming helps.

Strange. I've had it built as a module from day one and never had a problem resuming.

I guess it depends on what the BIOS does.

Signature · 1 line
Andreas Steinmetz                       SPAMmers use [email protected]
#19 lore raw
In reply to Pavel Machek
Quoted 3 lines · Pavel Machek

I assume it is in -rc6, too; it is long-standing bug and I am not aware of any attempts at fixing it. Please file bug report, assign to me.

I've filed it as Bug 5018.
Michal