Skip to contentSkip to conversation

swsusp: fix some obscure bugs

PATCH In review Rafael J. Wysocki 3 participants 15 messages Sep 25, 2005 – Sep 26, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover swsusp: fix some obscure bugs

Hi,

The following series of patches fixes some obscure bugs in swsusp.

Greetings,
Rafael

Signature · 3 lines
- Would you tell me, please, which way I ought to go from here?
- That depends a good deal on where you want to get to.
		-- Lewis Carroll "Alice's Adventures in Wonderland"
#1 lore raw

PATCH 1/3 swsusp: remove wrong code from data_free

The following patch removes some wrong code from the data_free() function in swsusp.

This function could only be called if there's an arror while writing the suspend image to swap, so it is not triggered easily. However, if triggered, it would probably corrupt some memory.

Please apply,
Rafael

kernel/power/swsusp.c +3 -4
Index: linux-2.6.14-rc2-git3/kernel/power/swsusp.c
===================================================================
@@ -402,15 +402,14 @@
 static void data_free(void)
 {
 	swp_entry_t entry;
-	int i;
+	struct pbe * p;
 
-	for (i = 0; i < nr_copy_pages; i++) {
-		entry = (pagedir_nosave + i)->swap_address;
+	for_each_pbe(p, pagedir_nosave) {
+		entry = p->swap_address;
 		if (entry.val)
 			swap_free(entry);
 		else
 			break;
-		(pagedir_nosave + i)->swap_address = (swp_entry_t){0};
 	}
 }
 
#2 lore raw

PATCH 2/3 swsusp: prevent possible memory leak

In reply to Rafael J. Wysocki

The following patch prevents swsusp from leaking some memory in case of an error in read_pagedir(). It also prevents the BUG_ON() from triggering if there's an error while reading swap.

Please apply,
Rafael

kernel/power/swsusp.c +3 -3
Index: linux-2.6.14-rc2-git3/kernel/power/swsusp.c
===================================================================
@@ -1437,9 +1437,9 @@
 	}
 
 	if (error)
-		free_page((unsigned long)pblist);
-
-	BUG_ON(i != swsusp_info.pagedir_pages);
+		free_pagedir(pblist);
+	else
+		BUG_ON(i != swsusp_info.pagedir_pages);
 
 	return error;
 }
#3 lore raw

PATCH 3/3 swsusp: prevent swsusp from failing if there's too many pagedir pages

In reply to Rafael J. Wysocki

There's a silent assumption in swsusp that always
sizeof(struct swsusp_info) <= PAGE_SIZE, which is wrong, because eg. on x86-64 sizeof(swp_entry_t) = 8. This causes swsusp to skip some pagedir pages while reading the image if there are too many of them (depending on the architecture, approx. 500 on x86-64).

The following patch makes swsusp more flexible with this respect, by allocating separate swap page(s) (as many as needed) for storing the swap offsets of pagedir pages (without the patch they are all stored in a struct swsusp_info, and there may be not enough room for them in this structure).

Please consider for applying,
Rafael

kernel/power/power.h +5 -2
Index: linux-2.6.14-rc2-git3/kernel/power/power.h
===================================================================
@@ -9,6 +9,7 @@
 #define SUSPEND_CONSOLE	(MAX_NR_CONSOLES-1)
 #endif
 
+#define MAX_PB_SWP_PAGES	128
 
 struct swsusp_info {
 	struct new_utsname	uts;
@@ -18,10 +19,12 @@
 	unsigned long		image_pages;
 	unsigned long		pagedir_pages;
 	suspend_pagedir_t	* suspend_pagedir;
-	swp_entry_t		pagedir[768];
+	unsigned		pb_swp_pages;
+	swp_entry_t		pb_swp[MAX_PB_SWP_PAGES];
+	swp_entry_t		* pb_swp_mem[MAX_PB_SWP_PAGES];
 } __attribute__((aligned(PAGE_SIZE)));
 
-
+/* pb_swp_mem and pb_swp_pages are needed for error recovery */
 
 #ifdef CONFIG_SOFTWARE_SUSPEND
 extern int pm_suspend_disk(void);
kernel/power/swsusp.c +77 -14

138 lines not on this page · raw message

#4 lore raw

Re: [PATCH 3/3][Fix] swsusp: prevent swsusp from failing if there's too many pagedir pages

Hi!

Quoted 5 lines

There's a silent assumption in swsusp that always
sizeof(struct swsusp_info) <= PAGE_SIZE, which is wrong, because eg. on x86-64 sizeof(swp_entry_t) = 8. This causes swsusp to skip some pagedir pages while reading the image if there are too many of them (depending on the architecture, approx. 500 on x86-64).

Last time I did the math, swsusp_info could cover a *lot* of memory. It was wrong not to check for overflow, but I do not think we want to introduce *yet another* linklist.

Lets see...

for i386, we have 768 pagedir entries. Each pagedir entry points to page with 1023 pointers to pages. That means that up-to 768*1023*4096 bytes image can be saved to swap ~= 768 * 1K * 4K ~= 3 GB. That's more than enough for i386.

for x86-64, we can have 128 pagedir entries (could not we fit more there? 384 entries should fit, no?). Each pagedir entry has 511 pointers to pages (IIRC)... that is up-to 128*511*4K ~= 64*1K*4K = 256 MB image. Hmm, that should still be enough for any 512MB machine, and probably okay for much bigger machines, too...

We can still get to 768 MB image (good enough for any 1.5GB machine, and probably for anything else, too).

If that is not good enough for you, can you simply allocate more than 1 page for swsusp_info? No need for linklists yet.

Andrew, please drop this one. It is too complex solution for quite a simple problem.
Pavel

Signature · 1 line
if you have sharp zaurus hardware you don't need... you know my address
#5 lore raw

Re: [PATCH 3/3][Fix] swsusp: prevent swsusp from failing if there's too many pagedir pages

Hi,

Quoted 11 lines · Pavel Machek

Hi!

Quoted 5 lines

There's a silent assumption in swsusp that always
sizeof(struct swsusp_info) <= PAGE_SIZE, which is wrong, because eg. on x86-64 sizeof(swp_entry_t) = 8. This causes swsusp to skip some pagedir pages while reading the image if there are too many of them (depending on the architecture, approx. 500 on x86-64).

Last time I did the math, swsusp_info could cover a *lot* of memory. It was wrong not to check for overflow, but I do not think we want to introduce *yet another* linklist.

Yes, I thought of another solution, but any of them would require more than one swap page and I'd have to track the swap offsets of them somehow.

Quoted 9 lines

Lets see...

for i386, we have 768 pagedir entries. Each pagedir entry points to page with 1023 pointers to pages. That means that up-to 768*1023*4096 bytes image can be saved to swap ~= 768 * 1K * 4K ~= 3 GB. That's more than enough for i386.

for x86-64, we can have 128 pagedir entries (could not we fit more there? 384 entries should fit, no?).

Yes. To be exact, 460.

Quoted 1 line

Each pagedir entry has 511 pointers to pages (IIRC)...

512, I think.

Quoted 9 lines

that is up-to 128*511*4K ~= 64*1K*4K = 256 MB image. Hmm, that should still be enough for any 512MB machine, and probably okay for much bigger machines, too...

We can still get to 768 MB image (good enough for any 1.5GB machine, and probably for anything else, too).

If that is not good enough for you, can you simply allocate more than 1 page for swsusp_info? No need for linklists yet.

I can. The problem is I have to track the swap offsets of these pages which is necessary for resume. Is it guaranteed that the swap offsets of pages allocated in a row will be consecutive?

Quoted 2 lines

Andrew, please drop this one. It is too complex solution for quite a simple problem.

Perhaps it is. Anyway the problem hit me when I was playing with swsusp on a machine with 768 MB of RAM.

Greetings,
Rafael

#6 lore raw

Re: [PATCH 3/3][Fix] swsusp: prevent swsusp from failing if there's too many pagedir pages

Hi!

Quoted 7 lines
Quoted 7 lines

Lets see...

for i386, we have 768 pagedir entries. Each pagedir entry points to page with 1023 pointers to pages. That means that up-to 768*1023*4096 bytes image can be saved to swap ~= 768 * 1K * 4K ~= 3 GB. That's more than enough for i386.

Quoted 8 lines
Quoted 2 lines

for x86-64, we can have 128 pagedir entries (could not we fit more there? 384 entries should fit, no?).

Yes. To be exact, 460.

Quoted 1 line

Each pagedir entry has 511 pointers to pages (IIRC)...

512, I think.

Okay, can we do simple solution where we put 460 there, plus a check if it overflows (printk, abort suspend), for now? That should fix 768MB machine...

Quoted 7 lines
Quoted 7 lines

that is up-to 128*511*4K ~= 64*1K*4K = 256 MB image. Hmm, that should still be enough for any 512MB machine, and probably okay for much bigger machines, too...

We can still get to 768 MB image (good enough for any 1.5GB machine, and probably for anything else, too).

Quoted 6 lines
Quoted 2 lines

If that is not good enough for you, can you simply allocate more than 1 page for swsusp_info? No need for linklists yet.

I can. The problem is I have to track the swap offsets of these pages which is necessary for resume. Is it guaranteed that the swap offsets of pages allocated in a row will be consecutive?

No, they probably will not be consecutive.

OTOH pagedirs are stored as a link list in memory already. Maybe we should be able to extend that link list for a disk, too, with minimal fuss? ...we'd have to write pagedir _backwards_ for that to work, probably not nice, and swap_free() would really like direct access.
Pavel

Signature · 1 line
if you have sharp zaurus hardware you don't need... you know my address
#7 lore raw

Re: [PATCH 3/3][Fix] swsusp: prevent swsusp from failing if there's too many pagedir pages

Hi,

Quoted 14 lines · Pavel Machek

Hi!

Quoted 12 lines
Quoted 9 lines

Lets see...

for i386, we have 768 pagedir entries. Each pagedir entry points to page with 1023 pointers to pages. That means that up-to 768*1023*4096 bytes image can be saved to swap ~= 768 * 1K * 4K ~= 3 GB. That's more than enough for i386.

for x86-64, we can have 128 pagedir entries (could not we fit more there? 384 entries should fit, no?).

Yes. To be exact, 460.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

Each pagedir entry has 511 pointers to pages (IIRC)...

512, I think.

Okay, can we do simple solution where we put 460 there, plus a check if it overflows (printk, abort suspend), for now? That should fix 768MB machine...

For now: a constant in power.h depending on sizeof(long) and PAGE_SIZE, the size of swsusp_info.pagedir[] depending on it and the overflow check in write_pagedir()?

Unfortunately it's not enough for what I'm cooking (think of resuming in 35 sec. to a fully responsive system - well, that's on my box). A preliminary patch is at http://www.sisk.pl/kernel/patches/2.6.14-rc2-git3/swsusp-improve-freeing-memory.patch

Quoted 14 lines
Quoted 7 lines
Quoted 7 lines

that is up-to 128*511*4K ~= 64*1K*4K = 256 MB image. Hmm, that should still be enough for any 512MB machine, and probably okay for much bigger machines, too...

We can still get to 768 MB image (good enough for any 1.5GB machine, and probably for anything else, too).

Quoted 6 lines
Quoted 2 lines

If that is not good enough for you, can you simply allocate more than 1 page for swsusp_info? No need for linklists yet.

I can. The problem is I have to track the swap offsets of these pages which is necessary for resume. Is it guaranteed that the swap offsets of pages allocated in a row will be consecutive?

Quoted 6 lines

No, they probably will not be consecutive.

OTOH pagedirs are stored as a link list in memory already. Maybe we should be able to extend that link list for a disk, too, with minimal fuss? ...we'd have to write pagedir _backwards_ for that to work, probably not nice, and swap_free() would really like direct access.

We write the pagedir after we have written the image, so the address field of each entry is not needed at that time, except for freeing the image memory in case of failure (with the "rework image freeing patch" they are not needed at all). Thus potentially we can use the address fields of pagedir entries to link the pages on the swap.

Greetings,
Rafael

#8 lore raw

Re: [PATCH 3/3][Fix] swsusp: prevent swsusp from failing if there's too many pagedir pages

Hi!

Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
Quoted 9 lines

Lets see...

for i386, we have 768 pagedir entries. Each pagedir entry points to page with 1023 pointers to pages. That means that up-to 768*1023*4096 bytes image can be saved to swap ~= 768 * 1K * 4K ~= 3 GB. That's more than enough for i386.

for x86-64, we can have 128 pagedir entries (could not we fit more there? 384 entries should fit, no?).

Yes. To be exact, 460.

Quoted 1 line

Each pagedir entry has 511 pointers to pages (IIRC)...

512, I think.

Quoted 7 lines
Quoted 3 lines

Okay, can we do simple solution where we put 460 there, plus a check if it overflows (printk, abort suspend), for now? That should fix 768MB machine...

For now: a constant in power.h depending on sizeof(long) and PAGE_SIZE, the size of swsusp_info.pagedir[] depending on it and the overflow check in write_pagedir()?

Yes, that would be very nice.

Quoted 3 lines

Unfortunately it's not enough for what I'm cooking (think of resuming in 35 sec. to a fully responsive system - well, that's on my box). A preliminary patch is at http://www.sisk.pl/kernel/patches/2.6.14-rc2-git3/swsusp-improve-freeing-memory.patch

Okay, I see, nice... We want to support that in future. (Actually it is last piece of puzzle for swsusp3).

Quoted 7 lines
Quoted 6 lines

No, they probably will not be consecutive.

OTOH pagedirs are stored as a link list in memory already. Maybe we should be able to extend that link list for a disk, too, with minimal fuss? ...we'd have to write pagedir _backwards_ for that to work, probably not nice, and swap_free() would really like direct access.

Quoted 5 lines

We write the pagedir after we have written the image, so the address field of each entry is not needed at that time, except for freeing the image memory in case of failure (with the "rework image freeing patch" they are not needed at all). Thus potentially we can use the address fields of pagedir entries to link the pages on the swap.

I plan to push "rework image freeing patch" for other reasons, too. I'd like to run longer tests on it, but so far it looks okay.

Pavel

Signature · 1 line
if you have sharp zaurus hardware you don't need... you know my address
#9 lore raw

Re: [PATCH 3/3][Fix] swsusp: prevent swsusp from failing if there's too many pagedir pages

Hi,

Quoted 2 lines · Pavel Machek

Hi!

]-- snip --[

Quoted 7 lines
Quoted 4 lines

Unfortunately it's not enough for what I'm cooking (think of resuming in 35 sec. to a fully responsive system - well, that's on my box). A preliminary patch is at http://www.sisk.pl/kernel/patches/2.6.14-rc2-git3/swsusp-improve-freeing-memory.patch

Okay, I see, nice... We want to support that in future. (Actually it is last piece of puzzle for swsusp3).

Well, it can be implemented within the current swsusp, IMO. It actually works for me now. :-)

]-- snip --[

Quoted 3 lines

I plan to push "rework image freeing patch" for other reasons, too. I'd like to run longer tests on it, but so far it looks okay.

I think it's fine. Could you please point me to the current version?

Greetings,
Rafael

#10 lore raw

Re: [PATCH 3/3][Fix] swsusp: prevent swsusp from failing if there's too many pagedir pages

Hi!

[Andrew killed from Cc, lets not flood his mailbox].

Quoted 9 lines
Quoted 6 lines
Quoted 3 lines

Unfortunately it's not enough for what I'm cooking (think of resuming in 35 sec. to a fully responsive system - well, that's on my box). A preliminary patch is at http://www.sisk.pl/kernel/patches/2.6.14-rc2-git3/swsusp-improve-freeing-memory.patch

Okay, I see, nice... We want to support that in future. (Actually it is last piece of puzzle for swsusp3).

Well, it can be implemented within the current swsusp, IMO. It actually works for me now. :-)

Agreed, and swsusp3 provides graphical progress bar, esc-to-escape, and "are you sure you want to crash your machine" dialogs :-).

Quoted 4 lines
Quoted 2 lines

I plan to push "rework image freeing patch" for other reasons, too. I'd like to run longer tests on it, but so far it looks okay.

I think it's fine. Could you please point me to the current version?

Well, its a bit complicated. I have 3 related trees on kernel.org:

linux-good -- that are basically patches I either already sent upstream or that I plan to send very soon. Unfortunately I did not yet have time to apply all your stuff that you sent me for akpm; I'll need to pull them from -mm and apply here.

linux -- that is my working tree. It contains some stuff you probably do not want (like special keymap).

linux-sw3 -- and that's a tree to test suspending from userland, aka swsusp3 (aka uswsusp). Currently stuck at 2.6.13 ("first make it work, then port it to HEAD"), and it seems to work (but you should really know what you are doing, and use ext2). usr/swsusp.c is the userland program to do suspending.

Pavel

Signature · 1 line
if you have sharp zaurus hardware you don't need... you know my address
#11 lore raw

PATCH 1/1 swsusp: avoid problems if there are too many pages to save

In reply to Pavel Machek

Hi,

The following patch makes swsusp avoid problems during resume if there are too many pages to save on suspend. It adds a constant that allows us to verify if we are going to save too many pages and implements the check (this is done as early as we can tell that the check will trigger, which is in swsusp_alloc()).

This is to replace swsusp-prevent-swsusp-from-failing-if-theres-too-many-pagedir-pages.patch

Please consider for applying.

Greetings,
Rafael

kernel/power/power.h +4 -1
Index: linux-2.6.14-rc2-git6/kernel/power/power.h
===================================================================
@@ -9,6 +9,9 @@
 #define SUSPEND_CONSOLE	(MAX_NR_CONSOLES-1)
 #endif
 
+#define MAX_PBES	((PAGE_SIZE - sizeof(struct new_utsname) \
+			- 4 - 3*sizeof(unsigned long) - sizeof(int) \
+			- sizeof(void *)) / sizeof(swp_entry_t))
 
 struct swsusp_info {
 	struct new_utsname	uts;
@@ -18,7 +21,7 @@
 	unsigned long		image_pages;
 	unsigned long		pagedir_pages;
 	suspend_pagedir_t	* suspend_pagedir;
-	swp_entry_t		pagedir[768];
+	swp_entry_t		pagedir[MAX_PBES];
 } __attribute__((aligned(PAGE_SIZE)));
 
 
kernel/power/swsusp.c +4 -0
Index: linux-2.6.14-rc2-git6/kernel/power/swsusp.c
===================================================================
@@ -931,6 +931,10 @@
 	if (!enough_swap())
 		return -ENOSPC;
 
+	if (MAX_PBES < nr_copy_pages / PBES_PER_PAGE +
+	    !!(nr_copy_pages % PBES_PER_PAGE))
+		return -ENOSPC;
+
 	if (!(pagedir_save = alloc_pagedir(nr_copy_pages))) {
 		printk(KERN_ERR "suspend: Allocating pagedir failed.\n");
 		return -ENOMEM;
#12 lore raw

Re: [PATCH][Fix] swsusp: avoid problems if there are too many pages to save

Hi!

Quoted 8 lines

The following patch makes swsusp avoid problems during resume if there are too many pages to save on suspend. It adds a constant that allows us to verify if we are going to save too many pages and implements the check (this is done as early as we can tell that the check will trigger, which is in swsusp_alloc()).

This is to replace swsusp-prevent-swsusp-from-failing-if-theres-too-many-pagedir-pages.patch

Please consider for applying.

Andrew, if you could apply this one, it would be great. ACKed-by:
Pavel Machek <[email protected]>

Pavel

Signature · 1 line
if you have sharp zaurus hardware you don't need... you know my address
#13 lore raw
In reply to Rafael J. Wysocki

Hi!

Quoted 1 line

The following series of patches fixes some obscure bugs in swsusp.

Andrew, could you apply parts #1 and #2? First is data-corruption in error path (nasty!), second is simple bugfix.

Alternatively, I'll eat them into my tree and mail you proper patches with all signed-offs, but that will be tommorow. (Have to get some sleep now.)
Pavel

Signature · 1 line
if you have sharp zaurus hardware you don't need... you know my address
#14 lore raw
Quoted 5 lines · Pavel Machek
Quoted 2 lines

The following series of patches fixes some obscure bugs in swsusp.

Andrew, could you apply parts #1 and #2? First is data-corruption in error path (nasty!), second is simple bugfix.

Sure.