swsuspend and CONFIG_DISCONTIGMEM=y

i686 -Iarch/i386/mach-generic -nostdinc -iwithprefix include -DKBUILD_BASENAME=suspend -DKBUILD_MODNAME=suspend -DEXPORT_SYMTAB -c -o kernel/suspend.o kernel/suspend.c
kernel/suspend.c:295: warning: #warning This might be broken. We need to somehow wait for data to reach the disk kernel/suspend.c: In function `free_some_memory':
kernel/suspend.c:627: `contig_page_data' undeclared (first use in this function) kernel/suspend.c:627: (Each undeclared identifier is reported only once
kernel/suspend.c:627: for each function it appears in.)
make[1]: ** [kernel/suspend.o] Erro 1
make: ** [kernel] Erro 2
[acme@oops hell_header-2.5]$ grep CONFIG_DISCONTIGMEM .config
CONFIG_DISCONTIGMEM=y

and in ./mm/page_alloc.c

#ifndef CONFIG_DISCONTIGMEM
static bootmem_data_t contig_bootmem_data;
struct pglist_data contig_page_data = { .bdata = &contig_bootmem_data };

void __init free_area_init(unsigned long *zones_size)

{
        free_area_init_node(0, &contig_page_data, NULL, zones_size, 0, NULL);
        mem_map = contig_page_data.node_mem_map;
}

#endif

So perhaps the following patch is in order? Its kind of brute force, disabling it altogether, but it at least fixes it for now.

You can import this changeset into BK by piping this whole message to:
'| bk receive [path to repository]' or apply the patch as usual.

===================================================================

[email protected], 2002-11-15 06:07:56-02:00, [email protected]
o swsusp: depends on CONFIG_DISCONTIGMEM=n

 Kconfig |    2 +-
 1 files changed, 1 insertion(+), 1 deletion(-)

diff -Nru a/arch/i386/Kconfig b/arch/i386/Kconfig
@@ -1518,7 +1518,7 @@
 
 config SOFTWARE_SUSPEND
 	bool "Software Suspend (EXPERIMENTAL)"
-	depends on EXPERIMENTAL && PM
+	depends on EXPERIMENTAL && PM && !DISCONTIGMEM
 	---help---
 	  Enable the possibilty of suspendig machine. It doesn't need APM.
 	  You may suspend your machine by 'swsusp' or 'shutdown -z <time>' 

===================================================================

This BitKeeper patch contains the following changesets:
1.851
## Wrapped with gzip_uu ##

begin 664 bkpatch30899 M'XL(`*ZKU#T``\V4T6K;,!2&KZ.GT"CT9M@^1[84Q>"1-LG:D*8-:0N#,8:J M:'%H;7+2=?AAY^=;$VW;BL;&TP2DM11[\^_>B7CM3Q"VE,T,.Z*EU9=S2 M-C>Z7&R4KVWFWQ1U8&IM'0A2FYG@>!2X.[=V*Y///.9S4L<GJM0IW9C"Q2WT MPX>5\GYEXM9T<')]=C0E)$EH+U7YW%R:DB8)*6VQ4<N9ZZHR7=K<+PN5N\R4 MVY.KAZT5V!UY=@.@8L*!43M2N,,445H9LB*:)]MD;F+W,A(@>!(?(*H(,1 MZ5/T)4<*+$,D%,0,;1C+CQ@,0!M^'2_YT)?(O6'-._>XT>T=32'>*8SDS# MV5&;T][%^>OAR?O^\+*>70U/QH-QDI,1%9P))),]6N+]9B$$%)!7]--BM3++ M[G*1KS]ZF9"WOBWF;[]>[UVE"IT&BU"*8%3#^+"8[VA""!"%#&7%(B%E)9"% M*$'==)"W!>@?X_M9MN9M)(B052Z`%O7/-G:N.<?J27%VI7WW:;7MEAMU:KU M\WF;TJZ51Q5PR>3.4]\Z2L0A>]91^!\XJN%^0;WB;MMJ@TR>/L$?N*R/G"%% M,OPRMAYI&;R9#*;#\>#\ZNB,'A[2R;CI7SS6MO]I=&KTK5MG"<=(1EPQ\AD, 'L(Q2R@0`````
`
end

The following dropped hunk from Pavel should repair it:

@@ -623,7 +623,7 @@
 static void free_some_memory(void)
 {
 	printk("Freeing memory: ");
-	while (try_to_free_pages(&contig_page_data.node_zones[ZONE_HIGHMEM], GFP_KSWAPD, 0))
+	while (shrink_all_memory(10000))
 		printk(".");
 	printk("|\n");
 }

[cc: list trimmed to spare the uninterested]

Hmm, there are some oddities here in count_and_copy_data_pages(). It looks like the CONFIG_HIGHMEM panic() is there because copy_page() is done without kmapping, and the CONFIG_DISCONTIGMEM panic() is there because the pgdat list etc. are not walked according to VM conventions.

So the traversal looks like it should go something like:

	for_each_zone(zone) {
		for (k = 0; k < zone->present_pages; ++k) {
			struct page *page = &zone->zone_mem_map[k];

			if (!PageReserved) {
				if (PageNosave(page))
					continue;

				chunk_size = is_head_of_free_region(page);

				/* c.f. k++ above */
				if (chunk_size) {
					k += chunk_size - 1;
					continue;
				}
			} else if (PageReserved(page)) {
				BUG_ON(PageNosave(page));

				if (page_to_pfn(page) >= nosave_begin_pfn
					&& page_to_pfn(page) < nosave_end_pfn)
					continue;
			}

			nr_copy_pages++;

			/*
			 * The general usage of page backup entries
			 * is unclear; this is probably incorrect in
			 * some cases, and needs some idea of the size
			 * and layout of the page backup entry array(s)
			 * if they cannot be contiguously allocated or
			 * simultaneously mapped by kernel pagetables.
			 */
			if (pagedir_p) {
				char *src, *dst;
				src = kmap_atomic(page, KM_SWSUSP0);
				dst = kmap_atomic(pagedir_p->page, KM_SWSUSP1);
				copy_page(dst, src);
				kunmap_atomic(dst, KM_SWSUSP0);
				kunmap_atomic(src, KM_SWSUSP1);
				++pagedir_p;
			}
		}
		return nr_copy_pages;
	}

I don't know what to make of highmem on laptops etc., but the VM's conventions should not be that hard to follow; also, there are uses for the swsusp functionality on other kinds of machines (e.g. checkpointing). Pure computationally-oriented systems such as would make use of this are somewhat different from my primary userbase to support, but I think it would be valuable to generalize swsusp in this way, and so provide rudimentary support for such users in addition to some small measure of cleanup (i.e. the cleanup adds functionality).

Pavel, what do you think?

Bill

Hi!

How much memory is needed for HIGHMEM to be neccessary? Is it 1GB? If so, I can well imagine 1GB laptop....

This certainly does not work. We'd need to do some deep magic in suspend_asm.S to copy pages back. [Well, deep magic... Same kmap_atomic.] But suspend_asm.S has to guarantee not touching any memory so the change is not quite trivial.

I definitely want to support swsusp for server boxes, but I'm not 100% sure how to do that easily.
Pavel

Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

[code snippet snipped]

I wasn't entirely expecting it to work out of the box. I was really trying to get the VM structure traversal and access in line with the general case, and then worry (later) about deeper issues like that once they arise. kmap_atomic() etc. should be no-ops without highmem, so otherwise it'd be safe (at least on the save path), aside from perhaps suggesting that highmem should work when it wouldn't. The restore path, OTOH...

I should take a look at suspend_asm.S to get an idea of what you're talking about with respect to its expectations being violated by the code snippet I posted.

I'm not entirely sure either. Mostly I suspect that the deep arch issues will be the tough ones, but things like this I can handle. =)

Thanks,
Bill

Hi!

Well, I'd really hate to do 64GB support for swsusp for i386. It would mean wider pointers in on-disk format and probably would not be exactly nice.

Pavel

Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?

i386 is actually a poor cpu for numerical workloads due to the design of its FPU, so the interest will likely be low there, but...

It looks like the struct pbe is storing a physical address in an unsigned long; using page frame numbers instead of raw physical addresses should at least catch 36-bit i386 and a substantial fraction of PPC highmem (40-bit, so not all) with very little effort. A good chunk of the VM's arch support address calculation API (if not all) has been converted to pfn-based calculations already, so we're already in very good shape, aside from a one-shot format change.

Bill

Depends on the arch & other matters. 768Mb on PPC at least, and it starting to be common within laptops as well.

At worst, that could be an arch provided routine. On most PPC32's I can then just disable data translation on the MMU and access all pages without kmap'ing them. But that's not terribly portable and each arch would need different kind of hacking.

Ben.

Hi!

I just hope we'll never ever see 64GB i386 laptop...

... We'll probably have to write simpler equivalent of kmap_atomic for use in suspend_asm.S. It is not really *so* deep magic.

It is arch-specific already. What's worse, resume has to be written in assembly, so you can't simply write kmap_atomic() there. See arch/i386/kernel/suspend_asm.S.

Pavel

Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?

64GB raises deadly scalability (read as: box won't boot) issues with boot-time memory reservations and other things. It's an open question as to whether it will ever be made to work with mainline Linux, but one I'd like to see answered "yes", if only because it is in some senses the ultimate test of leanness: "If you are bloated, you will die."

But like I said, it's very unlikely any strong interest will ever arise specifically in large-scale i386 checkpointing. Computational workloads are very attached to having clean and efficient FPU's, which i386 lacks. RISC etc. boxen with clean FPU's are more important for that. OTOH if highmem works, why wouldn't bigger highmem boxen work? NFI

The general framework is all I have a direct interest in accommodating. Aside from that I don't see a real need to make the low-level guts of large-scale highmem systems support swsusp, though 1GB laptops make sense.

Thanks,
Bill

Most large scale FP computation jobs are done on x86. RISC stuff isnt that much faster if at all. It may be elegant but the PIV and Athlon go at 2.5Ghz so make up for elegance by being stupid very fast (note that by the PIV and athlon the FPU's are not actually very veryy smart)

Alan

The clock speeds of i386 may very well now be up to the point where its clock speeds dominate all other considerations. But I've never seen large-scale i386 machines used in what numerics I've been involved in, which, granted, was not much and several years ago (SGI dominated).

I'm still somewhat skeptical given the cpu counts, process address space, and I/O architecture limitations on i386, but I don't honestly care. Numerics is not my game anymore if it ever was, and whatever additional functionality might come out of cleanups is only ammunition to push the cleanups to me.

Bill

Their built-in trancendental ops still beat the pants off RISC clock for clock, so even at the same clock rate, some number crunching stuff still wins big on x86. Throw in the cost/FLOPS difference and its little wonder that very few big machines are being built with anything but.

 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

Hi!

Please don't, better patch is pending to fix that.
Pavel

Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Em Fri, Nov 15, 2002 at 12:54:02PM +0100, Pavel Machek escreveu:

wli told me, going to use shrink_all_memory(), etc

- Arnaldo