problem with 64bit kernel, BOOT_ELF32 and memory outside CKSEG0

I tried to get a working 64bit kernel for SNI RM. Most of things to fix were quite obvious, but there is one thing, which I haven't understood yet.

The firmware is only able to boot ELF32 images, which mean I need to use BOOT_ELF32.

RM machines have 256MB memory mapped to KSEG0, anything else is outside. To me that would mean I need to use the default spaces from mach-generic/spaces.h. A kernel built that way will hang after calling schedule() in rest_init() (init/main.c). Has anybody seen this as well ?

Before digging into schedule() I decided to try mach-ip22/spaces.h and limit the installed memory to 256MB. This kernel boots and runs fine.

Then I booted that kernel with all memory (512MB) and it died, when init had troubles mapping libc. That result didn't surprise me that much, since the kernel probably tried to map memory > 256MB via CKSEG0, which won't work. Correct ?

Thomas.

Crap can work. Given enough thrust pigs will fly, but it's not necessary a
good idea.                                                [ RFC1925, 2.3 ]

No.

schedule() doesn't directly depend on very much else working so I get the feeling that your problem may be quite far away.

You should keep all memory mapped in the same kernel segment. Since you have too much to do that in CKSEG0 it will have to be XKPHYS which means the generic spaces.h which will be used when you have none in in mach-rm200/space.h.

I suspect what happens is the kernel is trying to access memory at above phys. 512MB. In your setup it would compute a CKSEG1 address for that and stomp over something it better shouldn't ...

Ralf

well it depends on getting a kernel thread started. And I guess there lies the problem.

Maybe I need to ask more precisely what I want to know: Should a kernel linked to CSKEG0 with a CAC_BASE 0x980000000000 work or is this setup "wrong" ? If the answer is yes, I'd say there is still a bug left. If no how do we solve the issue of having a 32bit firmware and a kernel linked for XPHYS (using some sort of bootloader is not want I want to have, if it's avoidable) ?

Thomas.

Crap can work. Given enough thrust pigs will fly, but it's not necessary a
good idea.                                                [ RFC1925, 2.3 ]

Typically systems try to put the kernel into CKSEG0 as long as the given platform happens to have memory mapped there. Using CKSEG0 allows using a shorter sequence to load the address of an in-kernel symbol resulting in a much smaller kernel binary.

One implication of this is that kernel modules will have to be allocated to CKSEG2/3 because they're built using the same code model. Modules are allocated using vmalloc so the vmalloc space needs to be in CKSEG2/3. Most people don't care if that space is restricted to just one gig.

But even if you get that wrong the expected failure mode is different ...

Ralf

Ralf and me had an debug session on IRC and I finally figured out what caused the problem: CONFIG_EARLY_PRINTK via prom calls.

I simply used call_o32.S from the decstation part and missed the fact, that it simply uses the normal kernel stack when calling firmware. This works quite good until the first kernel thread gets scheduled, which has a kernel stack via a CAC_BASE address. So after switching stack the next call to prom_putchar() killed the machine. Simply disabling EARLY_PRINTK gives me a working 64bit kernel, which sees the whole 512MB RAM :-)

Thomas.

Crap can work. Given enough thrust pigs will fly, but it's not necessary a
good idea.                                                [ RFC1925, 2.3 ]

You could do stack switching in call_o32() -- I just figured there was no
point in adding this complication as the DECstation always runs from KSEG0 -- it has the maximum of 480MB of RAM mapped linearly starting from 0.

Maciej