2.6.12-rc1 breaks dosemu

xdosemu 1.2.2 runs fine under 2.6.11.5, but fails under 2.6.12-rc1 with the following error:

<-- snip -->

$ xdosemu 
ERROR: cpu exception in dosemu code outside of VM86()!
trapno: 0x0e  errorcode: 0x00000005  cr2: 0xffffff8e
eip: 0x000069ee  esp: 0xbfdbffcc  eflags: 0x00010246
cs: 0x0073  ds: 0x007b  es: 0x007b  ss: 0x007b
Page fault: read instruction to linear address: 0xffffff8e
CPU was in user mode
Exception was caused by insufficient privilege

<-- snip -->

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

I just tried it here, and its ok while running 2.6.12-rc1

Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.34% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.

I had the same problem and found out that disabling address space randomization (echo 0 > /proc/sys/kernel/randomize_va_space) solves this reliably. With randomization enabled, I can start up dosemu maybe 1 out of 100 times.

I guess the randomization patches changed the mapping in a way that dosemu did not expect.

Arnd <><
-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

the randomisation patches came in a series of 8 patches (where several were general infrastructure); could you try to disable the individual randomisations one at a time to see which one causes this effect?

It's caused by top-of-stack-randomization.patch.

Arnd <><
-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

hmm interesting. Can you check if at the time of the crash, the esp is actually inside the stack vma? If it's not, I wonder what dosemu does to get its stack pointer outside the vma... (and on which side of the vma it is)

-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

To Arnd:

Another thing you should probably do is to build dosemu with debug information, and then look into ~/.dosemu/boot.log after it crashes. That will give you the contents of /proc/self/maps, a gdb backtrace and various other goodies.

I've checked it myself but can't reproduce, neither with plain dosemu 1.2.2 nor with current CVS.

Bart
-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

The esp value is always slightly below the stack vma and above ld.so. Running it a few times gives

stack VMA crash esp
bfc8f000-bfca4000 bfc5ffcc
bffa0000-bffb7000 bff5ffcc
bfe0c000-bfe23000 bfdbffcc
bf7ff000-bf814000 bf7bffcc
bfaa9000-bfabe000 bfa5ffcc
bfaa9000-bfabe000 bfa5ffcc
bffc5000-bffda000 bff7ffcc
bfba9000-bfbbf000 bfb5ffcc
bf865000-bf87b000 bf81ffcc
bfe7d000-bfe92000 bfe3ffcc
...
bff9f000-bffb4000 bff5ffcc
bfc73000-bfc89000 bfc3ffcc
bffe3000-bfff8000 -> works

I'm using the dosemu-1.2.1-3 binary that currently comes with debian sarge, and would prefer not having to build a new dosemu. As far as I can tell, the command.com that is started belongs to freedos, not comcom.
The crash however does happen shortly after the command.com file is opened.

Arnd <><
-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

the esp is 0x2F034/192564 bytes below the stack vma. That is a lot! I vaguely remember linux having a limit to how much below the stack vma it will allow accesses to auto-grow the stack, but I forgot what that limit actually was. I wonder if dosemu is somehow getting away with assuming a certain alignment by accident and then being inside the kernel grow limit, while with randomisation the alignment is only 4Kb and somehow a bigger-than-expected auto-grow is needed.

-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

hmm I just read back your first mail and it seems the actual memory access isn't to the stack at all but to 0xffffff8e sounds like dosemu had an internal underflow somewhere...

-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

looking at the dosemu code; the following bit looks a tad suspect:

unsigned long int stk_ptr, stk_beg, stk_end;
...

 if ((fp = fopen("/proc/self/maps", "r"))) {
    while(fgets(line, 100, fp)) {
      sscanf(line, "%lx-%lx", &stk_beg, &stk_end);
      if (stk_ptr >= stk_beg && stk_ptr < stk_end) {
        stack_init_top = stk_end;
        stack_init_bot = stk_beg;
        c_printf("CPU: Stack bottom %#lx, top %#lx, esp=%#lx\n",
	  stack_init_bot, stack_init_top, stk_ptr);
	break;
      }
    }
    fclose(fp);

}

do you see that printf somewhere in the logs?
(afaics stk_ptr never gets initialized; what the code meant probably was
if (&stk_ptr >= stk_beg && &stk_ptr < stk_end) {
but the dosemu code is missing the two &'s )

-
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I think stk_ptr is OK, and the log reports it correctly. Inline assembly gives it the value of esp. Also, this code is meant for DPMI programs so if it is the culprit it should not crash dosemu straight away, only if you'd start DOOM or something like that.

The log prints:
CPU: Stack bottom 0xbfd61000, top 0xbfd76000, esp=0xbfd75480

To Arnd:
There is one more improbable thing I can think of: comcom. This is dosemu's built-in command.com and uses some very tricky code (coopthreads), which certainly does not work any more with address space randomization. It's deprecated but was still present in 1.2.2, and Debian packages still used it with dosemu 1.2.1. The fix for that one is easy:
replace your command.com with a real DOS command.com (e.g. FreeDOS freecom).

Bart

#define STACK_GRAN_SHIFT 17 /* 128K address space granularity */
#define STACK_GRAN (1U << STACK_GRAN_SHIFT)
#define STACK_TOP 0xc0000000U
#define STACK_BOTTOM 0xa0000000U
#define STACK_TOP_PADDING STACK_GRAN
#define STACK_SLOTS ((STACK_TOP-STACK_BOTTOM) >> STACK_GRAN_SHIFT)

#define roundup_stacksize(size) ((size + STACK_GRAN - 1) & (-((int)STACK_GRAN)))

that certainly isn't boding well for things....

ok this thing is evil.
It hardcode assumes that the stack goes from 0xc00000 to 0xa0000000, divides it into slots of 128Kb and uses each such slot for a thread. With the randomisation there may be "slots" above the actual stack that appear free but are just entirely unmapped. This thing is broken beyond belief! (and it won't work for any other kind of split than 3:1, eg 2:2 or 4:4 or 0.5:3.5, as well as the 4Gig address space on x86-64 or ia64 in 32 bit emu mode)

I can't think of any reasonable way to work around this behavior other than suggesting to use the setarch option to disable randomisation for this process... this is just too much weirdness/brokenness to work around.

In some private correspondence with Arnd it turned out that this code was indeed the culprit for him. Fortunately it's easy to avoid -- when you do as I wrote above it becomes dead code, and dosemu works just fine (confirmed). In default dosemu 1.2.x setups it's also dead code; it's just Debian that chose to continue using it.

Fear not. The offending code has since been removed, in development versions of dosemu, if for no other reason than that except for the original author (Hans Lermen) nobody understood it.

Hope that clears things up.

Bart