Skip to contentSkip to conversation

Segfault on the i386 enter instruction

DISCUSSION Chuck Ebbert 2 participants 2 messages May 16, 2006

lore
Share & tools
lore
#0 lore raw

In-Reply-To: <[email protected]>

Quoted 10 lines · Stas Sergeev
Quoted 3 lines · Andi Kleen

Handling it like you expect would require to disassemble the function in the page fault handler and it's probably not worth doing that for this weird case.

Just wondering, is this case really that weird?
In fact, the check against %esp that the kernel does, looks strange. I realize that it can catch a (very rare) user-space bug of accessing below %esp, but other than that it looks redundant (IMHO) and as soon as it triggers the false-positives, what is it really good for?

I can't get a SIGSEGV on any native i386 kernel, not even when running on AMD64. It only happens on native x86_64 kernels.

Looking at the AMD instruction manual, the pseudo-code for 'enter' ends with:

RSP.s = RSP - temp_ALLOC_SPACE // Leave "temp_ALLOC_SPACE" free bytes on

                               // the stack
WRITE_MEM.v [SS:RSP.s] = temp_unused // ENTER finishes with a memory write
                                     // check on the final stack pointer,
                                     // but no write actually occurs.

RBP.v = temp_RBP
EXIT

And the Intel manual says:

IF 64-Bit Mode (StackSize = 64)
THEN

        RBP = FrameTemp;
        RSP = RSP - Size;
ELSE IF StackSize = 32
THEN
        EBP = FrameTemp;
        ESP = ESP - Size;
ELSE (* StackSize = 16 *)
        BP = FrameTemp;
        SP = SP - Size;

FI;
END;

Intel says nothing about a write check. Is that a mistake in the manual or is that something only AMD64 does, and then only in long mode?

Signature · 4 lines
Chuck

"The x86 isn't all that complex -- it just doesn't make a lot of sense."
                                                        -- Mike Johnson
#1 lore raw
Quoted 15 lines · Chuck Ebbert

In-Reply-To: <[email protected]>

Quoted 10 lines · Stas Sergeev
Quoted 3 lines · Andi Kleen

Handling it like you expect would require to disassemble the function in the page fault handler and it's probably not worth doing that for this weird case.

Just wondering, is this case really that weird?
In fact, the check against %esp that the kernel does, looks strange. I realize that it can catch a (very rare) user-space bug of accessing below %esp, but other than that it looks redundant (IMHO) and as soon as it triggers the false-positives, what is it really good for?

Quoted 2 lines

I can't get a SIGSEGV on any native i386 kernel, not even when running on AMD64. It only happens on native x86_64 kernels.

I reproduced the original SIGSEGV on several i386 kernels.

Quoted 2 lines

Intel says nothing about a write check. Is that a mistake in the manual or is that something only AMD64 does, and then only in long mode?

In 98+% of all cases when Intel and AMD documentation differ in subtle detail it's a documentation bug.

-Andi