Skip to contentSkip to conversation

32-bit syscalls from 64-bit process on x86-64?

DISCUSSION Jeremy Fitzhardinge 3 participants 8 messages Dec 2, 2004 – Dec 14, 2004

lore
Share & tools
lore
#0 lore raw

Hi Andi,

Is it possible for a 64-bit process to invoke the 32-bit syscall compatibility layer? I'm thinking this might be useful for Valgrind, since if it is running on an x86-64 host, it can take advantage of having more registers and a larger address space to do a better emulation of plain ia32. But this is only practical if I can reuse the kernel's 32-bit emulation layer, since duplicating it in Valgrind would be silly (particularly ioctls).

Quoted 1 line

From a quick look at the code, it seems to me that int 0x80 might still

work in 64-bit mode, but connect to 32-bit syscalls. Is that right? If not, could it be made to be right? Alternatively, something like adding a constant offset to the syscall numbers would work for me (ie, 0-N are 64-bit syscalls, 0x10000-N are 32-bit). Hm, no, it looks like int 0x80 just calls normal 64-bit syscalls...

And does the 32-bit layer keep any private state? For example, if I modify the signal state with 32-syscalls in one place, and 64-bit syscalls elsewhere, will that cause a problem or inconsistencies?

Thanks,
J

#1 lore raw
Quoted 1 line · On Thu, 2004-12-02 at 08:22 -0800, I wrote:

Hi Andi,

[ Or any other x86-64-heads. ]

J

#2 lore raw
In reply to Jeremy Fitzhardinge
Quoted 9 lines · Jeremy Fitzhardinge

Hi Andi,

Is it possible for a 64-bit process to invoke the 32-bit syscall compatibility layer? I'm thinking this might be useful for Valgrind, since if it is running on an x86-64 host, it can take advantage of having more registers and a larger address space to do a better emulation of plain ia32. But this is only practical if I can reuse the kernel's 32-bit emulation layer, since duplicating it in Valgrind would be silly (particularly ioctls).

It has been done actually (in the intel ia32el JIT)

Quoted 7 lines
Quoted 2 lines

From a quick look at the code, it seems to me that int 0x80 might still

work in 64-bit mode, but connect to 32-bit syscalls. Is that right? If not, could it be made to be right? Alternatively, something like adding a constant offset to the syscall numbers would work for me (ie, 0-N are 64-bit syscalls, 0x10000-N are 32-bit). Hm, no, it looks like int 0x80 just calls normal 64-bit syscalls....

int 0x80 from 64bit will call 32bit syscalls. But some things will not work properly, e.g. signals because they test the 32bit flag in the task_struct. So you would still get 64bit signal frames. There are some other such tests, but they probably wont affect you.

There were some plans at one point to allow to toggle the flag using personality or prctl, but so far it hasn't been implemented. There is also no way to do 64bit calls from a 32bit executable.

Quoted 4 lines

And does the 32-bit layer keep any private state? For example, if I modify the signal state with 32-syscalls in one place, and 64-bit syscalls elsewhere, will that cause a problem or inconsistencies?

64bit processes always get 64bit signals.

-Andi

#3 lore raw
Quoted 1 line · Andi Kleen

int 0x80 from 64bit will call 32bit syscalls.

Hm, that's not what I found. If I run int 0x80 with [re]ax=1, I get exit with a 32-bit process and write with a 64-bit one.

Quoted 3 lines

But some things
will not work properly, e.g. signals because they test the 32bit flag in the task_struct. So you would still get 64bit signal frames.

OK, I can deal with that, since signals are never directly delivered to the client.

Quoted 5 lines

There are some other such tests, but they probably wont affect you.

There were some plans at one point to allow to toggle the flag using personality or prctl, but so far it hasn't been implemented. There is also no way to do 64bit calls from a 32bit executable.

What about my idea of using distinct ranges of syscall numbers to disambiguate them?

J

#4 lore raw
Quoted 5 lines · Jeremy Fitzhardinge
Quoted 1 line · Andi Kleen

int 0x80 from 64bit will call 32bit syscalls.

Hm, that's not what I found. If I run int 0x80 with [re]ax=1, I get exit with a 32-bit process and write with a 64-bit one.

Seems to work for me. See second example below. If it won't work, you can always use reverse procedure to one I used in syscall.c - only problem is that you'll have to allocate some memory below 4GB to hold code & stack during 32bit syscall.

Quoted 7 lines
Quoted 3 lines

But some things
will not work properly, e.g. signals because they test the 32bit flag in the task_struct. So you would still get 64bit signal frames.

OK, I can deal with that, since signals are never directly delivered to the client.

Quoted 8 lines
Quoted 5 lines

There are some other such tests, but they probably wont affect you.

There were some plans at one point to allow to toggle the flag using personality or prctl, but so far it hasn't been implemented. There is also no way to do 64bit calls from a 32bit executable.

What about my idea of using distinct ranges of syscall numbers to disambiguate them?

What about this? This does 64bit syscalls from 32bit executable.

Developed/tested on 2.6.10-rc2-mm2. You may want to replace $0x33 with some other value on 2.4.x kernels, but otherwise it should work fine for you. And well, you may want to do some additional coding so you can actually pass & retrieve 64bit values to/from syscalls.

When it works correctly, program should print something and exit with code
111. If it sigsegvs or exits with 222, something went wrong. Petr

vana:~/64bit-test# more syscall.c
#include <stdio.h>
#include <string.h>
#define TOLM                            \
                "pushl %%cs\n"          \
                "pushl $91f\n"          \
                "ljmpl $0x33,$90f\n"    \
                ".code64\n"             \
                "90:\n"
#define FROMLM                          \
                "lretl\n"               \
                ".code32\n"             \
                "91:\n"

unsigned long x8664syscall1(unsigned long op, unsigned long value) {

        unsigned long result;

        asm volatile(
                TOLM
                "mov %1,%1\n"   /* Opteron does not need this, but em64t maybe needs */
                "mov %2,%2\n"
                "syscall\n"
                FROMLM
                : "=a"(result) : "0"(op), "D"(value));
        return result;
}

unsigned long x8664syscall3(unsigned long op, unsigned long v1, unsigned long v2, unsigned long v3) {

        unsigned long result;

        asm volatile(
                TOLM
                "mov %1,%1\n"
                "mov %2,%2\n"
                "mov %3,%3\n"
                "mov %4,%4\n"
                "syscall\n"
                FROMLM
                : "=a"(result) : "0"(op), "D"(v1), "S"(v2), "d"(v3));
        return result;
}
int main(int argc, char* argv[]) {
        printf("Write?\n");
        x8664syscall3(1 /* x86-64 __NR_write */, 1 /* stdout */, (unsigned long)"64-bit rules!\n", 14);
        printf("And now exit(111)\n");
        x8664syscall1(60 /* x86-64 __NR_exit */, 111);
        printf("Exit did not exit...\n");
        return 222;
}
vana:~/64bit-test# gcc -W -Wall -O2 -o syscall syscall.c
syscall.c: In function `main':
syscall.c:43: warning: unused parameter `argc'
syscall.c:43: warning: unused parameter `argv'
vana:~/64bit-test# ./syscall
Write?
64-bit rules!
And now exit(111)
vana:~/64bit-test# echo $?
111
vana:~/64bit-test# file syscall
syscall: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.0, dynamically linked (uses shared libs), not stripped
vana:~/64bit-test# more syscall64.c
#include <stdio.h>
#include <string.h>

unsigned long ia32syscall1(unsigned long op, unsigned long value) {

        unsigned long result;

        asm volatile(
                "int $0x80\n"
                : "=a"(result) : "0"(op), "b"(value));
        return result;
}

unsigned long ia32syscall3(unsigned long op, unsigned long v1, unsigned long v2, unsigned long v3) {

        unsigned long result;

        asm volatile(
                "int $0x80\n"
                : "=a"(result) : "0"(op), "b"(v1), "c"(v2), "d"(v3));
        return result;
}
int main(int argc, char* argv[]) {
        printf("Write?\n");
        ia32syscall3(4 /* 386 __NR_write */, 1 /* stdout */, (unsigned long)"32-bit rules!\n", 14);
        printf("And now exit(111)\n");
        ia32syscall1(1 /* 386 __NR_exit */, 111);
        printf("Exit did not exit...\n");
        return 222;
}
vana:~/64bit-test# x86_64-linux-gcc -W -Wall -O2 -o syscall64 syscall64.c
syscall64.c: In function `main':
syscall64.c:22: warning: unused parameter `argc'
syscall64.c:22: warning: unused parameter `argv'
vana:~/64bit-test# ./syscall64
Write?
32-bit rules!
And now exit(111)
vana:~/64bit-test# echo $?
111
vana:~/64bit-test# file ./syscall64
./syscall64: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.4.0, dynamically linked (uses shared libs), not stripped
vana:~/64bit-test#
#5 lore raw
Quoted 30 lines · Petr Vandrovec
Quoted 5 lines · Jeremy Fitzhardinge
Quoted 1 line · Andi Kleen

int 0x80 from 64bit will call 32bit syscalls.

Hm, that's not what I found. If I run int 0x80 with [re]ax=1, I get exit with a 32-bit process and write with a 64-bit one.

Seems to work for me. See second example below. If it won't work, you can always use reverse procedure to one I used in syscall.c - only problem is that you'll have to allocate some memory below 4GB to hold code & stack during 32bit syscall.

Quoted 7 lines
Quoted 3 lines

But some things
will not work properly, e.g. signals because they test the 32bit flag in the task_struct. So you would still get 64bit signal frames.

OK, I can deal with that, since signals are never directly delivered to the client.

Quoted 8 lines
Quoted 5 lines

There are some other such tests, but they probably wont affect you.

There were some plans at one point to allow to toggle the flag using personality or prctl, but so far it hasn't been implemented. There is also no way to do 64bit calls from a 32bit executable.

What about my idea of using distinct ranges of syscall numbers to disambiguate them?

What about this? This does 64bit syscalls from 32bit executable.

Quoted 7 lines

Developed/tested on 2.6.10-rc2-mm2. You may want to replace $0x33 with some other value on 2.4.x kernels, but otherwise it should work fine for you. And well, you may want to do some additional coding so you can actually pass & retrieve 64bit values to/from syscalls.

When it works correctly, program should print something and exit with code
111. If it sigsegvs or exits with 222, something went wrong.

Both of these work for me on a stock FC3 kernel (2.6.9-1.681_FC3). Not sure what I was doing wrong...

Is 32-from-64 syscalls considered to be a documented API - something which I can rely on?

The 64-from-32 switch is also very interesting; I hadn't realized you could switch modes that easily within a process.

J

#6 lore raw
In reply to Petr Vandrovec
Quoted 4 lines · Petr Vandrovec

Seems to work for me. See second example below. If it won't work, you can always use reverse procedure to one I used in syscall.c - only problem is that you'll have to allocate some memory below 4GB to hold code & stack during 32bit syscall.

I've been playing with this.

If I have a 64-bit process, and I switch to the 32-bit segment with ljmp it works fine - until I get a signal. The kernel doesn't reload cs with the 64-bit segment before calling the handler, which causes an obvious mess. I've been experimenting with setting the handler to this little trampoline:

handler32:
.code32
ljmp *1f
.code64
1: .long handler /* x86-64 handler */

	.word 0x33	/* USER_CS */

but this always gets a SIGSEGV on the ljmp. I don't know why - there are lots of reasons for getting a GPF, and there doesn't seem to be a way to distinguish them.

(Hm, if I change this to the "ljmp $0x33, $handler" form, it gets to handler, but then crashes because rsp has been truncated to 32-bits. Hmm2, OK, so if I set up a 32-bit stack and a signal stack, it all works OK.)

Any ideas? Is this a kernel bug: should it always make sure that CS has the right value before starting a signal handler? I notice that the ia32 kernel reloads cs before starting a signal handler.

J

#7 lore raw
In reply to Petr Vandrovec
Quoted 4 lines
#define TOLM                            \
                "pushl %%cs\n"          \
                "pushl $91f\n"          \
                "ljmpl $0x33,$90f\n"    \

It's useless, there is nothing in the kernel code that checks the 32bit segment.

-Andi