Skip to contentSkip to conversation

64-bit and N32 kernel interfaces

DISCUSSION Ralf Baechle 7 participants 16 messages Sep 4, 2002 – Sep 16, 2002 MIPS

lore
Share & tools
lore
#0 lore raw

(I bcc'ed this to a few people; please followup to [email protected]. I'd like this to be discussed widely before it's cast into stone as everybody will have to live with this for the years to come.)

Right now the 64-bit kernel interfaces are still pretty much an ad-hoc approach and can't be considered frozen. There is now some pressure to come up with a stable 64-bit API asap.

As first think I want to get rid of all the historic crap we have in our syscall tables for the 64-bit syscalls. Let's start here:

#define __NR_syscall (__NR_Linux + 0)

Deprecated because can be implemented in userspace.

#define __NR_ioperm                     (__NR_Linux + 101)
#define __NR_iopl                       (__NR_Linux + 110)
#define __NR_vm86                       (__NR_Linux + 113)

i386 braindamage we're never going to support. So why have it in our syscall table?

#define __NR_unused59                   (__NR_Linux +  59)
#define __NR_reserved82                 (__NR_Linux +  82)
#define __NR_unused109                  (__NR_Linux + 109)
#define __NR_unused150                  (__NR_Linux + 150)

Unused entries. Why keep them ...

#define __NR_break                      (__NR_Linux +  17)
#define __NR_stty                       (__NR_Linux +  31)
#define __NR_gtty                       (__NR_Linux +  32)
#define __NR_ftime                      (__NR_Linux +  35)
#define __NR_prof                       (__NR_Linux +  44)
#define __NR_signal                     (__NR_Linux +  48)
#define __NR_mpx                        (__NR_Linux +  56)
#define __NR_ulimit                     (__NR_Linux +  58)
#define __NR_readdir                    (__NR_Linux +  89)
#define __NR_profil                     (__NR_Linux +  98)
#define __NR_modify_ldt                 (__NR_Linux + 123)

Slots that data back to day one of UNIX way before Linux was born.

#define __NR_socketcall (__NR_Linux + 102)

Wrapper syscall, obsoleted since quite a while in the 32-bit kernel.

#define __NR_idle (__NR_Linux + 112)

Internal syscall, no longer used.

#define __NR_ipc (__NR_Linux + 117)

Yet another multiplexor syscall and imho another candidate for getting rid of.

#define __NR_oldstat                    (__NR_Linux +  18)
#define __NR_umount                     (__NR_Linux +  22)
#define __NR_oldfstat                   (__NR_Linux +  28)
#define __NR_oldlstat                   (__NR_Linux +  84)

Superseeded by newer versions.

#define __NR_uselib (__NR_Linux + 86)

a.out support. Do we really want that.

I probably missed a few. The primary purpose of this posting is to get a discussion about the 64-bit syscall interface started. It's still not cast into stone so we can modify it as we see fit. The entire syscall interface is still open for changes, this includes all structures etc. Along with a 64-bit ABI we'll also have to deciede about a N32 ABI.

Suggestions, comments etc?

Ralf

#1 lore raw
Quoted 3 lines · Ralf Baechle

Right now the 64-bit kernel interfaces are still pretty much an ad-hoc approach and can't be considered frozen. There is now some pressure to come up with a stable 64-bit API asap.

Well, they are not really used by anything, yet so they are far from
being frozen.

Quoted 2 lines

As first think I want to get rid of all the historic crap we have in our syscall tables for the 64-bit syscalls. Let's start here:

[...]

Definitely.

Quoted 5 lines

I probably missed a few. The primary purpose of this posting is to get a discussion about the 64-bit syscall interface started. It's still not cast into stone so we can modify it as we see fit. The entire syscall interface is still open for changes, this includes all structures etc. Along with a 64-bit ABI we'll also have to deciede about a N32 ABI.

It would be nice if we could keep a single set of syscalls for both (n)64
and n32. The address crop for n32 may be handled the Alpha way. I will investigate the topic soon.

Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#2 lore raw
Quoted 9 lines · Maciej W. Rozycki
Quoted 5 lines

I probably missed a few. The primary purpose of this posting is to get a discussion about the 64-bit syscall interface started. It's still not cast into stone so we can modify it as we see fit. The entire syscall interface is still open for changes, this includes all structures etc. Along with a 64-bit ABI we'll also have to deciede about a N32 ABI.

It would be nice if we could keep a single set of syscalls for both (n)64
and n32. The address crop for n32 may be handled the Alpha way. I will investigate the topic soon.

Can you describe how this is handled on the Alpha?

The primary problem is the differnet calling sequence for o32 and N64. As it looks we'll be able to use either the o32 function or the native syscall to implement all of the necessary N32 syscalls.

The question is if we want to reserve another 1000 entries in our already huge syscall table for N32 or if we got a better solution ...

Ralf

#3 lore raw
Quoted 5 lines · Ralf Baechle
Quoted 3 lines

It would be nice if we could keep a single set of syscalls for both (n)64
and n32. The address crop for n32 may be handled the Alpha way. I will investigate the topic soon.

Can you describe how this is handled on the Alpha?

I'm referring mostly to OSF/1 here as it was first to implement it.
Linux followed it in the sense it is able to execute OSF/1 binaries marked as "32-bit", but native ELF binaries used to be fully 64-bit always. I think by a popular demand GNU binutils are now able to create "cropped" Alpha/Linux ELF binaries as well, but this is unverified for sure. The implementation is two-fold.

First, the static linker (if given the "-taso" option) maps an executable
into the low 31-bit address space (coincidentally, this will probably be suitable for MIPS as well) and sets a special flag in the executable (it does it in a weird place, but this is ECOFF and we have suitable flags in the ELF header already).

Second, seeing the "31-bit" flag set, the kernel returns any maps
requested within the low 31-bit address space. This way both shared libraries (which thus need not be special, i.e. may be regular 64-bit ones) and areas allocated by mmap() are addressable by the executable.

To summarize, nothing much complicated.

Quoted 1 line

The primary problem is the differnet calling sequence for o32 and N64.

But we handle that already.

Quoted 2 lines

As it looks we'll be able to use either the o32 function or the native syscall to implement all of the necessary N32 syscalls.

The (n)64 versions seem suitable and the o32 ones do not as n32 only
crops addresses to 32-bit -- data may still be 64-bit (e.g. file position pointers).

Quoted 2 lines

The question is if we want to reserve another 1000 entries in our already huge syscall table for N32 or if we got a better solution ...

Aaarrgh, no more entries, please...

Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#4 lore raw
Quoted 3 lines · Maciej W. Rozycki
Quoted 1 line

The primary problem is the differnet calling sequence for o32 and N64.

But we handle that already.

Well, N32 is yet another case. Have to look into details again but some MIPS guy recently pointed out to me that there are a few syscalls which for N32 cannot be handled by the o32 or N64 syscall entry as they are right now.

Quoted 4 lines
Quoted 2 lines

The question is if we want to reserve another 1000 entries in our already huge syscall table for N32 or if we got a better solution ...

Aaarrgh, no more entries, please...

Understandable reflex :)

Btw, I did just chat with a Redhat guy. They say they're basically finished and are just waiting for us. Which means we need to deciede fast ...

I'll be on the Linux Kongress for the next few days so I'll not be able to answer as quickly as I'd like to but I'll try to read me email as often as I can.

Ralf

#5 lore raw
Quoted 4 lines · Ralf Baechle

Well, N32 is yet another case. Have to look into details again but some MIPS guy recently pointed out to me that there are a few syscalls which for N32 cannot be handled by the o32 or N64 syscall entry as they are right now.

Which ones? Maybe we might just add some padding to structures.

Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#6 lore raw
In reply to Maciej W. Rozycki
Quoted 30 lines · Maciej W. Rozycki
Quoted 5 lines · Ralf Baechle
Quoted 3 lines

It would be nice if we could keep a single set of syscalls for both (n)64
and n32. The address crop for n32 may be handled the Alpha way. I will investigate the topic soon.

Can you describe how this is handled on the Alpha?

I'm referring mostly to OSF/1 here as it was first to implement it.
Linux followed it in the sense it is able to execute OSF/1 binaries marked as "32-bit", but native ELF binaries used to be fully 64-bit always. I think by a popular demand GNU binutils are now able to create "cropped" Alpha/Linux ELF binaries as well, but this is unverified for sure. The implementation is two-fold.

First, the static linker (if given the "-taso" option) maps an executable
into the low 31-bit address space (coincidentally, this will probably be suitable for MIPS as well) and sets a special flag in the executable (it does it in a weird place, but this is ECOFF and we have suitable flags in the ELF header already).

Second, seeing the "31-bit" flag set, the kernel returns any maps
requested within the low 31-bit address space. This way both shared libraries (which thus need not be special, i.e. may be regular 64-bit ones) and areas allocated by mmap() are addressable by the executable.

To summarize, nothing much complicated.

Quoted 1 line

The primary problem is the differnet calling sequence for o32 and N64.

Quoted 9 lines

But we handle that already.

Quoted 2 lines

As it looks we'll be able to use either the o32 function or the native syscall to implement all of the necessary N32 syscalls.

The (n)64 versions seem suitable and the o32 ones do not as n32 only
crops addresses to 32-bit -- data may still be 64-bit (e.g. file position pointers).

Please notice, that a 'long' is 32-bit for n32, so we need to do the same conversion for a lot of syscalls, as we already do for o32.

Review

Quoted 10 lines
Quoted 3 lines

The question is if we want to reserve another 1000 entries in our already huge syscall table for N32 or if we got a better solution ...

Aaarrgh, no more entries, please...

--

(quoted diff) +3 -0
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +

--

_    _ ____  ___   Carsten Langgaard   Mailto:[email protected]
|\  /|||___)(___   MIPS Denmark        Direct: +45 4486 5527
| \/ |||    ____)  Lautrupvang 4B      Switch: +45 4486 5555
  TECHNOLOGIES     2750 Ballerup       Fax...: +45 4486 5556
                   Denmark             http://www.mips.com
#7 lore raw
Quoted 6 lines · Carsten Langgaard
Quoted 3 lines

The (n)64 versions seem suitable and the o32 ones do not as n32 only
crops addresses to 32-bit -- data may still be 64-bit (e.g. file position pointers).

Please notice, that a 'long' is 32-bit for n32, so we need to do the same conversion for a lot of syscalls, as we already do for o32.

Any reference? AFAIK, long is 64-bit for n32 and only void * is 32-bit.
It doesn't make sense otherwise.

Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#8 lore raw
Quoted 11 lines · Maciej W. Rozycki
Quoted 6 lines · Carsten Langgaard
Quoted 3 lines

The (n)64 versions seem suitable and the o32 ones do not as n32 only
crops addresses to 32-bit -- data may still be 64-bit (e.g. file position pointers).

Please notice, that a 'long' is 32-bit for n32, so we need to do the same conversion for a lot of syscalls, as we already do for o32.

Any reference? AFAIK, long is 64-bit for n32 and only void * is 32-bit.
It doesn't make sense otherwise.

Nope, it looks like long is 32-bit according to GCC. It means that 64-bit quantities (long long, structures) can go in integer registers.

Signature · 2 lines
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer
#9 lore raw
In reply to Ralf Baechle
Quoted 5 lines · Ralf Baechle

On Wed, Sep 04, 2002 at 04:14:13PM +0200, Maciej W. Rozycki wrote:

The primary problem is the differnet calling sequence for o32 and N64. As it looks we'll be able to use either the o32 function or the native syscall to implement all of the necessary N32 syscalls.

For 64bit kernel, do we intend to have one syscall table that support o32, n32 and n64 altogether? Or we will have multiple tables for them?

Quoted 4 lines

The question is if we want to reserve another 1000 entries in our already huge syscall table for N32 or if we got a better solution ...

It seems n32 can be naturally implemented through n64 syscalls, although I am sure there are some nasty details to work out.

Where can I find n32/n64 spec?

Jun

#10 lore raw
Quoted 2 lines · Jun Sun

For 64bit kernel, do we intend to have one syscall table that support o32, n32 and n64 altogether? Or we will have multiple tables for them?

There are two tables now, one for o32, currently used when executing o32
userland and the other one ready for native userland. They can't be unified as their semantics differ.

Quoted 1 line

Where can I find n32/n64 spec?

Search 'http://techpubs.sgi.com/' and see
'ftp://oss.sgi.com/pub/linux/mips/doc/ABI/'.

Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#11 lore raw
In reply to Jun Sun
Quoted 6 lines · Jun Sun
Quoted 3 lines

The primary problem is the differnet calling sequence for o32 and N64. As it looks we'll be able to use either the o32 function or the native syscall to implement all of the necessary N32 syscalls.

For 64bit kernel, do we intend to have one syscall table that support o32, n32 and n64 altogether? Or we will have multiple tables for them?

Several approach to solve that problem. Adding another 1000 entries - which will cost 8000 bytes of memory that will be mostly zeros. Having wrappers for each function that do the appropriate argument and result convertion is another. etc.

Quoted 5 lines
Quoted 2 lines

The question is if we want to reserve another 1000 entries in our already huge syscall table for N32 or if we got a better solution ...

It seems n32 can be naturally implemented through n64 syscalls, although I am sure there are some nasty details to work out.

Unfortunately there are ...

Quoted 1 line

Where can I find n32/n64 spec?

mipsabi.org which is no longer online. Anyway, I don't think there is a formal published N32 spec. And this whole thread is about the syscall interface. That isn't part of any ABI spec.

Ralf

#12 lore raw

On Wed, 4 Sep 2002, Ralf Baechle wrote:

...

Quoted 4 lines

Several approach to solve that problem. Adding another 1000 entries - which will cost 8000 bytes of memory that will be mostly zeros. Having wrappers for each function that do the appropriate argument and result convertion is another. etc.

Ralf, that's not true. In our current 64-bit kernel we have 234 entries for N64 which uses 8*234 = 1872 bytes. N32 has 241 entries using 1928 bytes. Even in an embedded environment that's not a lot of memory.

/Kjeld

Signature · 5 lines
_    _ ____  ___                       Mailto:[email protected]
|\  /|||___)(___    MIPS Denmark       Direct: +45 44 86 55 85
| \/ |||    ____)   Lautrupvang 4 B    Switch: +45 44 86 55 55
  TECHNOLOGIES      DK-2750 Ballerup   Fax...: +45 44 86 55 56
                    Denmark            http://www.mips.com/
#13 lore raw
In reply to Ralf Baechle
Quoted 3 lines · Ralf Baechle

#define __NR_uselib (__NR_Linux + 86)

a.out support. Do we really want that.

Well, it's support for Linux-a.out-style shared libraries, which are actually binary-format independent. Quick summary of how they work in ELF:

The file argument to uselib must have 1 or 2 program headers. Exactly one of them must be PT_LOAD. That segment is loaded at the fixed virtual address specified in in the header. It's marked readable, writable, executable, and any BSS region is zeroed.

I contemplated using uselib(2) in snow and decided that I wanted multiple segments to support text and rodata being read-only. I figured that attempting to communicate the read-only nature of the maps to the VM could elicit more efficient behavior.

Anyway, now that we have ELF interpreters you can get one library loaded into core for you by the kernel. That library can define a more reasonable version of uselib in userspace...

I guess my point is that even the tiny set of people doing statically linked shared libraries will probably avoid this syscall.

Jay

#14 lore raw
In reply to Ralf Baechle

On Wed, Sep 04, 2002 at 03:56:45PM +0200, Ralf Baechle wrote:

Additional changes I've done to my working version:

Quoted 21 lines

As first think I want to get rid of all the historic crap we have in our syscall tables for the 64-bit syscalls. Let's start here:

#define __NR_syscall (__NR_Linux + 0)

Deprecated because can be implemented in userspace.

#define __NR_ioperm                     (__NR_Linux + 101)
#define __NR_iopl                       (__NR_Linux + 110)
#define __NR_vm86                       (__NR_Linux + 113)

i386 braindamage we're never going to support. So why have it in our syscall table?

#define __NR_unused59                   (__NR_Linux +  59)
#define __NR_reserved82                 (__NR_Linux +  82)
#define __NR_unused109                  (__NR_Linux + 109)
#define __NR_unused150                  (__NR_Linux + 150)

Unused entries. Why keep them ...

Quoted 11 lines
#define __NR_break                      (__NR_Linux +  17)
#define __NR_stty                       (__NR_Linux +  31)
#define __NR_gtty                       (__NR_Linux +  32)
#define __NR_ftime                      (__NR_Linux +  35)
#define __NR_prof                       (__NR_Linux +  44)
#define __NR_signal                     (__NR_Linux +  48)
#define __NR_mpx                        (__NR_Linux +  56)
#define __NR_ulimit                     (__NR_Linux +  58)
#define __NR_readdir                    (__NR_Linux +  89)
#define __NR_profil                     (__NR_Linux +  98)
#define __NR_modify_ldt                 (__NR_Linux + 123)

One more for the same cathegory:

#define __NR_lock (__NR_Linux + 53)

Quoted 11 lines

Slots that data back to day one of UNIX way before Linux was born.

#define __NR_socketcall (__NR_Linux + 102)

Wrapper syscall, obsoleted since quite a while in the 32-bit kernel.

#define __NR_idle (__NR_Linux + 112)

Internal syscall, no longer used.

#define __NR_ipc (__NR_Linux + 117)

This implies eleven new entries for:

__NR_semget
__NR_semop
__NR_semctl
__NR_msgget
__NR_msgsnd
__NR_msgrcv
__NR_msgctl
__NR_shmget
__NR_shmat
__NR_shmdt
__NR_shmctl

Quoted 14 lines

Yet another multiplexor syscall and imho another candidate for getting rid of.

#define __NR_oldstat                    (__NR_Linux +  18)
#define __NR_umount                     (__NR_Linux +  22)
#define __NR_oldfstat                   (__NR_Linux +  28)
#define __NR_oldlstat                   (__NR_Linux +  84)

Superseeded by newer versions.

#define __NR_uselib (__NR_Linux + 86)

a.out support. Do we really want that.

Quoted 7 lines

I probably missed a few. The primary purpose of this posting is to get a discussion about the 64-bit syscall interface started. It's still not cast into stone so we can modify it as we see fit. The entire syscall interface is still open for changes, this includes all structures etc. Along with a 64-bit ABI we'll also have to deciede about a N32 ABI.

Suggestions, comments etc?

Ralf

#15 lore raw

On Mon, Sep 16, 2002 at 03:01:52PM +0200, Ralf Baechle wrote:

More chainswing ...

Quoted 51 lines
Quoted 21 lines

As first think I want to get rid of all the historic crap we have in our syscall tables for the 64-bit syscalls. Let's start here:

#define __NR_syscall (__NR_Linux + 0)

Deprecated because can be implemented in userspace.

#define __NR_ioperm                     (__NR_Linux + 101)
#define __NR_iopl                       (__NR_Linux + 110)
#define __NR_vm86                       (__NR_Linux + 113)

i386 braindamage we're never going to support. So why have it in our syscall table?

#define __NR_unused59                   (__NR_Linux +  59)
#define __NR_reserved82                 (__NR_Linux +  82)
#define __NR_unused109                  (__NR_Linux + 109)
#define __NR_unused150                  (__NR_Linux + 150)

Unused entries. Why keep them ...

Quoted 11 lines
#define __NR_break                      (__NR_Linux +  17)
#define __NR_stty                       (__NR_Linux +  31)
#define __NR_gtty                       (__NR_Linux +  32)
#define __NR_ftime                      (__NR_Linux +  35)
#define __NR_prof                       (__NR_Linux +  44)
#define __NR_signal                     (__NR_Linux +  48)
#define __NR_mpx                        (__NR_Linux +  56)
#define __NR_ulimit                     (__NR_Linux +  58)
#define __NR_readdir                    (__NR_Linux +  89)
#define __NR_profil                     (__NR_Linux +  98)
#define __NR_modify_ldt                 (__NR_Linux + 123)

One more for the same cathegory:

#define __NR_lock (__NR_Linux + 53)

Quoted 11 lines

Slots that data back to day one of UNIX way before Linux was born.

#define __NR_socketcall (__NR_Linux + 102)

Wrapper syscall, obsoleted since quite a while in the 32-bit kernel.

#define __NR_idle (__NR_Linux + 112)

Internal syscall, no longer used.

#define __NR_ipc (__NR_Linux + 117)

This implies eleven new entries for:

Quoted 31 lines

__NR_semget
__NR_semop
__NR_semctl
__NR_msgget
__NR_msgsnd
__NR_msgrcv
__NR_msgctl
__NR_shmget
__NR_shmat
__NR_shmdt
__NR_shmctl

Quoted 19 lines

Yet another multiplexor syscall and imho another candidate for getting rid of.

#define __NR_oldstat                    (__NR_Linux +  18)
#define __NR_umount                     (__NR_Linux +  22)
#define __NR_oldfstat                   (__NR_Linux +  28)
#define __NR_oldlstat                   (__NR_Linux +  84)

Superseeded by newer versions.

#define __NR_uselib (__NR_Linux + 86)

a.out support. Do we really want that.

I probably missed a few. The primary purpose of this posting is to get a discussion about the 64-bit syscall interface started. It's still not cast into stone so we can modify it as we see fit. The entire syscall interface is still open for changes, this includes all structures etc. Along with a 64-bit ABI we'll also have to deciede about a N32 ABI.

llseek, pread64/pwrite64, getdent64 are no longer needed as their standard counterparts are already 64 bit.

Modern libc doesn't use the old style signal calls __NR_sigaction, __NR_sigsuspend, __NR_sigpending, __NR_sigprocmask. That means __NR_sigreturn can also go.

__NR_sgetmask and __NR_ssetmask are only capable of dealing with signal masks of at most bitsof(long) and thus have become useless and replaced by sigprocmask(2).

__NR_waitpid can be implemented on top of wait(2).

__NR_stime can be implemented on top of settimeofday(2).

__NR_nice can be implemented on top of getpriority / setpriority and was a stupid interface anyway.

__NR_recv can be implemented on recvfrom(2) and __NR_send using sendfrom(2).

Ralf