Skip to contentSkip to conversation

x86_64: 32bit emulation problems

DISCUSSION Bernd Schubert 4 participants 5 messages Feb 28, 2005 – Mar 1, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Sorry, I'm sending it here again, since my first mail, in which I CCed LKML and Andi Kleen was refused by the listserver.

Hi,

I'm just looking into a very strange problem. Some of our systems have athlon64 CPUs. Due to our diskless nfs environment we currently still prefer a 32bit userspace environment, but would like to be able to use a 64-bit chroot environment.

Well, currently there seems to be a stat64() NFS problem when a x86_64 kernel is booted and stat64() comes from a 32bit libc.

Here's just an example:

hitchcock:/home/bernd/src/tests# ./test_stat64 /mnt/test/yp
stat() works fine.
hitchcock:/home/bernd/src/tests# ./test_stat32 /mnt/test/yp
stat for /mnt/test/yp failed 

The test program looks rather simple:

#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>

int main(int argc, char **argv)

{
        char *dir;
        struct stat buf;

        dir = argv[1];

        if (stat (dir, &buf) == -1)
                fprintf(stderr, "stat for %s failed \n", dir);
        else
                fprintf(stderr, "stat() works fine.\n");
        return (0);
}

Here are the strace outputs:
=====================

32bit:
------

hitchcock:/home/bernd/src/tests# strace32 ./test_stat32 /mnt/test/yp
execve("./test_stat32", ["./test_stat32", "/mnt/test/yp"], [/* 39 vars */]) = 
0
uname({sys="Linux", node="hitchcock", ...}) = 0
brk(0)                                  = 0x80ad000
brk(0x80ce000)                          = 0x80ce000
stat64("/mnt/test/yp", {st_mode=S_IFDIR|0755, st_size=2704, ...}) = 0
write(2, "stat for /mnt/test/yp failed \n", 30stat for /mnt/test/yp failed 
) = 30
exit_group(0)                           = ?

64bit:
-------

hitchcock:/home/bernd/src/tests# strace ./test_stat64 /mnt/test/yp
execve("./test_stat64", ["./test_stat64", "/mnt/test/yp"], [/* 39 vars */]) = 
0
uname({sys="Linux", node="hitchcock", ...}) = 0
brk(0)                                  = 0x572000
brk(0x593000)                           = 0x593000
stat("/mnt/test/yp", {st_mode=S_IFDIR|0755, st_size=2704, ...}) = 0
write(2, "stat() works fine.\n", 19stat() works fine.
)    = 19
_exit(0)                                = ?

Anyone having an idea whats going on? The ethereal capture also looks pretty normal. The kernel of this system is 2.6.9, but it also happens on another system with 2.6.11-rc5.
As usual we are using unfs3 for /etc and /var, but for me that looks like a client problem. I'm now also pretty sure that its not a problem for a local mount point (so a local disk).

Thanks in advance,
Bernd

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#1 lore raw

Given that strace shows that both syscalls (stat64() and stat()) succeed, I expect the "problem" is probably just glibc setting an EOVERFLOW error in the 32-bit case. That's what it is supposed to do if a 64 bit value overflows the 32-bit buffers.

Have you tried looking at errno?

Cheers,
Trond

m=E5 den 28.02.2005 Klokka 22:08 (+0100) skreiv Bernd Schubert:

Quoted 1 line

Sorry, I'm sending it here again, since my first mail, in which I CCed LK=

ML=20

Quoted 7 lines

and Andi Kleen was refused by the listserver.
=20
=20
Hi,
=20
I'm just looking into a very strange problem. Some of our systems have=20 athlon64 CPUs. Due to our diskless nfs environment we currently still pre=

fer=20

Quoted 4 lines

a 32bit userspace environment, but would like to be able to use a 64-bit=20 chroot environment.
=20
Well, currently there seems to be a stat64() NFS problem when a x86_64 k=

ernel=20

Quoted 37 lines

is booted and stat64() comes from a 32bit libc.
=20
Here's just an example:
=20

hitchcock:/home/bernd/src/tests# ./test_stat64 /mnt/test/yp
stat() works fine.
=20
=20
hitchcock:/home/bernd/src/tests# ./test_stat32 /mnt/test/yp
stat for /mnt/test/yp failed=20
=20
=20
The test program looks rather simple:
=20
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
=20
=20
int main(int argc, char **argv)
{
        char *dir;
        struct stat buf;
=20
        dir =3D argv[1];
=20
        if (stat (dir, &buf) =3D=3D -1)
                fprintf(stderr, "stat for %s failed \n", dir);
        else
                fprintf(stderr, "stat() works fine.\n");
        return (0);
}
=20
Quoted 8 lines
=20
Here are the strace outputs:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=20
32bit:
------
hitchcock:/home/bernd/src/tests# strace32 ./test_stat32 /mnt/test/yp
execve("./test_stat32", ["./test_stat32", "/mnt/test/yp"], [/* 39 vars */=

]) =3D=20

Quoted 5 lines

0
uname({sys=3D"Linux", node=3D"hitchcock", ...}) =3D 0
brk(0) =3D 0x80ad000
brk(0x80ce000) =3D 0x80ce000
stat64("/mnt/test/yp", {st_mode=3DS_IFDIR|0755, st_size=3D2704, ...}) =3D=

0

Quoted 1 line

write(2, "stat for /mnt/test/yp failed \n", 30stat for /mnt/test/yp faile=

d=20

Quoted 7 lines

) =3D 30
exit_group(0) =3D ?
=20
64bit:
-------

hitchcock:/home/bernd/src/tests# strace ./test_stat64 /mnt/test/yp
execve("./test_stat64", ["./test_stat64", "/mnt/test/yp"], [/* 39 vars */=

]) =3D=20

Quoted 4 lines

0
uname({sys=3D"Linux", node=3D"hitchcock", ...}) =3D 0
brk(0) =3D 0x572000
brk(0x593000) =3D 0x593000

Quoted 8 lines

stat("/mnt/test/yp", {st_mode=3DS_IFDIR|0755, st_size=3D2704, ...}) =3D 0 write(2, "stat() works fine.\n", 19stat() works fine.
) =3D 19
_exit(0) =3D ?
=20
=20
=20
Anyone having an idea whats going on? The ethereal capture also looks pre=

tty=20

Quoted 1 line

normal. The kernel of this system is 2.6.9, but it also happens on anothe=

r=20

Quoted 2 lines

system with 2.6.11-rc5.
As usual we are using unfs3 for /etc and /var, but for me that looks like=

a=20

Quoted 1 line

client problem. I'm now also pretty sure that its not a problem for a loc=

al=20

Quoted 7 lines

mount point (so a local disk).
=20
=20
Thanks in advance,
Bernd
=20
=20

Quoted 8 lines

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

--=20
Trond Myklebust <[email protected]>

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#2 lore raw
Quoted 4 lines · you

Given that strace shows that both syscalls (stat64() and stat()) succeed, I expect the "problem" is probably just glibc setting an EOVERFLOW error in the 32-bit case. That's what it is supposed to do if a 64 bit value overflows the 32-bit buffers.

Right, thanks.

Quoted 3 lines

Have you tried looking at errno?

bernd@hitchcock tests>./test_stat32 /mnt/test/yp
stat for /mnt/test/yp failed
ernno: 75 (Value too large for defined data type)

But why does stat64() on a 64-bit kernel tries to fill in larger data than on a 32-bit kernel and larger data also only for nfs-mount points? Hmm, I will tomorrow compare the tcp-packges sent by the server.

Many thanks,
Bernd

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#3 lore raw
In reply to Trond Myklebust
Quoted 4 lines · you

Given that strace shows that both syscalls (stat64() and stat()) succeed, I expect the "problem" is probably just glibc setting an EOVERFLOW error in the 32-bit case. That's what it is supposed to do if a 64 bit value overflows the 32-bit buffers.

Right, thanks.

Quoted 3 lines

Have you tried looking at errno?

bernd@hitchcock tests>./test_stat32 /mnt/test/yp
stat for /mnt/test/yp failed
ernno: 75 (Value too large for defined data type)

But why does stat64() on a 64-bit kernel tries to fill in larger data than on a 32-bit kernel and larger data also only for nfs-mount points? Hmm, I will tomorrow compare the tcp-packges sent by the server.

Many thanks,
Bernd

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#4 lore raw
In reply to Bernd Schubert

Sounds like a good time to try out Xen or similar:

http://dcs.nac.uci.edu/~strombrg/multios.html

Quoted 102 lines · Bernd Schubert

Sorry, I'm sending it here again, since my first mail, in which I CCed LKML and Andi Kleen was refused by the listserver.

Hi,

I'm just looking into a very strange problem. Some of our systems have athlon64 CPUs. Due to our diskless nfs environment we currently still prefer a 32bit userspace environment, but would like to be able to use a 64-bit chroot environment.

Well, currently there seems to be a stat64() NFS problem when a x86_64 kernel is booted and stat64() comes from a 32bit libc.

Here's just an example:

hitchcock:/home/bernd/src/tests# ./test_stat64 /mnt/test/yp
stat() works fine.
hitchcock:/home/bernd/src/tests# ./test_stat32 /mnt/test/yp
stat for /mnt/test/yp failed 

The test program looks rather simple:

#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>

int main(int argc, char **argv)

{
        char *dir;
        struct stat buf;

        dir = argv[1];

        if (stat (dir, &buf) == -1)
                fprintf(stderr, "stat for %s failed \n", dir);
        else
                fprintf(stderr, "stat() works fine.\n");
        return (0);
}

Here are the strace outputs:
=====================

32bit:
------

hitchcock:/home/bernd/src/tests# strace32 ./test_stat32 /mnt/test/yp
execve("./test_stat32", ["./test_stat32", "/mnt/test/yp"], [/* 39 vars */]) = 
0
uname({sys="Linux", node="hitchcock", ...}) = 0
brk(0)                                  = 0x80ad000
brk(0x80ce000)                          = 0x80ce000
stat64("/mnt/test/yp", {st_mode=S_IFDIR|0755, st_size=2704, ...}) = 0
write(2, "stat for /mnt/test/yp failed \n", 30stat for /mnt/test/yp failed 
) = 30
exit_group(0)                           = ?

64bit:
-------

hitchcock:/home/bernd/src/tests# strace ./test_stat64 /mnt/test/yp
execve("./test_stat64", ["./test_stat64", "/mnt/test/yp"], [/* 39 vars */]) = 
0
uname({sys="Linux", node="hitchcock", ...}) = 0
brk(0)                                  = 0x572000
brk(0x593000)                           = 0x593000
stat("/mnt/test/yp", {st_mode=S_IFDIR|0755, st_size=2704, ...}) = 0
write(2, "stat() works fine.\n", 19stat() works fine.
)    = 19
_exit(0)                                = ?

Anyone having an idea whats going on? The ethereal capture also looks pretty normal. The kernel of this system is 2.6.9, but it also happens on another system with 2.6.11-rc5.
As usual we are using unfs3 for /etc and /var, but for me that looks like a client problem. I'm now also pretty sure that its not a problem for a local mount point (so a local disk).

Thanks in advance,
Bernd

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs