[uml-devel] Programs in guest segfault, amd64 host and guest

Jan Dittmer

author

2006-10-18 15:34 UTC #0 lore raw

- Guest 2.6.19-rc1-git11
- Host Debian 2.6.18-1-amd64
- Debian etch for host and guest
- both using ext3
- command to start the guest

./linux-2.6.19-rc1-git11 ubd0=umlblob.cow,etch-amd64.img \
    eth0=daemon,10:00:00:00:00:02,unix,/var/run/uml-utilities/uml_switch.ctl \
         mem=128M \
         con=null umid=umlprey \
         >/dev/null 2>&1 &

$ strace btlaunchmany /tmp/
...
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|0x40, 
-1, 0) = 0x41c4e000
mprotect(0x41c4e000, 4096, PROT_NONE)   = 0
clone(Process 484 attached
child_stack=0x4244e280, 
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, 
parent_tidptr=0x4244e9f0, tls=0x4244e960, child_tidptr=0x4244e9f0) = 484
[pid   483] rt_sigprocmask(SIG_SETMASK, [],  <unfinished ...>
[pid   484] --- SIGSEGV (Segmentation fault) @ 0 (0) ---
Process 484 detached
<... rt_sigprocmask resumed> NULL, 8)   = 0
+++ killed by SIGSEGV +++
Process 483 detached

$ strace mlnet -stdout
...
gettimeofday({1161185543, 817872}, NULL) = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE|PROT_EXEC, 
MAP_PRIVATE|MAP_ANONYMOUS|0x40, -1, 0) = 0x41ccd000
mprotect(0x41ccd000, 4096, PROT_NONE)   = 0
clone(child_stack=0x424cd270, 
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, 
parent_tidptr=0x424cd9f0, tls=0x424cd960, child_tidptr=0x424cd9f0) = 502
gettimeofday({1161185543, 820912}, NULL) = 0
poll([{fd=7, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}, 
{fd=4, events=POLLIN}, {fd=3, events=POLLIN}], 5, 96) = -1 EINTR 
(Interrupted system call)
+++ killed by SIGSEGV +++
Process 501 detached

Additionally I'm having build failure compiling ARCH=um SUBARCH=i386.

Thanks for any pointers,

Jan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Jeff Dike

2006-10-18 19:11 UTC #1 lore raw

Quoted 9 lines · Jan Dittmer

clone(Process 484 attached
child_stack=0x4244e280, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0x4244e9f0, tls=0x4244e960, child_tidptr=0x4244e9f0) = 484
[pid 483] rt_sigprocmask(SIG_SETMASK, [], <unfinished ...>
[pid 484] --- SIGSEGV (Segmentation fault) @ 0 (0) ---
Process 484 detached
<... rt_sigprocmask resumed> NULL, 8) = 0
+++ killed by SIGSEGV +++

I suspect it's due to UML/x86_64 having incomplete NPTL support, although I've had almost no problems with up-to-date Fedora filesystems on x86_64. The one thing I do see is yum segfaulting occassionally, but I haven't been able to tie that to NPTL.

Jeff

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Jan Dittmer

author

2006-10-18 19:17 UTC #2 lore raw

Quoted 15 lines · Jeff Dike

Quoted 9 lines · Jan Dittmer
clone(Process 484 attached
child_stack=0x4244e280, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0x4244e9f0, tls=0x4244e960, child_tidptr=0x4244e9f0) = 484
[pid 483] rt_sigprocmask(SIG_SETMASK, [], <unfinished ...>
[pid 484] --- SIGSEGV (Segmentation fault) @ 0 (0) ---
Process 484 detached
<... rt_sigprocmask resumed> NULL, 8) = 0
+++ killed by SIGSEGV +++
I suspect it's due to UML/x86_64 having incomplete NPTL support, although I've had almost no problems with up-to-date Fedora filesystems on x86_64. The one thing I do see is yum segfaulting occassionally, but I haven't been able to tie that to NPTL.

This is 100% reproducible here. Perfect test case. How to debug?

Jan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Nikola Ciprich

2006-10-18 21:03 UTC #3 lore raw

try exporting LD_ASSUME_KERNEL=2.4.1 before executing Your application, to see if that works - this disables usage of NPTL

Quoted 24 lines · Jan Dittmer

Quoted 17 lines · Jeff Dike
Quoted 10 lines · Jan Dittmer
clone(Process 484 attached
child_stack=0x4244e280, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0x4244e9f0, tls=0x4244e960, child_tidptr=0x4244e9f0) = 484
[pid 483] rt_sigprocmask(SIG_SETMASK, [], <unfinished ...>
[pid 484] --- SIGSEGV (Segmentation fault) @ 0 (0) ---
Process 484 detached
<... rt_sigprocmask resumed> NULL, 8) = 0
+++ killed by SIGSEGV +++
I suspect it's due to UML/x86_64 having incomplete NPTL support, although I've had almost no problems with up-to-date Fedora filesystems on x86_64. The one thing I do see is yum segfaulting occassionally, but I haven't been able to tie that to NPTL.
This is 100% reproducible here. Perfect test case. How to debug?
Jan

Quoted 10 lines

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Jeff Dike

2006-10-18 21:06 UTC #4 lore raw

In reply to Jan Dittmer

Quoted 1 line · Jan Dittmer

This is 100% reproducible here. Perfect test case. How to debug?

You want to dig into the code and see what's wrong? Put a breakpoint on the force_sig_info(SIGSEGV, ...) in segv(). Then find the faulting instruction -
RIP is regs.skas.regs[16]
pid = cpu_tasks[0].pid
Look in the host's /proc/$pid/maps for the page containing RIP add uml_physmem to the offset in column 3 - that's the
physical page containing RIP
add the RIP page offset to that to get the actual physical
address for RIP
disassemble the range of memory around it - I usually do 10 -
20 bytes before it to ~10 bytes after
see that the asm looks sane, if not, bump the start of the
disassembly by a byte in either direction until it does
see what the instruction at RIP is - if it involves a gs:
reference, then it's very likely a NPTL problem

If you don't want to do that, tell me what to yum, and how to run whatever it is, and I'll look at it.

Jeff

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Jan Dittmer

author

2006-10-18 21:26 UTC #5 lore raw

Quoted 19 lines · Jeff Dike

Quoted 1 line · Jan Dittmer
This is 100% reproducible here. Perfect test case. How to debug?
You want to dig into the code and see what's wrong? Put a breakpoint on the force_sig_info(SIGSEGV, ...) in segv(). Then find the faulting instruction -
RIP is regs.skas.regs[16]
pid = cpu_tasks[0].pid
Look in the host's /proc/$pid/maps for the page containing RIP add uml_physmem to the offset in column 3 - that's the
physical page containing RIP
add the RIP page offset to that to get the actual physical
address for RIP
disassemble the range of memory around it - I usually do 10 -
20 bytes before it to ~10 bytes after
see that the asm looks sane, if not, bump the start of the
disassembly by a byte in either direction until it does
see what the instruction at RIP is - if it involves a gs:
reference, then it's very likely a NPTL problem

Well tomorrow perhaps. I've very little time currently :-(

Quoted 2 lines

If you don't want to do that, tell me what to yum, and how to run whatever it is, and I'll look at it.

As I wrote in the original message: Debian/etch amd64 host+guest, running btlaunchmany (from the bittorrent package). I can put you the compressed root + kernel somewhere if that helps. It's roughly 100mb.

Jan

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Jeff Dike

2006-10-20 16:34 UTC #6 lore raw

Quoted 4 lines · Jan Dittmer

As I wrote in the original message: Debian/etch amd64 host+guest, running btlaunchmany (from the bittorrent package). I can put you the compressed root + kernel somewhere if that helps. It's roughly 100mb.

Yeah, that would be useful.

Jeff

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel