negative pid -516 possible ?
Subsystems (3)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
I do fuzz testing with trinity (latest git version) a stable 32 bit Gentoo Linux user mode linux image. The host is a stable 32 bit vanilla 3.12.7 kernel, the guest runs latest git tree + 2 patches (attached).
The trinity call in the UML guest is :
$> trinity -q -l off -N 10000 -C 2 -x move_pages -x mremap -v /mnt/ramdisk
After a while there's no progress on the command line seen at the host system - the trinity process seems to just hangs/idling. When this does occur I cannot longer ssh into the system. The system however runs furthermore. In another terminal I still see the output of this command:
$> ssh root@trinity "tail -f /var/log/messages"
That's why I do know that the system does not hang completely. The output of top at the host system gives me the pid of the linux exe. A gdb call gives for that pid :
$ date; sudo gdb /home/tfoerste/devel/linux/linux 25224 -n -batch -ex 'bt full' Sat Jan 11 11:36:47 CET 2014
warning: Could not load shared library symbols for linux-gate.so.1.
Do you need "set solib-search-path" or "set sysroot"?0xb7800424 in __kernel_vsyscall ()
#0 0xb7800424 in __kernel_vsyscall ()
No symbol table info available.
#1 0x083d63ff in __nanosleep_nocancel ()
No symbol table info available.
#2 0x0807266c in idle_sleep (nsecs=602496380195307520) at arch/um/os-Linux/time.c:183
ts = {tv_sec = 0, tv_nsec = 8436602}#3 0x0805fc0f in arch_cpu_idle () at arch/um/kernel/process.c:208 No locals.
#4 0x080a8971 in cpu_idle_loop () at kernel/cpu/idle.c:98 No locals.
#5 cpu_startup_entry (state=CPUHP_ONLINE) at kernel/cpu/idle.c:140 No locals.
#6 0x084215e9 in rest_init () at init/main.c:402
pid = -516
__func__ = "rest_init"
#7 0x080487e1 in start_kernel () at init/main.c:656
command_line = 0x85b8400 <command_line> "earlyprintk ubda=/home/tfoerste/virtual/uml/trinity ubdb=/mnt/ramdisk/trinity_swap eth0=tuntap,tap0,72:ef:3d:9f:c3:5a mem=1025M con0=fd:0,fd:1 con=pts rootfstype=ext4 root=98:0"
#8 0x08049e42 in start_kernel_proc (unused=0x0) at arch/um/kernel/skas/process.c:48
pid = -516
__func__ = "start_kernel_proc"
#9 0x0805f7cb in new_thread_handler () at arch/um/kernel/process.c:129
fn = 0x0#10 0x00000000 in ?? ()
No symbol table info available.
Please note that BUG_ON was not triggered. For completeness here are the gdb traces from all linux processes currently running at the host:
$ pgrep linux | xargs -n1 -I {} sudo gdb /home/tfoerste/devel/linux/linux {} -n -batch -ex 'bt'
warning: process 1613 is already traced by process 25224
ptrace: Operation not permitted.
/home/tfoerste/1613: No such file or directory.
No stack.
warning: process 21849 is already traced by process 25224
ptrace: Operation not permitted.
/home/tfoerste/21849: No such file or directory.
No stack.warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"? 0xb7800424 in __kernel_vsyscall ()
#0 0xb7800424 in __kernel_vsyscall () #1 0x083d63ff in __nanosleep_nocancel () #2 0x0807266c in idle_sleep (nsecs=602496380205307520) at arch/um/os-Linux/time.c:183 #3 0x0805fc0f in arch_cpu_idle () at arch/um/kernel/process.c:208 #4 0x080a8971 in cpu_idle_loop () at kernel/cpu/idle.c:98 #5 cpu_startup_entry (state=CPUHP_ONLINE) at kernel/cpu/idle.c:140 #6 0x084215e9 in rest_init () at init/main.c:402 #7 0x080487e1 in start_kernel () at init/main.c:656 #8 0x08049e42 in start_kernel_proc (unused=0x0) at arch/um/kernel/skas/process.c:48 #9 0x0805f7cb in new_thread_handler () at arch/um/kernel/process.c:129
#10 0x00000000 in ?? ()
warning: process 25231 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"? 0xb7800424 in __kernel_vsyscall ()
#0 0xb7800424 in __kernel_vsyscall () #1 0x083da446 in syscall () #2 0x0806e861 in io_getevents (events=<optimized out>, ctx_id=<optimized out>, min_nr=<optimized out>, nr=<optimized out>, timeout=<optimized out>) at arch/um/os-Linux/aio.c:49 #3 aio_thread (arg=0x0) at arch/um/os-Linux/aio.c:109 #4 0x083db56e in clone ()
warning: process 25232 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"? 0xb7800424 in __kernel_vsyscall ()
#0 0xb7800424 in __kernel_vsyscall () #1 0x083d82c2 in __read_nocancel () #2 0x0806f3ff in read (__nbytes=<optimized out>, __buf=<optimized out>, __fd=<optimized out>) at /usr/include/bits/unistd.h:44 #3 os_read_file (fd=-512, buf=0xfffffe00, len=-512) at arch/um/os-Linux/file.c:253 #4 0x0806bafc in io_thread (arg=0x0) at arch/um/drivers/ubd_kern.c:1482 #5 0x083db56e in clone ()
warning: process 25233 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"? 0xb7800424 in __kernel_vsyscall ()
#0 0xb7800424 in __kernel_vsyscall () #1 0x083d9132 in __poll_nocancel () #2 0x08071114 in poll (__timeout=<optimized out>, __nfds=<optimized out>, __fds=<optimized out>) at /usr/include/bits/poll2.h:46 #3 write_sigio_thread (unused=0x0) at arch/um/os-Linux/sigio.c:61 #4 0x083db56e in clone ()
warning: process 25234 is a zombie - the process has already terminated
ptrace: Operation not permitted./home/tfoerste/25234: No such file or directory.
No stack.
...
Please Cc: me I'm not subscribed.
- --
MfG/Sincerely
Toralf Förster
pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iF4EAREIAAYFAlLRISQACgkQxOrN3gB26U54pwD9Eq49Oog5KpSC4+e19t4HG6LA 5d3Oz4/qq98wCb+rF9UA/0j+fT4xjdHbYmLtc8Z0wctVO3DjdQG49/+n81s/gLx3 =eP08
-----END PGP SIGNATURE-----
mm/filemap.c +8 -1
@@ -1108,18 +1108,25 @@ static void do_generic_file_read(struct file *filp, loff_t *ppos,pgoff_t prev_index;unsigned long offset;/* offset into pagecache page */ unsigned int prev_offset; +loff_t isize;int error; +/* we need to trim desc->count to avoid expose stale data to user */ + isize =i_size_read (inode); +if (*ppos + desc->count >= isize) + desc->count = isize - *ppos; index = *ppos >>PAGE_CACHE_SHIFT ; prev_index = ra->prev_pos >>PAGE_CACHE_SHIFT ; prev_offset = ra->prev_pos & (PAGE_CACHE_SIZE -1 ); last_index = (*ppos + desc->count +PAGE_CACHE_SIZE -1 ) >>PAGE_CACHE_SHIFT ; offset = *ppos & ~PAGE_CACHE_MASK ; +if (desc->count ==0 ) + goto out; +for (;;) {struct page *page;pgoff_t end_index; -loff_t isize;unsigned long nr, ret;cond_resched ();
arch/um/kernel/skas/process.c +2 -0
@@ -38,6 +38,8 @@ static int __init start_kernel_proc(void *unused)block_signals (); pid =os_getpid (); +BUG_ON (pid ==-516 ); + cpu_tasks[0 ].pid = pid; cpu_tasks[0 ].task = current;#ifdef CONFIG_SMP
init/main.c +1 -0
@@ -386,6 +386,7 @@ static noinline void __init_refok rest_init(void)kernel_thread (kernel_init,NULL ,CLONE_FS |CLONE_SIGHAND );numa_default_policy (); pid =kernel_thread (kthreadd,NULL ,CLONE_FS |CLONE_FILES ); +BUG_ON (pid ==-516 );rcu_read_lock (); kthreadd_task =find_task_by_pid_ns (pid, &init_pid_ns);rcu_read_unlock ();
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
Quoted 10 lines · Toralf Förster
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256I do fuzz testing with trinity (latest git version) a stable 32 bit Gentoo Linux user mode linux image. The host is a stable 32 bit vanilla 3.12.7 kernel, the guest runs latest git tree + 2 patches (attached).
The trinity call in the UML guest is :
$> trinity -q -l off -N 10000 -C 2 -xmove_pages-x mremap -v/mnt/ramdiskAfter a while there's no progress on the command line seen at the host system - the trinity process seems to just hangs/idling. When this does occur I cannot longer ssh into the system. The system however runs furthermore. In another terminal I still see the output of this command:
Does it consume 100% CPU?
Quoted 32 lines
$> ssh root@trinity "tail -f
/var/log/messages"That's why I do know that the system does not hang completely. The output of top at the host system gives me the pid of the linux exe. A gdb call gives for that pid :
$ date; sudo gdb /home/tfoerste/devel/linux/linux 25224 -n -batch -ex 'bt full' Sat Jan 11 11:36:47 CET 2014warning: Could not load shared library symbols for linux-gate.so.1.
Do you need "set solib-search-path" or "set sysroot"?0xb7800424in__kernel_vsyscall()
#00xb7800424in__kernel_vsyscall()
No symbol table info available.
#10x083d63ffin__nanosleep_nocancel()
No symbol table info available.
#20x0807266cinidle_sleep(nsecs=602496380195307520) atarch/um/os-Linux/time.c:183ts = {tv_sec = 0, tv_nsec = 8436602}#3
0x0805fc0finarch_cpu_idle() atarch/um/kernel/process.c:208No locals.
#40x080a8971incpu_idle_loop() atkernel/cpu/idle.c:98No locals.
#5cpu_startup_entry(state=CPUHP_ONLINE) atkernel/cpu/idle.c:140No locals.
#60x084215e9inrest_init() atinit/main.c:402pid = -516 __func__ = "rest_init" #7 0x080487e1 in start_kernel () at init/main.c:656 command_line = 0x85b8400 <command_line> "earlyprintk ubda=/home/tfoerste/virtual/uml/trinity ubdb=/mnt/ramdisk/trinity_swap eth0=tuntap,tap0,72:ef:3d:9f:c3:5a mem=1025M con0=fd:0,fd:1 con=pts rootfstype=ext4 root=98:0" #8 0x08049e42 in start_kernel_proc (unused=0x0) at arch/um/kernel/skas/process.c:48 pid = -516 __func__ = "start_kernel_proc" #9 0x0805f7cb in new_thread_handler () at arch/um/kernel/process.c:129 fn = 0x0
Quoted 6 lines
#10
0x00000000in ?? ()
No symbol table info available.Please note that
BUG_ONwas not triggered. For completeness here are the gdb traces from all linux processes currently running at the host:
So let's forget the 516 issue for now.
What we no for now is that you manage to trigger a lockup within UML.
Quoted 59 lines
$ pgrep linux | xargs -n1 -I {} sudo gdb /home/tfoerste/devel/linux/linux {} -n -batch -ex 'bt' warning: process 1613 is already traced by process 25224 ptrace: Operation not permitted. /home/tfoerste/1613: No such file or directory. No stack. warning: process 21849 is already traced by process 25224 ptrace: Operation not permitted. /home/tfoerste/21849: No such file or directory. No stack.warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083d63ff in __nanosleep_nocancel () #2 0x0807266c in idle_sleep (nsecs=602496380205307520) at arch/um/os-Linux/time.c:183 #3 0x0805fc0f in arch_cpu_idle () at arch/um/kernel/process.c:208 #4 0x080a8971 in cpu_idle_loop () at kernel/cpu/idle.c:98 #5 cpu_startup_entry (state=CPUHP_ONLINE) at kernel/cpu/idle.c:140 #6 0x084215e9 in rest_init () at init/main.c:402 #7 0x080487e1 in start_kernel () at init/main.c:656 #8 0x08049e42 in start_kernel_proc (unused=0x0) at arch/um/kernel/skas/process.c:48 #9 0x0805f7cb in new_thread_handler () at arch/um/kernel/process.c:129#10
0x00000000in ?? ()warning: process 25231 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083da446 in syscall () #2 0x0806e861 in io_getevents (events=<optimized out>, ctx_id=<optimized out>, min_nr=<optimized out>, nr=<optimized out>, timeout=<optimized out>) at arch/um/os-Linux/aio.c:49 #3 aio_thread (arg=0x0) at arch/um/os-Linux/aio.c:109 #4 0x083db56e in clone ()warning: process 25232 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083d82c2 in __read_nocancel () #2 0x0806f3ff in read (__nbytes=<optimized out>, __buf=<optimized out>, __fd=<optimized out>) at /usr/include/bits/unistd.h:44 #3 os_read_file (fd=-512, buf=0xfffffe00, len=-512) at arch/um/os-Linux/file.c:253 #4 0x0806bafc in io_thread (arg=0x0) at arch/um/drivers/ubd_kern.c:1482 #5 0x083db56e in clone ()warning: process 25233 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083d9132 in __poll_nocancel () #2 0x08071114 in poll (__timeout=<optimized out>, __nfds=<optimized out>, __fds=<optimized out>) at /usr/include/bits/poll2.h:46 #3 write_sigio_thread (unused=0x0) at arch/um/os-Linux/sigio.c:61 #4 0x083db56e in clone ()
Quoted 8 lines
warning: process 25234 is a zombie - the process has already terminated
ptrace: Operation not permitted./home/tfoerste/25234: No such file or directory.
No stack.
...Please Cc: me I'm not subscribed.
Wouldn't it make sense to subscribe?
You post very often on this list. :)
Quoted 26 lines
- --
MfG/Sincerely
Toralf Förster
pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/iF4EAREIAAYFAlLRISQACgkQxOrN3gB26U54pwD9Eq49Oog5KpSC4+e19t4HG6LA 5d3Oz4/qq98wCb+rF9UA/0j+fT4xjdHbYmLtc8Z0wctVO3DjdQG49/+n81s/gLx3 =eP08
-----END PGP SIGNATURE-----------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
Signature · 13 lines
Thanks, //richard ------------------------------------------------------------------------------ CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk _______________________________________________ User-mode-linux-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Quoted 11 lines · Richard Weinberger
On Sat, Jan 11, 2014 at 11:47 AM, Toralf Förster <[email protected]> wrote:
I do fuzz testing with trinity (latest git version) a stable 32 bit Gentoo Linux user mode linux image. The host is a stable 32 bit vanilla 3.12.7 kernel, the guest runs latest git tree + 2 patches (attached).The trinity call in the UML guest is :
$> trinity -q -l off -N 10000 -C 2 -xmove_pages-x mremap -v/mnt/ramdiskAfter a while there's no progress on the command line seen at the host system - the trinity process seems to just hangs/idling. When this does occur I cannot longer ssh into the system. The system however runs furthermore. In another terminal I still see the output of this command:
Quoted 1 line
Does it consume 100% CPU?
No.
It just doesnt allow new ssh connections. Existing ssh conenctinos are still working.
Quoted 32 lines
$> ssh root@trinity "tail -f
/var/log/messages"That's why I do know that the system does not hang completely. The output of top at the host system gives me the pid of the linux exe. A gdb call gives for that pid :
$ date; sudo gdb /home/tfoerste/devel/linux/linux 25224 -n -batch -ex 'bt full' Sat Jan 11 11:36:47 CET 2014warning: Could not load shared library symbols for linux-gate.so.1.
Do you need "set solib-search-path" or "set sysroot"?0xb7800424in__kernel_vsyscall()
#00xb7800424in__kernel_vsyscall()
No symbol table info available.
#10x083d63ffin__nanosleep_nocancel()
No symbol table info available.
#20x0807266cinidle_sleep(nsecs=602496380195307520) atarch/um/os-Linux/time.c:183ts = {tv_sec = 0, tv_nsec = 8436602}#3
0x0805fc0finarch_cpu_idle() atarch/um/kernel/process.c:208No locals.
#40x080a8971incpu_idle_loop() atkernel/cpu/idle.c:98No locals.
#5cpu_startup_entry(state=CPUHP_ONLINE) atkernel/cpu/idle.c:140No locals.
#60x084215e9inrest_init() atinit/main.c:402pid = -516 __func__ = "rest_init" #7 0x080487e1 in start_kernel () at init/main.c:656 command_line = 0x85b8400 <command_line> "earlyprintk ubda=/home/tfoerste/virtual/uml/trinity ubdb=/mnt/ramdisk/trinity_swap eth0=tuntap,tap0,72:ef:3d:9f:c3:5a mem=1025M con0=fd:0,fd:1 con=pts rootfstype=ext4 root=98:0" #8 0x08049e42 in start_kernel_proc (unused=0x0) at arch/um/kernel/skas/process.c:48 pid = -516 __func__ = "start_kernel_proc" #9 0x0805f7cb in new_thread_handler () at arch/um/kernel/process.c:129 fn = 0x0
Quoted 10 lines
#10
0x00000000in ?? ()
No symbol table info available.Please note that
BUG_ONwas not triggered. For completeness here are the gdb traces from all linux processes currently running at the host:Quoted 2 lines
So let's forget the 516 issue for now.
What we no for now is that you manage to trigger a lockup within UML.
Agreed, especially b/c I added this patch too :
$ cat ~/devel/priv/uml/pid516_2.patch
- --- init/main.c_orig 2014-01-12 16:43:48.585439158 +0100
+++ init/main.c 2014-01-12 16:44:01.706438453 +0100
@@ -389,6 +389,7 @@
BUG_ON(pid == -516);
rcu_read_lock();
kthreadd_task = find_task_by_pid_ns(pid, &init_pid_ns);
+ BUG_ON(pid == -516);
rcu_read_unlock();
complete(&kthreadd_done);and this wasn't triggered (/me wonders if the -516 is somehow garbage).
But I can narrow down the problem. In an still open ssh sessions I made :
$ lsof | grep t3 bash 6129 tfoerste cwd DIR 98,0 4096 734 /home/tfoerste/t3 logger 6135 tfoerste cwd DIR 98,0 4096 734 /home/tfoerste/t3
(t3 is the ~/t3 directory where I cd into it bewfore I run trinity.
And after killing the logger command the trinity batch continues :
$ ps xf -eo pid,start_time,command | grep trinity 6412 20:48 | \_ grep --colour=auto trinity 6129 19:17 \_ bash -c cd ~; sudo su -c 'if [[ -d ./t3 ]]; then sudo chmod -R a+rwx ./t3; sudo rm -rf ./t3; fi'; mkdir ./t3; cd ./t3; logger "17#-1, M=/mnt/ramdisk"; if [[ -n /mnt/ramdisk ]]; then if [[ -d /mnt/ramdisk/victims/v1 ]]; then sudo chmod -R a+rwx /mnt/ramdisk/victims/v1; sudo rm -rf /mnt/ramdisk/victims/v1; fi; mkdir -p /mnt/ramdisk/victims/v1/v2; for i in $(seq -w 0 99); do touch /mnt/ramdisk/victims/v1/v2/f$i 2>/dev/null; mkdir /mnt/ramdisk/victims/v1/v2/d$i 2>/dev/null; done; fi; trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6390 20:46 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6391 20:46 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6392 20:46 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6408 20:47 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6410 20:48 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2
FWIW a ssh into the UML guest is however still no longer possible. So I'm pretty sure that trinity damage there something really but I'd expect that such a damage should be seen somewhere in the logs, or ?
And finally - now the the batch trinity command hangs again and now not even killing logger helps. And a shutdown ("sudo halt; exit") hangs too.
Quoted 60 lines
$ pgrep linux | xargs -n1 -I {} sudo gdb /home/tfoerste/devel/linux/linux {} -n -batch -ex 'bt' warning: process 1613 is already traced by process 25224 ptrace: Operation not permitted. /home/tfoerste/1613: No such file or directory. No stack. warning: process 21849 is already traced by process 25224 ptrace: Operation not permitted. /home/tfoerste/21849: No such file or directory. No stack.warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083d63ff in __nanosleep_nocancel () #2 0x0807266c in idle_sleep (nsecs=602496380205307520) at arch/um/os-Linux/time.c:183 #3 0x0805fc0f in arch_cpu_idle () at arch/um/kernel/process.c:208 #4 0x080a8971 in cpu_idle_loop () at kernel/cpu/idle.c:98 #5 cpu_startup_entry (state=CPUHP_ONLINE) at kernel/cpu/idle.c:140 #6 0x084215e9 in rest_init () at init/main.c:402 #7 0x080487e1 in start_kernel () at init/main.c:656 #8 0x08049e42 in start_kernel_proc (unused=0x0) at arch/um/kernel/skas/process.c:48 #9 0x0805f7cb in new_thread_handler () at arch/um/kernel/process.c:129#10
0x00000000in ?? ()warning: process 25231 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083da446 in syscall () #2 0x0806e861 in io_getevents (events=<optimized out>, ctx_id=<optimized out>, min_nr=<optimized out>, nr=<optimized out>, timeout=<optimized out>) at arch/um/os-Linux/aio.c:49 #3 aio_thread (arg=0x0) at arch/um/os-Linux/aio.c:109 #4 0x083db56e in clone ()warning: process 25232 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083d82c2 in __read_nocancel () #2 0x0806f3ff in read (__nbytes=<optimized out>, __buf=<optimized out>, __fd=<optimized out>) at /usr/include/bits/unistd.h:44 #3 os_read_file (fd=-512, buf=0xfffffe00, len=-512) at arch/um/os-Linux/file.c:253 #4 0x0806bafc in io_thread (arg=0x0) at arch/um/drivers/ubd_kern.c:1482 #5 0x083db56e in clone ()warning: process 25233 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083d9132 in __poll_nocancel () #2 0x08071114 in poll (__timeout=<optimized out>, __nfds=<optimized out>, __fds=<optimized out>) at /usr/include/bits/poll2.h:46 #3 write_sigio_thread (unused=0x0) at arch/um/os-Linux/sigio.c:61 #4 0x083db56e in clone ()
Quoted 12 lines
warning: process 25234 is a zombie - the process has already terminated
ptrace: Operation not permitted./home/tfoerste/25234: No such file or directory.
No stack.
...Please Cc: me I'm not subscribed.
Quoted 2 lines
Wouldn't it make sense to subscribe?
You post very often on this list. :)
done ;)
Quoted 17 lines
Quoted 14 lines
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
- --
MfG/Sincerely
Toralf Förster
pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iF4EAREIAAYFAlLURGIACgkQxOrN3gB26U44RQD+KUqGBeP6/nJk1K/1Wx6nz7ij /JXcjNN+ZBt8PsMWrV4A/jx7w7Xrl0RPWcwXVFYm+Ixo0dSbtr+zvh/2pdcCNU2c =uGid
-----END PGP SIGNATURE-----
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Quoted 163 lines · Toralf Förster
Quoted 10 lines · Richard Weinberger
On Sat, Jan 11, 2014 at 11:47 AM, Toralf Förster <[email protected]> wrote:
I do fuzz testing with trinity (latest git version) a stable 32 bit Gentoo Linux user mode linux image. The host is a stable 32 bit vanilla 3.12.7 kernel, the guest runs latest git tree + 2 patches (attached).The trinity call in the UML guest is :
$> trinity -q -l off -N 10000 -C 2 -xmove_pages-x mremap -v/mnt/ramdiskAfter a while there's no progress on the command line seen at the host system - the trinity process seems to just hangs/idling. When this does occur I cannot longer ssh into the system. The system however runs furthermore. In another terminal I still see the output of this command:
Quoted 1 line
Does it consume 100% CPU?
No.
It just doesnt allow new ssh connections. Existing ssh conenctinos are still working.Quoted 32 lines
$> ssh root@trinity "tail -f
/var/log/messages"That's why I do know that the system does not hang completely. The output of top at the host system gives me the pid of the linux exe. A gdb call gives for that pid :
$ date; sudo gdb /home/tfoerste/devel/linux/linux 25224 -n -batch -ex 'bt full' Sat Jan 11 11:36:47 CET 2014warning: Could not load shared library symbols for linux-gate.so.1.
Do you need "set solib-search-path" or "set sysroot"?0xb7800424in__kernel_vsyscall()
#00xb7800424in__kernel_vsyscall()
No symbol table info available.
#10x083d63ffin__nanosleep_nocancel()
No symbol table info available.
#20x0807266cinidle_sleep(nsecs=602496380195307520) atarch/um/os-Linux/time.c:183ts = {tv_sec = 0, tv_nsec = 8436602}#3
0x0805fc0finarch_cpu_idle() atarch/um/kernel/process.c:208No locals.
#40x080a8971incpu_idle_loop() atkernel/cpu/idle.c:98No locals.
#5cpu_startup_entry(state=CPUHP_ONLINE) atkernel/cpu/idle.c:140No locals.
#60x084215e9inrest_init() atinit/main.c:402pid = -516 __func__ = "rest_init" #7 0x080487e1 in start_kernel () at init/main.c:656 command_line = 0x85b8400 <command_line> "earlyprintk ubda=/home/tfoerste/virtual/uml/trinity ubdb=/mnt/ramdisk/trinity_swap eth0=tuntap,tap0,72:ef:3d:9f:c3:5a mem=1025M con0=fd:0,fd:1 con=pts rootfstype=ext4 root=98:0" #8 0x08049e42 in start_kernel_proc (unused=0x0) at arch/um/kernel/skas/process.c:48 pid = -516 __func__ = "start_kernel_proc" #9 0x0805f7cb in new_thread_handler () at arch/um/kernel/process.c:129 fn = 0x0Quoted 9 lines
#10
0x00000000in ?? ()
No symbol table info available.Please note that
BUG_ONwas not triggered. For completeness here are the gdb traces from all linux processes currently running at the host:Quoted 2 lines
So let's forget the 516 issue for now.
What we no for now is that you manage to trigger a lockup within UML.Agreed, especially b/c I added this patch too :
$ cat ~/devel/priv/uml/pid516_2.patchmain.c 2014-01-12 16:44:01.706438453 +0100 renamed from main.c_orig 2014-01-12 16:43:48.585439158 +0100 +1
-0@@ -389,6 +389,7 @@ BUG_ON(pid == -516); rcu_read_lock(); kthreadd_task = find_task_by_pid_ns(pid, &init_pid_ns); + BUG_ON(pid == -516); rcu_read_unlock(); complete(&kthreadd_done);and this wasn't triggered (/me wonders if the -516 is somehow garbage).
But I can narrow down the problem. In an still open ssh sessions I made :
$ lsof | grep t3 bash 6129 tfoerste cwd DIR 98,0 4096 734 /home/tfoerste/t3 logger 6135 tfoerste cwd DIR 98,0 4096 734 /home/tfoerste/t3(t3 is the ~/t3 directory where I cd into it bewfore I run trinity.
And after killing the logger command the trinity batch continues :
$ ps xf -eo pid,start_time,command | grep trinity 6412 20:48 | \_ grep --colour=auto trinity 6129 19:17 \_ bash -c cd ~; sudo su -c 'if [[ -d ./t3 ]]; then sudo chmod -R a+rwx ./t3; sudo rm -rf ./t3; fi'; mkdir ./t3; cd ./t3; logger "17#-1, M=/mnt/ramdisk"; if [[ -n /mnt/ramdisk ]]; then if [[ -d /mnt/ramdisk/victims/v1 ]]; then sudo chmod -R a+rwx /mnt/ramdisk/victims/v1; sudo rm -rf /mnt/ramdisk/victims/v1; fi; mkdir -p /mnt/ramdisk/victims/v1/v2; for i in $(seq -w 0 99); do touch /mnt/ramdisk/victims/v1/v2/f$i 2>/dev/null; mkdir /mnt/ramdisk/victims/v1/v2/d$i 2>/dev/null; done; fi; trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6390 20:46 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6391 20:46 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6392 20:46 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6408 20:47 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2 6410 20:48 \_ trinity -q -N 10000 -C 2 -x move_pages -x mremap -V /mnt/ramdisk/victims/v1/v2FWIW a ssh into the UML guest is however still no longer possible. So I'm pretty sure that trinity damage there something really but I'd expect that such a damage should be seen somewhere in the logs, or ?
And finally - now the the batch trinity command hangs again and now not even killing logger helps. And a shutdown ("sudo halt; exit") hangs too.
Quoted 63 lines
$ pgrep linux | xargs -n1 -I {} sudo gdb /home/tfoerste/devel/linux/linux {} -n -batch -ex 'bt' warning: process 1613 is already traced by process 25224 ptrace: Operation not permitted. /home/tfoerste/1613: No such file or directory. No stack. warning: process 21849 is already traced by process 25224 ptrace: Operation not permitted. /home/tfoerste/21849: No such file or directory. No stack.warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083d63ff in __nanosleep_nocancel () #2 0x0807266c in idle_sleep (nsecs=602496380205307520) at arch/um/os-Linux/time.c:183 #3 0x0805fc0f in arch_cpu_idle () at arch/um/kernel/process.c:208 #4 0x080a8971 in cpu_idle_loop () at kernel/cpu/idle.c:98 #5 cpu_startup_entry (state=CPUHP_ONLINE) at kernel/cpu/idle.c:140 #6 0x084215e9 in rest_init () at init/main.c:402 #7 0x080487e1 in start_kernel () at init/main.c:656 #8 0x08049e42 in start_kernel_proc (unused=0x0) at arch/um/kernel/skas/process.c:48 #9 0x0805f7cb in new_thread_handler () at arch/um/kernel/process.c:129#10
0x00000000in ?? ()warning: process 25231 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083da446 in syscall () #2 0x0806e861 in io_getevents (events=<optimized out>, ctx_id=<optimized out>, min_nr=<optimized out>, nr=<optimized out>, timeout=<optimized out>) at arch/um/os-Linux/aio.c:49 #3 aio_thread (arg=0x0) at arch/um/os-Linux/aio.c:109 #4 0x083db56e in clone ()warning: process 25232 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083d82c2 in __read_nocancel () #2 0x0806f3ff in read (__nbytes=<optimized out>, __buf=<optimized out>, __fd=<optimized out>) at /usr/include/bits/unistd.h:44 #3 os_read_file (fd=-512, buf=0xfffffe00, len=-512) at arch/um/os-Linux/file.c:253 #4 0x0806bafc in io_thread (arg=0x0) at arch/um/drivers/ubd_kern.c:1482 #5 0x083db56e in clone ()warning: process 25233 is a cloned process
warning: Could not load shared library symbols for linux-gate.so.1. Do you need "set solib-search-path" or "set sysroot"?
0xb7800424in__kernel_vsyscall()#0 0xb7800424 in __kernel_vsyscall () #1 0x083d9132 in __poll_nocancel () #2 0x08071114 in poll (__timeout=<optimized out>, __nfds=<optimized out>, __fds=<optimized out>) at /usr/include/bits/poll2.h:46 #3 write_sigio_thread (unused=0x0) at arch/um/os-Linux/sigio.c:61 #4 0x083db56e in clone ()warning: process 25234 is a zombie - the process has already terminated
ptrace: Operation not permitted./home/tfoerste/25234: No such file or directory.
No stack.
...
Quoted 26 lines
Quoted 4 lines
Please Cc: me I'm not subscribed.
Quoted 2 lines
Wouldn't it make sense to subscribe?
You post very often on this list. :)done ;)
Quoted 12 lines
Quoted 12 lines
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
A funny variant of the problem is , that with clatest Linus' tree I do now get pid = 0 - all others are unchanged :
$ date; sudo gdb /home/tfoerste/devel/linux/linux 18483 -n -batch -ex 'bt full' Sat Feb 15 16:41:42 CET 2014
warning: Could not load shared library symbols for linux-gate.so.1.
Do you need "set solib-search-path" or "set sysroot"?0xb7792424 in __kernel_vsyscall ()
#0 0xb7792424 in __kernel_vsyscall ()
No symbol table info available.
#1 0x083ded0f in __nanosleep_nocancel ()
No symbol table info available.
#2 0x0807269c in idle_sleep (nsecs=602637203593008768) at arch/um/os-Linux/time.c:183
ts = {tv_sec = 0, tv_nsec = 10000000}#3 0x0805fc2f in arch_cpu_idle () at arch/um/kernel/process.c:208 No locals.
#4 0x080a99c1 in cpu_idle_loop () at kernel/cpu/idle.c:98 No locals.
#5 cpu_startup_entry (state=CPUHP_ONLINE) at kernel/cpu/idle.c:143 No locals.
#6 0x08429ec2 in rest_init () at init/main.c:397
pid = 0
#7 0x080487e9 in start_kernel () at init/main.c:652
command_line = 0x85c2420 <command_line> "earlyprintk ubda=/home/tfoerste/virtual/uml/trinity ubdb=/mnt/ramdisk/trinity_swap eth0=tuntap,tap0,72:ef:3d:9f:c3:5a mem=1025M con0=fd:0,fd:1 con=pts rootfstype=ext4 root=98:0"
#8 0x08049e19 in start_kernel_proc (unused=0x0) at arch/um/kernel/skas/process.c:46
pid = 0
#9 0x0805f7eb in new_thread_handler () at arch/um/kernel/process.c:129
fn = 0x0#10 0x00000000 in ?? ()
No symbol table info available.
- --
MfG/Sincerely
Toralf Förster
pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iF4EAREIAAYFAlL/i1MACgkQxOrN3gB26U6HCwD/WRTDhGO38eNIMaZla2RPLCcW AVbaR7p7PLtFHP/I7AsA/Rzz9ASZyvxpx+TufWWl/3xKkv7fFs/Z6/laEseKhVpM =v42x
-----END PGP SIGNATURE-----
------------------------------------------------------------------------------
Android apps run on BlackBerry 10
Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now.
http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel