Gentoo guest with smp: emerge freeze while recompile world

This is a multi-part message in MIME format.

Hi,
I have a server dual xeon quad core with gentoo and qemu:
app-emulation/qemu-kvm-0.12.3-r1 USE="aio gnutls ncurses sasl vde -alsa

(quoted diff) +0 -2

-bluetooth -curl -esd -fdt -hardened -kvm-trace -pulseaudio -qemu-ifup -sdl
-static"

There is a lot of vm running with ubuntu and fedora that running without problems.

I installed one vm with latest gentoo amd64, stage3 and portage. When I try to do an emerge -e world the process freeze after a while, any time in differente package and there aren't any errors in the logs. This is a screenshot of the freezed vm:
http://yfrog.com/0iscre1j

top - 10:00:50 up 10:53, 2 users, load average: 0.00, 0.00, 0.00

Tasks: 130 total,   1 running, 124 sleeping,   0 stopped,   5 zombie
Cpu(s):  0.1%us,  0.0%sy,  0.0%ni, 99.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8194776k total,   886508k used,  7308268k free,   225080k buffers
Swap:  2048248k total,        0k used,  2048248k free,   476956k cached

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     1 root      20   0  3760  656  552 S    0  0.0   0:00.86 init
     2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd
     3 root      RT   0     0    0    0 S    0  0.0   0:01.78 migration/0
     4 root      20   0     0    0    0 S    0  0.0   0:00.00 ksoftirqd/0
     5 root      RT   0     0    0    0 S    0  0.0   0:02.01 migration/1
     6 root      20   0     0    0    0 S    0  0.0   0:00.01 ksoftirqd/1
     7 root      RT   0     0    0    0 S    0  0.0   0:02.05 migration/2
     8 root      20   0     0    0    0 S    0  0.0   0:00.01 ksoftirqd/2
     9 root      RT   0     0    0    0 S    0  0.0   0:02.15 migration/3
    10 root      20   0     0    0    0 S    0  0.0   0:00.01 ksoftirqd/3
    11 root      RT   0     0    0    0 S    0  0.0   0:01.69 migration/4
    12 root      20   0     0    0    0 S    0  0.0   0:00.00 ksoftirqd/4
    13 root      RT   0     0    0    0 S    0  0.0   0:01.49 migration/5
    14 root      20   0     0    0    0 S    0  0.0   0:00.00 ksoftirqd/5
    15 root      20   0     0    0    0 S    0  0.0   0:00.11 events/0
    16 root      20   0     0    0    0 S    0  0.0   0:00.21 events/1

ps -elf:
1 S root 776 2 0 80 0 - 0 scsi_e May18 ? 00:00:00
[scsi_eh_1]
1 S root 810 2 0 80 0 - 0 worker May18 ? 00:00:00
[kpsmoused]
1 S root 818 2 0 80 0 - 0 worker May18 ? 00:00:00
[kstriped]
1 S root 821 2 0 80 0 - 0 worker May18 ? 00:00:00
[kmpathd/0]
1 S root 822 2 0 80 0 - 0 worker May18 ? 00:00:00
[kmpathd/1]
1 S root 823 2 0 80 0 - 0 worker May18 ? 00:00:00
[kmpathd/2]
1 S root 824 2 0 80 0 - 0 worker May18 ? 00:00:00
[kmpathd/3]
1 S root 825 2 0 80 0 - 0 worker May18 ? 00:00:00
[kmpathd/4]
1 S root 826 2 0 80 0 - 0 worker May18 ? 00:00:00
[kmpathd/5]
1 S root 827 2 0 80 0 - 0 worker May18 ? 00:00:00
[kmpath_handlerd]
1 S root 828 2 0 80 0 - 0 worker May18 ? 00:00:00
[ksnapd]
1 S root 859 2 0 80 0 - 0 worker May18 ? 00:00:00
[usbhid_resumer]
1 S root 900 2 0 80 0 - 0 kjourn May18 ? 00:00:00
[jbd2/vda3-8]
1 S root 901 2 0 80 0 - 0 worker May18 ? 00:00:00
[ext4-dio-unwrit]
1 S root 902 2 0 80 0 - 0 worker May18 ? 00:00:00
[ext4-dio-unwrit]
1 S root 903 2 0 80 0 - 0 worker May18 ? 00:00:00
[ext4-dio-unwrit]
1 S root 904 2 0 80 0 - 0 worker May18 ? 00:00:00
[ext4-dio-unwrit]
1 S root 905 2 0 80 0 - 0 worker May18 ? 00:00:00
[ext4-dio-unwrit]
1 S root 906 2 0 80 0 - 0 worker May18 ? 00:00:00
[ext4-dio-unwrit]
5 S root 1005 1 0 76 -4 - 3098 poll_s May18 ? 00:00:00
/sbin/udevd --daemon
1 S root 2661 1 0 80 0 - 7492 wait May18 ? 00:00:00
supervising syslog-ng
5 S root 2662 2661 0 80 0 - 7525 poll_s May18 ? 00:00:00
/usr/sbin/syslog-ng
1 S root 3250 1 0 80 0 - 9477 poll_s May18 ? 00:00:00
/usr/sbin/sshd
1 S root 3370 1 0 80 0 - 4086 hrtime May18 ? 00:00:00
/usr/sbin/cron
4 S root 3437 1 0 80 0 - 13988 wait May18 tty1 00:00:00
/bin/login --
0 S root 3438 1 0 80 0 - 1464 n_tty_ May18 tty2 00:00:00
/sbin/agetty 38400 tty2 linux
0 S root 3439 1 0 80 0 - 1465 n_tty_ May18 tty3 00:00:00
/sbin/agetty 38400 tty3 linux
0 S root 3440 1 0 80 0 - 1464 n_tty_ May18 tty4 00:00:00
/sbin/agetty 38400 tty4 linux
0 S root 3441 1 0 80 0 - 1465 n_tty_ May18 tty5 00:00:00
/sbin/agetty 38400 tty5 linux
0 S root 3442 1 0 80 0 - 1465 n_tty_ May18 tty6 00:00:00
/sbin/agetty 38400 tty6 linux

4 S root      3443  3437  0  80   0 -  4431 n_tty_ May18 tty1     00:00:00 -bash
5 S root      3449     1  0  80   0 -  6152 poll_s May18 ?        00:01:08

SCREEN -S cs1
4 S root 3450 3449 0 80 0 - 4403 wait May18 pts/0 00:00:00
-/bin/bash
4 S root 3454 3450 0 75 -5 - 35103 n_tty_ May18 pts/0 00:00:49
/usr/bin/python2.6 /usr/bin/emerge --resume
1 S root 5222 2 0 80 0 - 0 bdi_wr 09:50 ? 00:00:00
[flush-253:0]
4 S root 5241 3250 0 80 0 - 17913 poll_s 10:00 ? 00:00:00
sshd: root@pts/2

4 S root      5244  5241  0  80   0 -  4401 wait   10:00 pts/2    00:00:00 -bash
4 R root      5251  5244  0  80   0 -  3671 -      10:02 pts/2    00:00:00 ps -elf
0 S root     12724  3454  0  75  -5 -   955 wait   May18 pts/0    00:00:00

[sys-libs/glibc-2.10.1-r1] sandbox "/usr/lib64/portage/bin/ebuild.sh" compile
4 S root 12725 12724 0 75 -5 - 6431 wait May18 pts/0 00:00:00
/bin/bash /usr/lib64/portage/bin/ebuild.sh compile
5 S root 12756 12725 0 75 -5 - 6936 wait May18 pts/0 00:00:00
/bin/bash /usr/lib64/portage/bin/ebuild.sh compile
4 S root 14901 12756 0 75 -5 - 3631 wait May18 pts/0 00:00:00
make PARALLELMFLAGS=-j8
4 S root 14902 14901 0 75 -5 - 5155 wait May18 pts/0 00:00:01
make -r PARALLELMFLAGS=-j8 CVSOPTS= -C /var/tmp/portage/sys-libs/glibc-2.10..1-r1/work/glibc-2.10.1 objdir=/var/tmp/
4 S root 30286 14902 0 75 -5 - 5784 n_tty_ May18 pts/0 00:00:03
make -j8 subdir=nptl -C nptl ..=../ subdir_lib
4 Z root 31542 30286 0 75 -5 - 0 exit 01:11 pts/0 00:00:00
[x86_64-pc-linux]<defunct>
4 Z root 31543 30286 0 75 -5 - 0 exit 01:11 pts/0 00:00:00
[x86_64-pc-linux]<defunct>
4 Z root 31544 30286 0 75 -5 - 0 exit 01:11 pts/0 00:00:00
[x86_64-pc-linux]<defunct>
4 Z root 31545 30286 0 75 -5 - 0 exit 01:11 pts/0 00:00:00
[x86_64-pc-linux]<defunct>
4 Z root 31547 30286 0 75 -5 - 0 exit 01:11 pts/0 00:00:00
[x86_64-pc-linux]<defunct>

I found the same problem with gentoo-sources-2.6.32-r7,2.6.33,2.6.33-r1,2.6.33-2 always with -smp 2, 4 or 6. Kernel .config:
http://pastebin.org/252943
With 2.6.32-r7 I notice the emerge will freeze after a lot of compiled packages, while with the 2.6.33 series after 30-40 compiled packages, emerge freeze.

If I run vm without -smp there aren't any freeze of emerge.

In the same server there are other VMs with ubuntu and -smp 6 and works without any problems.

There is a limit max cpus for -smp ? for the total of the VMs running in the server?
Any suggestions?
It's possible to enable a log for what service?

Quoted 9 lines

This is a multi-part message in MIME format.

Hi,
I have a server dual xeon quad core with gentoo and qemu:
app-emulation/qemu-kvm-0.12.3-r1 USE="aio gnutls ncurses sasl vde -alsa -bluetooth -curl -esd -fdt -hardened -kvm-trace -pulseaudio -qemu- ifup -sdl
-static"

Quoted 3 lines

Any suggestions?
It's possible to enable a log for what service?

There are almost impossible to debug.

Try copying vmlinux out of your guest and attach with gdb when it hangs. Then issue the command

(gdb) thread apply all backtrace

to see what the guest is doing.

Signature · 7 lines

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

There are almost impossible to debug.

Try copying vmlinux out of your guest and attach with gdb when it hangs. Then issue the command

(gdb) thread apply all backtrace

to see what the guest is doing.

--
Do not meddle in the internals of kernels, for they are subtle and quick to

...

Quoted 12 lines

There are almost impossible to debug.

Try copying vmlinux out of your guest and attach with gdb when it hangs. Then issue the command

(gdb) thread apply all backtrace

to see what the guest is doing.

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to

panic.
------- End of Original Message -------

Hi,
I compile gentoo-sources-2.6.31-r10 and with this kernel "emerge -e world" complete without errors!

I always use the same .config

After I try gentoo-sources-2.6.34 and vanilla-sources-2.6.34 but the problem remain, the compile freeze and I see this in ps -elf:

5 S root 1013 1 0 76 -4 - 3125 poll_s 13:00 ? 00:00:00
/sbin/udevd --daemon
1 S root 2669 1 0 80 0 - 7523 wait 13:00 ? 00:00:00
supervising syslog-ng
5 S root 2670 2669 0 80 0 - 7556 poll_s 13:00 ? 00:00:00
/usr/sbin/syslog-ng
1 S root 3258 1 0 80 0 - 9505 poll_s 13:00 ? 00:00:00
/usr/sbin/sshd
1 S root 3378 1 0 80 0 - 4115 hrtime 13:00 ? 00:00:00
/usr/sbin/cron
0 S root 3446 1 0 80 0 - 1493 n_tty_ 13:00 tty2 00:00:00
/sbin/agetty 38400 tty2 linux
0 S root 3447 1 0 80 0 - 1493 n_tty_ 13:00 tty3 00:00:00
/sbin/agetty 38400 tty3 linux
0 S root 3448 1 0 80 0 - 1493 n_tty_ 13:00 tty4 00:00:00
/sbin/agetty 38400 tty4 linux
0 S root 3449 1 0 80 0 - 1493 n_tty_ 13:00 tty5 00:00:00
/sbin/agetty 38400 tty5 linux
0 S root 3450 1 0 80 0 - 1493 n_tty_ 13:00 tty6 00:00:00
/sbin/agetty 38400 tty6 linux
5 S root 3457 1 0 80 0 - 5959 poll_s 13:00 ? 00:00:00
SCREEN -S sb1
4 S root 3458 3457 0 80 0 - 4454 wait 13:00 pts/0 00:00:00
-/bin/bash
4 S root 3462 3458 0 75 -5 - 45171 poll_s 13:00 pts/0 00:00:34
/usr/bin/python2.6 /usr/bin/emerge -e world
4 S root 3613 1 0 80 0 - 14014 wait 13:01 tty1 00:00:00
/bin/login --

4 S root      3953  3613  0  80   0 -  4429 n_tty_ 13:01 tty1     00:00:00 -bash
0 S root      6614  3462  0  75  -5 -   972 wait   14:26 pts/0    00:00:00

[dev-util/pkgconfig-0.23] sandbox "/usr/lib64/portage/bin/ebuild.sh" compile
4 S root 6615 6614 0 75 -5 - 6362 wait 14:26 pts/0 00:00:00
/bin/bash /usr/lib64/portage/bin/ebuild.sh compile
5 S root 6646 6615 0 75 -5 - 6745 wait 14:26 pts/0 00:00:00
/bin/bash /usr/lib64/portage/bin/ebuild.sh compile
4 S root 13235 6646 0 75 -5 - 3651 wait 14:27 pts/0 00:00:00
make -j8
4 S root 13238 13235 0 75 -5 - 3652 wait 14:27 pts/0 00:00:00
make all-recursive
4 S root 13239 13238 0 75 -5 - 5956 wait 14:27 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 13243 13239 0 75 -5 - 5956 wait 14:27 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
4 S root 13244 13243 0 75 -5 - 3686 wait 14:27 pts/0 00:00:00
make all
4 S root 13358 13244 0 75 -5 - 3684 wait 14:27 pts/0 00:00:00
make all-recursive
4 S root 13359 13358 0 75 -5 - 5956 wait 14:27 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 16546 13359 0 75 -5 - 5956 wait 14:28 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
4 S root 16547 16546 0 75 -5 - 3652 wait 14:28 pts/0 00:00:00
make all
4 S root 16548 16547 0 75 -5 - 3652 n_tty_ 14:28 pts/0 00:00:00
make all-am
4 S root 16599 3258 0 80 0 - 17937 poll_s 15:07 ? 00:00:00
sshd: root@pts/2

4 S root     16602 16599  0  80   0 -  4429 wait   15:07 pts/2    00:00:00 -bash
4 R root     16611 16602  0  80   0 -  3698 -      15:08 pts/2    00:00:00 ps -elf
1 S root     31506     2  0  80   0 -     0 bdi_wr 14:25 ?        00:00:00

[flush-253:0]

All in wait?

After this test I reboot in 2.6.31-r10 and I complete emerge -e world succefully. The problem show always with all kernels>=2.6.32
I have setup something wrong in kernel? I post the .config in the previous email.

Quoted 17 lines

...

Quoted 12 lines

There are almost impossible to debug.

Try copying vmlinux out of your guest and attach with gdb when it hangs. Then issue the command

(gdb) thread apply all backtrace

to see what the guest is doing.

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to 

panic.
------- End of Original Message -------

Quoted 4 lines

Hi,
I compile gentoo-sources-2.6.31-r10 and with this kernel "emerge -e world" complete without errors!

Interesing. Can you so a git bisect to see where it stops working?

Quoted 26 lines

I always use the same .config

After I try gentoo-sources-2.6.34 and vanilla-sources-2.6.34 but the problem remain, the compile freeze and I see this in ps -elf:

5 S root 1013 1 0 76 -4 - 3125 poll_s 13:00 ? 00:00:00
/sbin/udevd --daemon
1 S root 2669 1 0 80 0 - 7523 wait 13:00 ? 00:00:00
supervising syslog-ng
5 S root 2670 2669 0 80 0 - 7556 poll_s 13:00 ? 00:00:00
/usr/sbin/syslog-ng
1 S root 3258 1 0 80 0 - 9505 poll_s 13:00 ? 00:00:00
/usr/sbin/sshd
1 S root 3378 1 0 80 0 - 4115 hrtime 13:00 ? 00:00:00
/usr/sbin/cron
0 S root 3446 1 0 80 0 - 1493 n_tty_ 13:00 tty2 00:00:00
/sbin/agetty 38400 tty2 linux
0 S root 3447 1 0 80 0 - 1493 n_tty_ 13:00 tty3 00:00:00
/sbin/agetty 38400 tty3 linux
0 S root 3448 1 0 80 0 - 1493 n_tty_ 13:00 tty4 00:00:00
/sbin/agetty 38400 tty4 linux
0 S root 3449 1 0 80 0 - 1493 n_tty_ 13:00 tty5 00:00:00
/sbin/agetty 38400 tty5 linux
0 S root 3450 1 0 80 0 - 1493 n_tty_ 13:00 tty6 00:00:00
/sbin/agetty 38400 tty6 linux
5 S root 3457 1 0 80 0 - 5959 poll_s 13:00 ? 00:00:00

Quoted 8 lines

SCREEN -S sb1
4 S root 3458 3457 0 80 0 - 4454 wait 13:00 pts/0 00:00:00
-/bin/bash
4 S root 3462 3458 0 75 -5 - 45171 poll_s 13:00 pts/0 00:00:34
/usr/bin/python2.6 /usr/bin/emerge -e world
4 S root 3613 1 0 80 0 - 14014 wait 13:01 tty1 00:00:00
/bin/login --
4 S root 3953 3613 0 80 0 - 4429 n_tty_ 13:01 tty1

Quoted 25 lines

0 S root 6614 3462 0 75 -5 - 972 wait 14:26 pts/0 00:00:00
[dev-util/pkgconfig-0.23] sandbox "/usr/lib64/portage/bin/ebuild.sh" compile
4 S root 6615 6614 0 75 -5 - 6362 wait 14:26 pts/0 00:00:00
/bin/bash /usr/lib64/portage/bin/ebuild.sh compile
5 S root 6646 6615 0 75 -5 - 6745 wait 14:26 pts/0 00:00:00
/bin/bash /usr/lib64/portage/bin/ebuild.sh compile
4 S root 13235 6646 0 75 -5 - 3651 wait 14:27 pts/0 00:00:00
make -j8
4 S root 13238 13235 0 75 -5 - 3652 wait 14:27 pts/0 00:00:00
make all-recursive
4 S root 13239 13238 0 75 -5 - 5956 wait 14:27 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 13243 13239 0 75 -5 - 5956 wait 14:27 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
4 S root 13244 13243 0 75 -5 - 3686 wait 14:27 pts/0 00:00:00
make all
4 S root 13358 13244 0 75 -5 - 3684 wait 14:27 pts/0 00:00:00
make all-recursive
4 S root 13359 13358 0 75 -5 - 5956 wait 14:27 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 16546 13359 0 75 -5 - 5956 wait 14:28 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=`echo

Quoted 8 lines

all-recursive | sed s/-recursive//`; \?list=
4 S root 16547 16546 0 75 -5 - 3652 wait 14:28 pts/0 00:00:00
make all
4 S root 16548 16547 0 75 -5 - 3652 n_tty_ 14:28 pts/0 00:00:00
make all-am
4 S root 16599 3258 0 80 0 - 17937 poll_s 15:07 ? 00:00:00
sshd: root@pts/2
4 S root 16602 16599 0 80 0 - 4429 wait 15:07 pts/2 00:00:00

Quoted 1 line

4 R root 16611 16602 0 80 0 - 3698 - 15:08 pts/2 00:00:00

Quoted 5 lines

1 S root 31506 2 0 80 0 - 0 bdi_wr 14:25 ? 00:00:00
[flush-253:0]

All in wait?

Maybe a block driver problem? Are you using virtio?

Quoted 1 line

After this test I reboot in 2.6.31-r10 and I complete emerge -e world

Quoted 2 lines

The problem show always with all kernels>=2.6.32
I have setup something wrong in kernel? I post the .config in the previous

Quoted 2 lines

It should work for all .configs.

---------- Original Message -----------
From: Avi Kivity <[email protected]>
To: Riccardo <[email protected]>
Cc: [email protected]
Sent: Fri, 21 May 2010 18:21:20 +0300
Subject: Re: Gentoo guest with smp: emerge freeze while recompile world

Quoted 12 lines · Riccardo

Quoted 11 lines

...

Quoted 8 lines

There are almost impossible to debug.

Try copying vmlinux out of your guest and attach with gdb when it hangs. Then issue the command

(gdb) thread apply all backtrace

to see what the guest is doing.

Quoted 8 lines

Quoted 6 lines

panic.
------- End of Original Message -------

Hi,
I compile gentoo-sources-2.6.31-r10 and with this kernel "emerge -e world" complete without errors!

Interesing. Can you so a git bisect to see where it stops working?

Ehm sorry I don't understand the request have you a link?

Quoted 26 lines

Quoted 26 lines

I always use the same .config

After I try gentoo-sources-2.6.34 and vanilla-sources-2.6.34 but the problem remain, the compile freeze and I see this in ps -elf:

5 S root 1013 1 0 76 -4 - 3125 poll_s 13:00 ?
00:00:00 /sbin/udevd --daemon
1 S root 2669 1 0 80 0 - 7523 wait 13:00 ?
00:00:00 supervising syslog-ng
5 S root 2670 2669 0 80 0 - 7556 poll_s 13:00 ?
00:00:00 /usr/sbin/syslog-ng
1 S root 3258 1 0 80 0 - 9505 poll_s 13:00 ?
00:00:00 /usr/sbin/sshd
1 S root 3378 1 0 80 0 - 4115 hrtime 13:00 ?
00:00:00 /usr/sbin/cron
0 S root 3446 1 0 80 0 - 1493 n_tty_ 13:00 tty2
00:00:00 /sbin/agetty 38400 tty2 linux
0 S root 3447 1 0 80 0 - 1493 n_tty_ 13:00 tty3
00:00:00 /sbin/agetty 38400 tty3 linux
0 S root 3448 1 0 80 0 - 1493 n_tty_ 13:00 tty4
00:00:00 /sbin/agetty 38400 tty4 linux
0 S root 3449 1 0 80 0 - 1493 n_tty_ 13:00 tty5
00:00:00 /sbin/agetty 38400 tty5 linux
0 S root 3450 1 0 80 0 - 1493 n_tty_ 13:00 tty6
00:00:00 /sbin/agetty 38400 tty6 linux
5 S root 3457 1 0 80 0 - 5959 poll_s 13:00 ?

Quoted 8 lines

Quoted 8 lines

00:00:00 SCREEN -S sb1
4 S root 3458 3457 0 80 0 - 4454 wait 13:00 pts/0
00:00:00 -/bin/bash
4 S root 3462 3458 0 75 -5 - 45171 poll_s 13:00 pts/0
00:00:34 /usr/bin/python2.6 /usr/bin/emerge -e world
4 S root 3613 1 0 80 0 - 14014 wait 13:01 tty1
00:00:00 /bin/login --
4 S root 3953 3613 0 80 0 - 4429 n_tty_ 13:01 tty1

00:00:00 -bash

Quoted 27 lines

Quoted 27 lines

0 S root 6614 3462 0 75 -5 - 972 wait 14:26 pts/0
00:00:00 [dev-util/pkgconfig-0.23] sandbox
"/usr/lib64/portage/bin/ebuild.sh" compile 4 S root 6615 6614
0 75 -5 - 6362 wait 14:26 pts/0 00:00:00 /bin/bash
/usr/lib64/portage/bin/ebuild.sh compile
5 S root 6646 6615 0 75 -5 - 6745 wait 14:26 pts/0
00:00:00 /bin/bash /usr/lib64/portage/bin/ebuild.sh compile
4 S root 13235 6646 0 75 -5 - 3651 wait 14:27 pts/0
00:00:00 make -j8
4 S root 13238 13235 0 75 -5 - 3652 wait 14:27 pts/0
00:00:00 make all-recursive
4 S root 13239 13238 0 75 -5 - 5956 wait 14:27 pts/0
00:00:00 /bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 13243 13239 0 75 -5 - 5956 wait 14:27 pts/0
00:00:00 /bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
4 S root 13244 13243 0 75 -5 - 3686 wait 14:27 pts/0
00:00:00 make all
4 S root 13358 13244 0 75 -5 - 3684 wait 14:27 pts/0
00:00:00 make all-recursive
4 S root 13359 13358 0 75 -5 - 5956 wait 14:27 pts/0
00:00:00 /bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 16546 13359 0 75 -5 - 5956 wait 14:28 pts/0
00:00:00 /bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=

Quoted 8 lines

Quoted 8 lines

4 S root 16547 16546 0 75 -5 - 3652 wait 14:28 pts/0
00:00:00 make all
4 S root 16548 16547 0 75 -5 - 3652 n_tty_ 14:28 pts/0
00:00:00 make all-am
4 S root 16599 3258 0 80 0 - 17937 poll_s 15:07 ?
00:00:00 sshd: root@pts/2
4 S root 16602 16599 0 80 0 - 4429 wait 15:07 pts/2
00:00:00

-bash

Quoted 2 lines

Quoted 2 lines

4 R root 16611 16602 0 80 0 - 3698 - 15:08 pts/2
00:00:00

ps -elf

Quoted 6 lines

Quoted 4 lines

1 S root 31506 2 0 80 0 - 0 bdi_wr 14:25 ?
00:00:00 [flush-253:0]

All in wait?

Maybe a block driver problem? Are you using virtio?

Yes, I always used virtio.
In the server the kernel is sys-kernel/gentoo-sources-2.6.33 and in dmesg I see this new lines about clock?:

 [  885.500098] CE: hpet increasing min_delta_ns to 15000 nsec
 [ 1354.020020] CE: hpet increasing min_delta_ns to 22500 nsec
 [ 2657.777387] hrtimer: interrupt took 7542 ns
 [ 7148.533547] CE: hpet increasing min_delta_ns to 33750 nsec

Quoted 2 lines

Quoted 2 lines

After this test I reboot in 2.6.31-r10 and I complete emerge -e world

succefully.

Quoted 3 lines

Quoted 3 lines

The problem show always with all kernels>=2.6.32
I have setup something wrong in kernel? I post the .config in the previous

email.

Quoted 1 line

It should work for all .configs.

Thanks for your checks.

------- End of Original Message -------
Now I have leave the VM freeze.
If you can debug I can send you the data for connect to the VM.

Riccardo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 117 lines

---------- Original Message -----------
From: Avi Kivity <[email protected]>
To: Riccardo <[email protected]>
Cc: [email protected]
Sent: Fri, 21 May 2010 18:21:20 +0300
Subject: Re: Gentoo guest with smp: emerge freeze while recompile world

Quoted 12 lines · Riccardo

Quoted 11 lines

...

Quoted 8 lines

There are almost impossible to debug.

Try copying vmlinux out of your guest and attach with gdb when it hangs. Then issue the command

(gdb) thread apply all backtrace

to see what the guest is doing.

Quoted 8 lines

Quoted 6 lines

panic.
------- End of Original Message -------

Hi,
I compile gentoo-sources-2.6.31-r10 and with this kernel "emerge -e world" complete without errors!

Interesing. Can you so a git bisect to see where it stops working?

Ehm sorry I don't understand the request have you a link?

Quoted 26 lines

Quoted 26 lines

I always use the same .config

After I try gentoo-sources-2.6.34 and vanilla-sources-2.6.34 but the problem remain, the compile freeze and I see this in ps -elf:

5 S root 1013 1 0 76 -4 - 3125 poll_s 13:00 ?
00:00:00 /sbin/udevd --daemon
1 S root 2669 1 0 80 0 - 7523 wait 13:00 ?
00:00:00 supervising syslog-ng
5 S root 2670 2669 0 80 0 - 7556 poll_s 13:00 ?
00:00:00 /usr/sbin/syslog-ng
1 S root 3258 1 0 80 0 - 9505 poll_s 13:00 ?
00:00:00 /usr/sbin/sshd
1 S root 3378 1 0 80 0 - 4115 hrtime 13:00 ?
00:00:00 /usr/sbin/cron
0 S root 3446 1 0 80 0 - 1493 n_tty_ 13:00 tty2
00:00:00 /sbin/agetty 38400 tty2 linux
0 S root 3447 1 0 80 0 - 1493 n_tty_ 13:00 tty3
00:00:00 /sbin/agetty 38400 tty3 linux
0 S root 3448 1 0 80 0 - 1493 n_tty_ 13:00 tty4
00:00:00 /sbin/agetty 38400 tty4 linux
0 S root 3449 1 0 80 0 - 1493 n_tty_ 13:00 tty5
00:00:00 /sbin/agetty 38400 tty5 linux
0 S root 3450 1 0 80 0 - 1493 n_tty_ 13:00 tty6
00:00:00 /sbin/agetty 38400 tty6 linux
5 S root 3457 1 0 80 0 - 5959 poll_s 13:00 ?

Quoted 8 lines

Quoted 8 lines

00:00:00 SCREEN -S sb1
4 S root 3458 3457 0 80 0 - 4454 wait 13:00 pts/0
00:00:00 -/bin/bash
4 S root 3462 3458 0 75 -5 - 45171 poll_s 13:00 pts/0
00:00:34 /usr/bin/python2.6 /usr/bin/emerge -e world
4 S root 3613 1 0 80 0 - 14014 wait 13:01 tty1
00:00:00 /bin/login --
4 S root 3953 3613 0 80 0 - 4429 n_tty_ 13:01 tty1

00:00:00 -bash

Quoted 27 lines

Quoted 27 lines

0 S root 6614 3462 0 75 -5 - 972 wait 14:26 pts/0
00:00:00 [dev-util/pkgconfig-0.23] sandbox
"/usr/lib64/portage/bin/ebuild.sh" compile 4 S root 6615 6614
0 75 -5 - 6362 wait 14:26 pts/0 00:00:00 /bin/bash
/usr/lib64/portage/bin/ebuild.sh compile
5 S root 6646 6615 0 75 -5 - 6745 wait 14:26 pts/0
00:00:00 /bin/bash /usr/lib64/portage/bin/ebuild.sh compile
4 S root 13235 6646 0 75 -5 - 3651 wait 14:27 pts/0
00:00:00 make -j8
4 S root 13238 13235 0 75 -5 - 3652 wait 14:27 pts/0
00:00:00 make all-recursive
4 S root 13239 13238 0 75 -5 - 5956 wait 14:27 pts/0
00:00:00 /bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 13243 13239 0 75 -5 - 5956 wait 14:27 pts/0
00:00:00 /bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
4 S root 13244 13243 0 75 -5 - 3686 wait 14:27 pts/0
00:00:00 make all
4 S root 13358 13244 0 75 -5 - 3684 wait 14:27 pts/0
00:00:00 make all-recursive
4 S root 13359 13358 0 75 -5 - 5956 wait 14:27 pts/0
00:00:00 /bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 16546 13359 0 75 -5 - 5956 wait 14:28 pts/0
00:00:00 /bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=

Quoted 8 lines

Quoted 8 lines

4 S root 16547 16546 0 75 -5 - 3652 wait 14:28 pts/0
00:00:00 make all
4 S root 16548 16547 0 75 -5 - 3652 n_tty_ 14:28 pts/0
00:00:00 make all-am
4 S root 16599 3258 0 80 0 - 17937 poll_s 15:07 ?
00:00:00 sshd: root@pts/2
4 S root 16602 16599 0 80 0 - 4429 wait 15:07 pts/2
00:00:00

-bash

Quoted 2 lines

Quoted 2 lines

4 R root 16611 16602 0 80 0 - 3698 - 15:08 pts/2
00:00:00

ps -elf

Quoted 6 lines

Quoted 4 lines

1 S root 31506 2 0 80 0 - 0 bdi_wr 14:25 ?
00:00:00 [flush-253:0]

All in wait?

Maybe a block driver problem? Are you using virtio?

Quoted 7 lines

Yes, I always used virtio.
In the server the kernel is sys-kernel/gentoo-sources-2.6.33 and in dmesg I see this new lines about clock?:

 [  885.500098] CE: hpet increasing min_delta_ns to 15000 nsec 
 [ 1354.020020] CE: hpet increasing min_delta_ns to 22500 nsec 
 [ 2657.777387] hrtimer: interrupt took 7542 ns 
 [ 7148.533547] CE: hpet increasing min_delta_ns to 33750 nsec 

If you are using kvm-clock, maybe try disabling that.

Quoted 19 lines

Quoted 2 lines

Quoted 2 lines

After this test I reboot in 2.6.31-r10 and I complete emerge -e world

succefully.

Quoted 3 lines

Quoted 3 lines

The problem show always with all kernels>=2.6.32
I have setup something wrong in kernel? I post the .config in the previous

email.

Quoted 1 line

It should work for all .configs.

Thanks for your checks.

------- End of Original Message -------
Now I have leave the VM freeze.
If you can debug I can send you the data for connect to the VM.

Quoted 5 lines

Riccardo

Signature · 3 lines

To unsubscribe from this list: send the line "unsubscribe kvm" in 
the body of a message to [email protected] 
More majordomo info at  http://vger.kernel.org/majordomo-info.html 

Quoted 4 lines

If you are using kvm-clock, maybe try disabling that.

$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource
hpet

This is from server, not from VM (that have freeze).

Quoted 4 lines

Quoted 4 lines

If you are using kvm-clock, maybe try disabling that.

Quoted 5 lines

$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource 
hpet 

This is from server, not from VM (that have freeze).

What about the guest?

Signature · 1 line

error compiling committee.c: too many arguments to function 

---------- Original Message -----------
From: Avi Kivity<[email protected]>
To: Riccardo<[email protected]>
Cc: [email protected]
Sent: Sun, 23 May 2010 14:38:42 +0300
Subject: Re: Gentoo guest with smp: emerge freeze while recompile world

Quoted 7 lines · Riccardo

Quoted 6 lines

Quoted 6 lines

If you are using kvm-clock, maybe try disabling that.

Quoted 11 lines

Quoted 6 lines

$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource
hpet

This is from server, not from VM (that have freeze).

What about the guest?

Signature · 1 line

error compiling committee.c: too many arguments to function

------- End of Original Message -------

# dmesg | grep clock
[    0.000000] kvm-clock: cpu 0, msr 0:1845bc1, boot clock
[    0.000000] kvm-clock: cpu 0, msr 0:28047bc1, primary cpu clock
[    0.000999] hpet clockevent registered
[    0.084027] kvm-clock: cpu 1, msr 0:28063bc1, secondary cpu clock
[    0.102059] kvm-clock: cpu 2, msr 0:2807fbc1, secondary cpu clock
[    0.120035] kvm-clock: cpu 3, msr 0:2809bbc1, secondary cpu clock
[    0.148051] kvm-clock: cpu 4, msr 0:280b7bc1, secondary cpu clock
[    0.173039] kvm-clock: cpu 5, msr 0:280d3bc1, secondary cpu clock
# cat /sys/devices/system/clocksource/clocksource0/current_clocksource
kvm-clock
# cat /sys/devices/system/clocksource/clocksource0/available_clocksource
kvm-clock tsc hpet acpi_pm

Do you think it's a related issue with the system freeze under compiling with newer kernel?

Quoted 28 lines

---------- Original Message -----------
From: Avi Kivity<[email protected]>
To: Riccardo<[email protected]>
Cc: [email protected]
Sent: Sun, 23 May 2010 14:38:42 +0300
Subject: Re: Gentoo guest with smp: emerge freeze while recompile world

Quoted 7 lines · Riccardo

Quoted 6 lines

Quoted 6 lines

If you are using kvm-clock, maybe try disabling that.

Quoted 11 lines

Quoted 6 lines

$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource 
hpet 

This is from server, not from VM (that have freeze).

What about the guest?

Signature · 1 line

error compiling committee.c: too many arguments to function 

------- End of Original Message -------

Quoted 15 lines

# dmesg | grep clock 
[    0.000000] kvm-clock: cpu 0, msr 0:1845bc1, boot clock 
[    0.000000] kvm-clock: cpu 0, msr 0:28047bc1, primary cpu clock 
[    0.000999] hpet clockevent registered 
[    0.084027] kvm-clock: cpu 1, msr 0:28063bc1, secondary cpu clock 
[    0.102059] kvm-clock: cpu 2, msr 0:2807fbc1, secondary cpu clock 
[    0.120035] kvm-clock: cpu 3, msr 0:2809bbc1, secondary cpu clock 
[    0.148051] kvm-clock: cpu 4, msr 0:280b7bc1, secondary cpu clock 
[    0.173039] kvm-clock: cpu 5, msr 0:280d3bc1, secondary cpu clock 
# cat /sys/devices/system/clocksource/clocksource0/current_clocksource 
kvm-clock 
# cat /sys/devices/system/clocksource/clocksource0/available_clocksource 
kvm-clock tsc hpet acpi_pm 

Please try with kvmclock disabled.

Quoted 3 lines

Do you think it's a related issue with the system freeze under compiling with newer kernel?

That's what we're trying to find out.

Quoted 3 lines

Please try with kvmclock disabled.

I have recompile gentoo-sources-2.6.34 without kvm-clock:

# cat /sys/devices/system/clocksource/clocksource0/current_clocksource
tsc
# dmesg | grep clock
[    0.000000] hpet clockevent registered
[    0.661050] Switching to clocksource tsc

And with this kernel all working fine! (emerge -e world) It's a problem in the kvm-clock for kernel>=2.6.32

Quoted 7 lines

4 S root 3458 3457 0 80 0 - 4454 wait 13:00 pts/0 00:00:00
-/bin/bash
4 S root 3462 3458 0 75 -5 - 45171 poll_s 13:00 pts/0 00:00:34
/usr/bin/python2.6 /usr/bin/emerge -e world
4 S root 3613 1 0 80 0 - 14014 wait 13:01 tty1 00:00:00
/bin/login --
4 S root 3953 3613 0 80 0 - 4429 n_tty_ 13:01 tty1

Quoted 25 lines

0 S root 6614 3462 0 75 -5 - 972 wait 14:26 pts/0 00:00:00
[dev-util/pkgconfig-0.23] sandbox "/usr/lib64/portage/bin/ebuild.sh" compile
4 S root 6615 6614 0 75 -5 - 6362 wait 14:26 pts/0 00:00:00
/bin/bash /usr/lib64/portage/bin/ebuild.sh compile
5 S root 6646 6615 0 75 -5 - 6745 wait 14:26 pts/0 00:00:00
/bin/bash /usr/lib64/portage/bin/ebuild.sh compile
4 S root 13235 6646 0 75 -5 - 3651 wait 14:27 pts/0 00:00:00
make -j8
4 S root 13238 13235 0 75 -5 - 3652 wait 14:27 pts/0 00:00:00
make all-recursive
4 S root 13239 13238 0 75 -5 - 5956 wait 14:27 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 13243 13239 0 75 -5 - 5956 wait 14:27 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
4 S root 13244 13243 0 75 -5 - 3686 wait 14:27 pts/0 00:00:00
make all
4 S root 13358 13244 0 75 -5 - 3684 wait 14:27 pts/0 00:00:00
make all-recursive
4 S root 13359 13358 0 75 -5 - 5956 wait 14:27 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=echo all-recursive | sed s/-recursive//; \?list=
5 S root 16546 13359 0 75 -5 - 5956 wait 14:28 pts/0 00:00:00
/bin/sh -c set fnord $MAKEFLAGS; amf=$2; \?dot_seen=no; \?target=`echo

Quoted 8 lines

all-recursive | sed s/-recursive//`; \?list=
4 S root 16547 16546 0 75 -5 - 3652 wait 14:28 pts/0 00:00:00
make all
4 S root 16548 16547 0 75 -5 - 3652 n_tty_ 14:28 pts/0 00:00:00
make all-am
4 S root 16599 3258 0 80 0 - 17937 poll_s 15:07 ? 00:00:00
sshd: root@pts/2
4 S root 16602 16599 0 80 0 - 4429 wait 15:07 pts/2 00:00:00

Quoted 1 line

4 R root 16611 16602 0 80 0 - 3698 - 15:08 pts/2 00:00:00

Quoted 5 lines

1 S root 31506 2 0 80 0 - 0 bdi_wr 14:25 ? 00:00:00
[flush-253:0]

All in wait?

Maybe a block driver problem? Are you using virtio?

=2.6.32