KVM Migration fails

DISCUSSION jd 4 participants 7 messages Sep 25, 2008 KVM

lore

Share & tools

lore

jd

author

2008-09-25 04:22 UTC #0 lore raw

Hi
I have a setup using shared nfs disks. When migration is attempted, it fails... any ideas on how to debug this..?

/Jd

Details
=======

migration: write failed (Connection reset by peer)^M Migration failed! ret=0 error=9

Source : KVM-73, Cent OS 5.2, 64 bit.

qemu-system-x86_64 -net nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m 1024 -no-acpi -vnc :22 -name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait -pidfile /var/run/kvm/pids/XPSP2-KVM -daemonize

Dest : KVM-70, Fedora 8, 64bit

qemu-system-x86_64 -net nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m 1024 -no-acpi -vnc :23 -incoming tcp://0:8002 -name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait -pidfile /var/run/kvm/pids/XPSP2-KVM -daemonize

Chris Lalancette

2008-09-25 05:56 UTC #1 lore raw

Quoted 9 lines · jd

Hi I have a setup using shared nfs disks. When migration is attempted, it fails... any ideas on how to debug this..?
/Jd
Details =======
migration: write failed (Connection reset by peer)^M Migration failed! ret=0 error=9

Not a lot of detail here, but when does it fail during the process? Immediately? After transferring some of the image? Right at the end? Connection reset by peer sort of sounds like there is a firewall or something in the way, although if it transfers some data before crashing, then it's probably something else. The only other thing I can think of based on your command-line below is something to do with your -net options; I've only successfully migrated with -net tap before, I've never tried -net user.

Chris Lalancette

Quoted 18 lines

Source : KVM-73, Cent OS 5.2, 64 bit.
qemu-system-x86_64 -net nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0
(quoted diff) +0 -2
-hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m 1024 -no-acpi  -vnc :22
-name XPSP2-KVM -smp 2 -monitor
unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait -pidfile /var/run/kvm/pids/XPSP2-KVM -daemonize
Dest : KVM-70, Fedora 8, 64bit
qemu-system-x86_64 -net nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0
(quoted diff) +0 -2
-hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m 1024 -no-acpi  -vnc :23
-incoming tcp://0:8002 -name XPSP2-KVM -smp 2 -monitor
unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait -pidfile /var/run/kvm/pids/XPSP2-KVM -daemonize

jd

author

2008-09-25 07:18 UTC #2 lore raw

Hi Chris,

Thanks for the response.

It errors out bit after the command is given.. so I guess... it must have transferred some pages/bits.

What kind of information do you want me to collect ? Is there a way to turn on logging..debugging for kvm process ?

Will also try tap option.

thanks.
/Jd

Quoted 64 lines · Chris Lalancette

From: Chris Lalancette <[email protected]>
Subject: Re: KVM Migration fails
To: [email protected]
Cc: "KVM List" <[email protected]>
Date: Wednesday, September 24, 2008, 10:56 PM
Quoted 1 line · jd
Hi I have a setup using shared nfs disks. When
migration is attempted, it
Quoted 7 lines
fails... any ideas on how to debug this..?
/Jd
Details =======
migration: write failed (Connection reset by peer)^M
Migration failed! ret=0
Quoted 1 line
error=9
Not a lot of detail here, but when does it fail during the process?
Immediately? After transferring some of the image? Right at the end?
Connection reset by peer sort of sounds like there is a firewall or something in
the way, although if it transfers some data before crashing, then it's probably
something else. The only other thing I can think of based on your command-line
below is something to do with your -net options; I've only successfully migrated
with -net tap before, I've never tried -net user.
Chris Lalancette
Quoted 4 lines
Source : KVM-73, Cent OS 5.2, 64 bit.
qemu-system-x86_64 -net
nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0
Quoted 1 line
-hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m
1024 -no-acpi -vnc :22
Quoted 2 lines
-name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait
-pidfile
Quoted 7 lines
/var/run/kvm/pids/XPSP2-KVM -daemonize
Dest : KVM-70, Fedora 8, 64bit
qemu-system-x86_64 -net
nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0
Quoted 1 line
-hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m
1024 -no-acpi -vnc :23
Quoted 2 lines
-incoming tcp://0:8002 -name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait
-pidfile
Quoted 1 line
/var/run/kvm/pids/XPSP2-KVM -daemonize
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html

Yang, Sheng

2008-09-25 06:23 UTC #3 lore raw

In reply to jd

Quoted 3 lines · On Thursday 25 September 2008 12:22:42 jd wrote:

Hi
I have a setup using shared nfs disks. When migration is attempted, it
fails... any ideas on how to debug this..?

It's a regression bug recently.

Please refer to

https://sourceforge.net/tracker/index.php?func=detail&aid=2106661&group_id=180599&atid=893831

I think a git bisect can also help.
--
regards
Yang, Sheng

Quoted 37 lines

/Jd
Details
=======
migration: write failed (Connection reset by peer)^M Migration failed! ret=0 error=9
Source : KVM-73, Cent OS 5.2, 64 bit.
qemu-system-x86_64 -net nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m 1024 -no-acpi -vnc :22 -name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait -pidfile /var/run/kvm/pids/XPSP2-KVM -daemonize
Dest : KVM-70, Fedora 8, 64bit
qemu-system-x86_64 -net nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m 1024 -no-acpi -vnc :23 -incoming tcp://0:8002 -name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait -pidfile /var/run/kvm/pids/XPSP2-KVM -daemonize
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

jd

author

2008-09-25 07:23 UTC #4 lore raw

The error code and the messages seem bit different here.

/Jd

Quoted 74 lines · Sheng

From: Yang, Sheng <[email protected]>
Subject: Re: KVM Migration fails
To: [email protected], "[email protected]" <[email protected]>
Date: Wednesday, September 24, 2008, 11:23 PM
Quoted 2 lines · On Thursday 25 September 2008 12:22:42 jd wrote:
Hi
I have a setup using shared nfs disks. When
migration is attempted, it
Quoted 1 line
fails... any ideas on how to debug this..?
It's a regression bug recently.
Please refer to
https://sourceforge.net/tracker/index.php?func=detail&aid=2106661&group_id=180599&atid=893831
I think a git bisect can also help.
--
regards
Yang, Sheng
Quoted 6 lines
/Jd
Details
=======
Quoted 6 lines
migration: write failed (Connection reset by peer)^M Migration failed! ret=0 error=9
Source : KVM-73, Cent OS 5.2, 64 bit.
qemu-system-x86_64 -net
nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net
Quoted 1 line
user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm
-boot c -m 1024
Quoted 2 lines
-no-acpi -vnc :22 -name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait
-pidfile
Quoted 7 lines
/var/run/kvm/pids/XPSP2-KVM -daemonize
Dest : KVM-70, Fedora 8, 64bit
qemu-system-x86_64 -net
nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net
Quoted 1 line
user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm
-boot c -m 1024
Quoted 1 line
-no-acpi -vnc :23 -incoming tcp://0:8002 -name
XPSP2-KVM -smp 2 -monitor
Quoted 1 line
unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait
-pidfile
Quoted 10 lines
/var/run/kvm/pids/XPSP2-KVM -daemonize
--
To unsubscribe from this list: send the line
"unsubscribe kvm" in
Quoted 2 lines
the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html

Yang, Sheng

2008-09-25 07:31 UTC #5 lore raw

Quoted 2 lines · On Thursday 25 September 2008 15:23:22 jd wrote:

The error code and the messages seem bit different here.

Um... At least I believe it's a regression, and our migration test never success after that. So I think it's worth to look into it. If you are lucky enough, you would got two bugs. :)

--
regards
Yang, Sheng

Quoted 103 lines

/Jd
Quoted 89 lines · Sheng
From: Yang, Sheng <[email protected]>
Subject: Re: KVM Migration fails
To: [email protected], "[email protected]" <[email protected]>
Date: Wednesday, September 24, 2008, 11:23 PM
Quoted 2 lines · On Thursday 25 September 2008 12:22:42 jd wrote:
Hi
I have a setup using shared nfs disks. When
migration is attempted, it
Quoted 1 line
fails... any ideas on how to debug this..?
It's a regression bug recently.
Please refer to
https://sourceforge.net/tracker/index.php?func=detail&aid=2106661&group_i
d=180599&atid=893831
I think a git bisect can also help.
--
regards
Yang, Sheng
Quoted 11 lines
/Jd
Details
=======
migration: write failed (Connection reset by peer)^M Migration failed! ret=0 error=9
Source : KVM-73, Cent OS 5.2, 64 bit.
qemu-system-x86_64 -net
nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net
Quoted 1 line
user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm
-boot c -m 1024
Quoted 2 lines
-no-acpi -vnc :22 -name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait
-pidfile
Quoted 7 lines
/var/run/kvm/pids/XPSP2-KVM -daemonize
Dest : KVM-70, Fedora 8, 64bit
qemu-system-x86_64 -net
nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net
Quoted 1 line
user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm
-boot c -m 1024
Quoted 1 line
-no-acpi -vnc :23 -incoming tcp://0:8002 -name
XPSP2-KVM -smp 2 -monitor
Quoted 1 line
unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait
-pidfile
Quoted 10 lines
/var/run/kvm/pids/XPSP2-KVM -daemonize
--
To unsubscribe from this list: send the line
"unsubscribe kvm" in
Quoted 2 lines
the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
Quoted 6 lines
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Uri Lublin

2008-09-25 07:34 UTC #6 lore raw

In reply to jd

Quoted 13 lines · jd

Hi
I have a setup using shared nfs disks. When migration is attempted, it fails... any ideas on how to debug this..?
/Jd
Details
=======
migration: write failed (Connection reset by peer)^M Migration failed! ret=0 error=9
Source : KVM-73, Cent OS 5.2, 64 bit.

Quoted 8 lines

qemu-system-x86_64 -net nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m 1024 -no-acpi -vnc :22 -name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait -pidfile /var/run/kvm/pids/XPSP2-KVM -daemonize
Dest : KVM-70, Fedora 8, 64bit
qemu-system-x86_64 -net nic,vlan=0,macaddr=00:16:3e:16:f4:f0 -net user,vlan=0 -hda /mnt/nfs/vmdisks/XPSP2-KVM.disk.xm -boot c -m 1024 -no-acpi -vnc :23 -incoming tcp://0:8002 -name XPSP2-KVM -smp 2 -monitor unix:/var/run/kvm/monitors/XPSP2-KVM,server,nowait -pidfile /var/run/kvm/pids/XPSP2-KVM -daemonize

Hi,

The error on the source is "write failed ...". Many times that means the destination has exited due to an error on its side. Do you see any error message on the destination ?

Uri.