xfrd inifinite loop reloading pages

PATCH In review Charles Coffing 2 participants 3 messages Mar 17, 2005 – Mar 18, 2005 Xen hypervisor interface

Share & tools

Charles Coffing

author

2005-03-17 23:27 UTC #0 lore raw

It's a live migration. The domain is idle -- booted, but basically 0% cpu usage. I gave the domain 256MB, but hadn't used the balloon driver.
I left one machine receiving the migration for well over 10 minutes,
but it never unpaused the domain. The xfrd log was just curning with ever increasing percentages, no other messages.

Also, it appears as if there is some hysteresis at work here. Migration failed on my first two attempts, but all subsequent attempts today have worked.

Aaahhh... now I'm remembering better: When I first saw this, months back, I poked around and found some possible corner cases where EOF could be missed in libxutil/file_stream.c and elsewhere. I suspect that's related. Let me see if I can get a patch together for that...

Thanks.

Quoted 7 lines

Quoted 1 line
Quoted 1 line
"Ian Pratt" <[email protected]> 03/17/05 1:41 pm >>>
I'm seeing an infinite loop in xfrd on the destination physical machine,
when migrating a domain. It passes 100% while reloading memory pages, and keeps going. I've seen this on an older build (an early 2.0.x IIRC)
but have never yet had time to dig into it. Anyone else seen this?

Are you using the '-l' live option to migrate? If you are, going past 100% isn't actually a bug (though looping forever obviously is).

What are you seeing in the sending xfrd log? Have you used the ballon driver or mem= command line options to the domain?

Also, what's running in the domain you are trying to migrate?

Best,
Ian

Quoted 67 lines

Here's /var/log/xfrd.log:
9657 [INF] XFRD> Accepted connection from 151.155.190.17:32859 on 2 10346 [INF] XFRD> Xfr service for 151.155.190.17:32859 [DEBUG] Conn_init> flags=1
[DEBUG] Conn_init> write stream...
[DEBUG] stream_init>mode=w flags=1 compress=0
[DEBUG] stream_init> unbuffer...
[DEBUG] stream_init< err=0
[DEBUG] Conn_init> read stream...
[DEBUG] stream_init>mode=r flags=1 compress=0
[DEBUG] stream_init> unbuffer...
[DEBUG] stream_init< err=0
[DEBUG] Conn_sxpr>
(xfr.hello 1 0)[DEBUG] Conn_sxpr< err=0
[DEBUG] Conn_sxpr>
(xfr.xfr 8)[DEBUG] Conn_sxpr< err=0
[1111078930.273966] xc_linux_restore start 
xc_linux_restore start
[1111078930.287904] Created domain 1 
Created domain 1
(Domain-1 Domain-0)'domain id=1 name=SuSE.33 memory=256 console=9601 image=/shared/boot/vmlinuz-2.6.10-xenU'[1111078930.389380] Reloading
memory pages:   0% 
Reloading memory pages:   0  6% 
^H^H^H^H 6 12%
^H^H^H^H 12 17%
^H^H^H^H 17 23%
^H^H^H^H 23 28%
^H^H^H^H 28 34%
^H^H^H^H 34 39%
^H^H^H^H 39 45%
^H^H^H^H 45 50%
^H^H^H^H 50 56%
^H^H^H^H 56 62%
^H^H^H^H 62 67%
^H^H^H^H 67 73%
^H^H^H^H 73 78%
^H^H^H^H 78 84%
^H^H^H^H 84 89%
^H^H^H^H 89 95%
^H^H^H^H 95100%
^H^H^H^H100105%
^H^H^H^H105110%
^H^H^H^H110115%
^H^H^H^H115120%
^H^H^H^H120125%
^H^H^H^H125130%
^H^H^H^H130135%
^H^H^H^H135140%
^H^H^H^H140145%
^H^H^H^H145150%
^H^H^H^H150155%
[chomp]
-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Quoted 4 lines

Xen-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xen-devel

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Nivedita Singhvi

2005-03-17 23:53 UTC #1 lore raw

PATCH 1/1 mem alloc handling in xlvbd_init()

Minor nit: would be nice to die gracefully if we fail to allocate memory in xlvbd_init() instead of blowing up in xlvbd_get_vbd_info().

thanks,
Nivedita

drivers/xen/blkfront/vbd.c +5 -0

diff -urN linux-2.6.11-xen-sparse/drivers/xen/blkfront/vbd.c linux-2.6.11-xen-sparse.new/drivers/xen/blkfront/vbd.c
@@ -548,6 +548,11 @@
     }
 
     vbd_info = kmalloc(MAX_VBDS * sizeof(vdisk_t), GFP_KERNEL);
+    if (unlikely(vbd_info == NULL)) {
+	printk("KERN_ALERT Failed to allocate memory for disk info\n");
+	nr_vbds = 0;
+	return 0;	
+    }
     nr_vbds  = xlvbd_get_vbd_info(vbd_info);
 
     if (nr_vbds < 0) {

Nivedita Singhvi

2005-03-18 00:41 UTC #2 lore raw

Re: [Patch xen-unstable] mem alloc handling in xlvbd_init()

Quoted 3 lines · Nivedita Singhvi

Minor nit: would be nice to die gracefully if we fail to allocate memory in xlvbd_init() instead of blowing up in xlvbd_get_vbd_info().

I was about to post a bunch of patches in this area and just rechecked the current bk tree and saw that this has already been updated..Rats, please ignore..

thanks,
Nivedita