fix stubdom memory corruption
PATCH 1/1 fix stubdom memory corruption
Hi all,
this patch fixes a memory corruption in blkfront that happens every time we pass a sector aligned buffer (instead of a page aligned buffer) to blkfront_aio.
To trigger the COW we have to write at least a byte to each page of the buffer, but we must be careful not to overwrite useful content.
extras/mini-os/blkfront.c +7 -5
@@ -317,19 +317,21 @@ req->sector_number = aiocbp->aio_offset / dev->info.sector_size;for (j =0 ; j < n; j++) { + req->seg[j].first_sect =0 ; + req->seg[j].last_sect =PAGE_SIZE / dev->info.sector_size -1 ; + } + req->seg[0 ].first_sect = ((uintptr_t )aiocbp->aio_buf & ~PAGE_MASK ) / dev->info.sector_size; + req->seg[n-1 ].last_sect = (((uintptr_t )aiocbp->aio_buf + aiocbp->aio_nbytes -1 ) & ~PAGE_MASK ) / dev->info.sector_size; +for (j =0 ; j < n; j++) {uintptr_t data = start + j *PAGE_SIZE ;if (!write) {/* Trigger CoW if needed */ - *(char *)data =0 ; + *(char *)(data + (req->seg[j].first_sect <<9 )) =0 ;barrier (); } aiocbp->gref[j] = req->seg[j].gref =gnttab_grant_access (dev->dom,virtual_to_mfn (data), write); - req->seg[j].first_sect =0 ; - req->seg[j].last_sect =PAGE_SIZE / dev->info.sector_size -1 ; } - req->seg[0 ].first_sect = ((uintptr_t )aiocbp->aio_buf & ~PAGE_MASK ) / dev->info.sector_size; - req->seg[n-1 ].last_sect = (((uintptr_t )aiocbp->aio_buf + aiocbp->aio_nbytes -1 ) & ~PAGE_MASK ) / dev->info.sector_size; dev->ring.req_prod_pvt = i +1 ;
This one and vl.c patch have been applied.
stubdom has been rebuilt and reinstalled.
No improvement.
Name ID Mem VCPUs State Time(s) Domain-0 0 7019 2 r----- 633.9 RHELhvm 1 1024 1 ------ 0.0
root@ServerXen331:/etc/xen# netstat -a|grep 590 tcp 0 0 *:5901 *:* LISTEN
Boris
--- On Tue, 4/14/09, Stefano Stabellini <[email protected]> wrote:
From: Stefano Stabellini <[email protected]>
Subject: [Xen-devel] [PATCH] fix stubdom memory corruption
To: "xen-devel" <[email protected]>
Date: Tuesday, April 14, 2009, 5:27 AM
Hi all,
this patch fixes a memory corruption in blkfront that happens every time we pass a sector aligned buffer (instead of a page aligned buffer) to blkfront_aio.
To trigger the COW we have to write at least a byte to each page of the buffer, but we must be careful not to overwrite useful content.
extras/mini-os/blkfront.c +4 -0
@@ -317,19 +317,21 @@ req->sector_number = aiocbp->aio_offset / dev->info.sector_size;for (j =0 ; j < n; j++) { + req->seg[j].first_sect =0 ; + req->seg[j].last_sect =PAGE_SIZE / dev->info.sector_size -1 ; + } + req->seg[0].first_sect = ((uintptr_t)aiocbp->aio_buf &
~PAGE_MASK) / dev->info.sector_size;
+ req->seg[n-1].last_sect = (((uintptr_t)aiocbp->aio_buf +aiocbp->aio_nbytes - 1) & ~PAGE_MASK) / dev->info.sector_size;
+ for (j = 0; j < n; j++) {
uintptr_t data = start + j * PAGE_SIZE;
if (!write) {
/* Trigger CoW if needed */
- *(char*)data = 0;
+ *(char*)(data + (req->seg[j].first_sect << 9)) = 0;
barrier();
}
aiocbp->gref[j] = req->seg[j].gref =
gnttab_grant_access(dev->dom, virtual_to_mfn(data), write);
- req->seg[j].first_sect = 0;
- req->seg[j].last_sect = PAGE_SIZE / dev->info.sector_size - 1;
}- req->seg[0].first_sect = ((uintptr_t)aiocbp->aio_buf &
~PAGE_MASK) / dev->info.sector_size;
- req->seg[n-1].last_sect = (((uintptr_t)aiocbp->aio_buf +aiocbp->aio_nbytes - 1) & ~PAGE_MASK) / dev->info.sector_size;
dev->ring.req_prod_pvt = i + 1;
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Quoted 3 lines · Boris Derzhavets
This one and
vl.cpatch have been applied.
stubdom has been rebuilt and reinstalled.
No improvement.
Are you sure you correctly build the new code and running fs-backend? I am only asking this because it is really easy to have this kind of problems with stubdoms.
If your build and test environments are correct, are there any errors in the logfiles?
After patching i did:-
# cd /usr/src/xen-unstable.hg/stubdom # make # make install
I haven't rebuilt tools.
Started fs-backend in separate session as usual and tried to start stub domain
Boris
--- On Tue, 4/14/09, Stefano Stabellini <[email protected]> wrote:
From: Stefano Stabellini <[email protected]>
Subject: Re: [Xen-devel] [PATCH] fix stubdom memory corruption
To: "[email protected]" <[email protected]>
Cc: "xen-devel" <[email protected]>
Date: Tuesday, April 14, 2009, 8:55 AM
Quoted 3 lines · Boris Derzhavets
This one and
vl.cpatch have been applied.
stubdom has been rebuilt and reinstalled.
No improvement.
Are you sure you correctly build the new code and running fs-backend? I am only asking this because it is really easy to have this kind of problems with stubdoms.
If your build and test environments are correct, are there any errors in the logfiles?
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
root@ServerXen331:/etc/xen# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 7019 2 r----- 702.0 RHELhvm 7 1024 1 ------ 0.0
************************
Three logs generated
************************
root@ServerXen331:/var/log/xen# cat qemu-dm-RHELhvm-dm.log domid: 9 Warning: vlan 0 is not connected to host network Watching /local/domain/0/device-model/9/logdirty/next-active Watching /local/domain/0/device-model/9/command char device redirected to /dev/pts/3 /usr/src/xen-unstable.hg/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap pipes xs_read(): vncpasswd get error. /vm/3f8762a5-b806-dca4-6442-0dfeb7da0917/vncpasswd. medium change watch on `hdc' (index: 0): /dev/loop0 root@ServerXen331:/var/log/xen# cat qemu-dm-RHELhvm-dm.log.1 domid: 8 Warning: vlan 0 is not connected to host network Watching /local/domain/0/device-model/8/logdirty/next-active Watching /local/domain/0/device-model/8/command char device redirected to /dev/pts/3 /usr/src/xen-unstable.hg/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap pipes xs_read(): vncpasswd get error. /vm/3f8762a5-b806-dca4-6442-0dfeb7da0917/vncpasswd. medium change watch on `hdc' (index: 0): /dev/loop0 root@ServerXen331:/var/log/xen# cat qemu-dm-RHELhvm.log Using config file "/etc/xen/RHELhvm-dm". Started domain RHELhvm-dm (id=8) Using config file "/etc/xen/RHELhvm-dm". Unexpected error: <type 'exceptions.OSError'>
Please report to [email protected]
Traceback (most recent call last):
File "/usr/sbin/xm", line 7, in <module>
main.main(sys.argv)
File "usr/lib/python2.5/site-packages/xen/xm/main.py", line 2980, in main _, rc = _run_cmd(cmd, cmd_name, args)
File "usr/lib/python2.5/site-packages/xen/xm/main.py", line 3004, in _run_cmd return True, cmd(args)
File "<string>", line 1, in <lambda>
File "usr/lib/python2.5/site-packages/xen/xm/main.py", line 1360, in xm_importcommand cmd.main([command] + args)
File "usr/lib/python2.5/site-packages/xen/xm/create.py", line 1370, in main do_console(sxp.child_value(config, 'name', -1))
File "usr/lib/python2.5/site-packages/xen/xm/create.py", line 1397, in do_console (p, rv) = os.waitpid(cpid, os.WNOHANG)
OSError: [Errno 10] No child processes
Boris
--- On Tue, 4/14/09, Boris Derzhavets <[email protected]> wrote:
From: Boris Derzhavets <[email protected]>
Subject: Re: [Xen-devel] [PATCH] fix stubdom memory corruption
To: "Stefano Stabellini" <[email protected]>
Cc: "xen-devel" <[email protected]>
Date: Tuesday, April 14, 2009, 9:01 AM
After patching i did:-
# cd /usr/src/xen-unstable.hg/stubdom # make # make install
I haven't rebuilt tools.
Started fs-backend in separate session as usual and tried to start stub domain
Boris
--- On Tue, 4/14/09, Stefano Stabellini <[email protected]> wrote:
From: Stefano Stabellini <[email protected]>
Subject: Re: [Xen-devel] [PATCH] fix stubdom memory corruption
To: "[email protected]" <[email protected]>
Cc: "xen-devel" <[email protected]>
Date: Tuesday, April 14, 2009, 8:55 AM
Quoted 3 lines · Boris Derzhavets
This one and
vl.cpatch have been applied.
stubdom has been rebuilt and reinstalled.
No improvement.
Are you sure you correctly build the new code and running fs-backend? I am only asking this because it is really easy to have this kind of problems with stubdoms.
If your build and test environments are correct, are there any errors in the logfiles?
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Xen unstable 3.4-rc2-pre had been installed earlier
Boris.
--- On Tue, 4/14/09, Stefano Stabellini <[email protected]> wrote:
From: Stefano Stabellini <[email protected]>
Subject: Re: [Xen-devel] [PATCH] fix stubdom memory corruption
To: "[email protected]" <[email protected]>
Cc: "xen-devel" <[email protected]>
Date: Tuesday, April 14, 2009, 8:55 AM
Quoted 3 lines · Boris Derzhavets
This one and
vl.cpatch have been applied.
stubdom has been rebuilt and reinstalled.
No improvement.
Are you sure you correctly build the new code and running fs-backend? I am only asking this because it is really easy to have this kind of problems with stubdoms.
If your build and test environments are correct, are there any errors in the logfiles?
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Quoted 2 lines · Boris Derzhavets
Xen unstable 3.4-rc2-pre had been installed earlier
I think it may be worth to do a clean build and try again, because I can confirm that with my previous patches stubdoms work for me on xen and qemu tip.
I believe you worked with 2.6.18.8-xen kernel in Dom0.
Thanks.
Boris.
--- On Tue, 4/14/09, Stefano Stabellini <[email protected]> wrote:
From: Stefano Stabellini <[email protected]>
Subject: Re: [Xen-devel] [PATCH] fix stubdom memory corruption
To: "[email protected]" <[email protected]>
Cc: "xen-devel" <[email protected]>
Date: Tuesday, April 14, 2009, 11:33 AM
Quoted 2 lines · Boris Derzhavets
Xen unstable 3.4-rc2-pre had been installed earlier
I think it may be worth to do a clean build and try again, because I can confirm that with my previous patches stubdoms work for me on xen and qemu tip.
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Quoted 2 lines · Boris Derzhavets
I believe you worked with 2.6.18.8-xen kernel in Dom0.
Yes, I'll try with a different kernel.
I had rebuilt Xen 3.4-rc2-pre from scratch.
Building Xen after fresh clone xen-unstable.hg :-
# make xen # make tools # make uninstall # make install-xen # make install-tools # cd stubdom # make # make install
then tried Suse's 2.6.27.5 and 2.6.30-rc1-tip under Xen 3.4-rc2-pre. Same error in logs:- /usr/src/xen-unstable.hg/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap pipes I guess pvops kernel is just missing blktap support
I need to check "make menuconfig" for Suse's one.
fs-backend is absolutly irresponsive with pvops kernel and seems to work as usual with 2.6.27.5
Thanks.
Boris.
--- On Wed, 4/15/09, Stefano Stabellini <[email protected]> wrote:
From: Stefano Stabellini <[email protected]>
Subject: Re: [Xen-devel] [PATCH] fix stubdom memory corruption
To: "[email protected]" <[email protected]>
Cc: "xen-devel" <[email protected]>
Date: Wednesday, April 15, 2009, 5:14 AM
Quoted 2 lines · Boris Derzhavets
I believe you worked with 2.6.18.8-xen kernel in Dom0.
Yes, I'll try with a different kernel.
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Quoted 13 lines · Boris Derzhavets
I had rebuilt Xen 3.4-rc2-pre from scratch.
Building Xen after fresh clone xen-unstable.hg :-
# make xen # make tools # make uninstall # make install-xen # make install-tools # cd stubdom # make # make install
Quoted 5 lines
then tried Suse's 2.6.27.5 and 2.6.30-rc1-tip under Xen 3.4-rc2-pre. Same error in logs:-
/usr/src/xen-unstable.hg/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap pipes I guess pvops kernel is just missing blktap support
Yep, pv_ops dom0 kernel is still missing blktap support.. Citrix blktap2 will be merged later to pv_ops dom0 kernel, iirc.
Quoted 3 lines
fs-backend is absolutly irresponsive with pvops kernel and seems to work as usual with 2.6.27.5
I've been using phy: LVM-volumes successfully with pv_ops dom0 kernel.
-- Pasi
Quoted 53 lines · Boris Derzhavets
root@ServerXen331:/etc/xen# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 7019 2 r----- 702.0 RHELhvm 7 1024 1 ------ 0.0************************
Three logs generated
************************root@ServerXen331:/var/log/xen# cat qemu-dm-RHELhvm-dm.log domid: 9 Warning: vlan 0 is not connected to host network Watching /local/domain/0/device-model/9/logdirty/next-active Watching /local/domain/0/device-model/9/command char device redirected to /dev/pts/3 /usr/src/xen-unstable.hg/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap pipes xs_read(): vncpasswd get error. /vm/3f8762a5-b806-dca4-6442-0dfeb7da0917/vncpasswd. medium change watch on `hdc' (index: 0): /dev/loop0 root@ServerXen331:/var/log/xen# cat qemu-dm-RHELhvm-dm.log.1 domid: 8 Warning: vlan 0 is not connected to host network Watching /local/domain/0/device-model/8/logdirty/next-active Watching /local/domain/0/device-model/8/command char device redirected to /dev/pts/3 /usr/src/xen-unstable.hg/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap pipes xs_read(): vncpasswd get error. /vm/3f8762a5-b806-dca4-6442-0dfeb7da0917/vncpasswd. medium change watch on `hdc' (index: 0): /dev/loop0 root@ServerXen331:/var/log/xen# cat qemu-dm-RHELhvm.log Using config file "/etc/xen/RHELhvm-dm". Started domain RHELhvm-dm (id=8) Using config file "/etc/xen/RHELhvm-dm". Unexpected error: <type 'exceptions.OSError'>Please report to [email protected]
Traceback (most recent call last):
File "/usr/sbin/xm", line 7, in <module>
main.main(sys.argv)
File "usr/lib/python2.5/site-packages/xen/xm/main.py", line 2980, in main_, rc = _run_cmd(cmd, cmd_name, args)File "
usr/lib/python2.5/site-packages/xen/xm/main.py", line 3004, in_run_cmd
return True, cmd(args)
File "<string>", line 1, in <lambda>
File "usr/lib/python2.5/site-packages/xen/xm/main.py", line 1360, inxm_importcommand
cmd.main([command] + args)
File "usr/lib/python2.5/site-packages/xen/xm/create.py", line 1370, in
main
Quoted 5 lines
do_console(sxp.child_value(config, 'name', -1))File "
usr/lib/python2.5/site-packages/xen/xm/create.py", line 1397, indo_console
(p, rv) =os.waitpid(cpid, os.WNOHANG)
OSError: [Errno 10] No child processes
If this is the only error you get, it seems to be unrelated to stubdoms. Could you please try to start a stubdom without autoconnect (no -c)?
Quoted 1 line
if this is the only error you get, it seems to be unrelated to stubdoms.
Yes
Quoted 1 line
Could you please try to start a stubdom without autoconnect (no -c)?
Sorry , what means "-c" ?
After fs-backend start i run in different terminal session :-
xm create RHELhvm-stubdom
vncviewer localhost:0
Maybe i follow outdated instructions.
Files RHELhvm-stubdom, RHELhvm-dm are located under /etc/xen
[root@ServerXen xen]# cat RHELhvm-stubdom
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory =1024
name = "RHELhvm"
vif = [ 'type=ioemu, bridge=eth0' ]device_model = "/usr/lib/xen/bin/stubdom-dm"
boot="d"
sdl=0
opengl=0
vnc=0
stdvga=0
[root@ServerXen xen]# cat RHELhvm-dm
kernel = "/usr/lib/xen/boot/ioemu-stubdom.gz"
vif = [ ' ', 'bridge=eth0']
# vfb = [ 'type=sdl' ] vfb = [ 'type=vnc' ] disk = ['phy:/dev/loop0,hdc:cdrom,r','phy:/dev/sdb8,hda,w']
Boris.
--- On Wed, 4/15/09, Stefano Stabellini <[email protected]> wrote:
From: Stefano Stabellini <[email protected]>
Subject: Re: [Xen-devel] [PATCH] fix stubdom memory corruption
To: "[email protected]" <[email protected]>
Cc: "xen-devel" <[email protected]>
Date: Wednesday, April 15, 2009, 9:37 AM
Quoted 4 lines · Boris Derzhavets
root@ServerXen331:/etc/xen# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 7019 2 r-----
702.0
Quoted 1 line
RHELhvm 7 1024 1 ------
0.0
Quoted 33 lines
************************
Three logs generated
************************root@ServerXen331:/var/log/xen# cat qemu-dm-RHELhvm-dm.log domid: 9 Warning: vlan 0 is not connected to host network Watching /local/domain/0/device-model/9/logdirty/next-active Watching /local/domain/0/device-model/9/command char device redirected to /dev/pts/3 /usr/src/xen-unstable.hg/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap pipes xs_read(): vncpasswd get error. /vm/3f8762a5-b806-dca4-6442-0dfeb7da0917/vncpasswd. medium change watch on `hdc' (index: 0): /dev/loop0 root@ServerXen331:/var/log/xen# cat qemu-dm-RHELhvm-dm.log.1 domid: 8 Warning: vlan 0 is not connected to host network Watching /local/domain/0/device-model/8/logdirty/next-active Watching /local/domain/0/device-model/8/command char device redirected to /dev/pts/3 /usr/src/xen-unstable.hg/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap pipes xs_read(): vncpasswd get error. /vm/3f8762a5-b806-dca4-6442-0dfeb7da0917/vncpasswd. medium change watch on `hdc' (index: 0): /dev/loop0 root@ServerXen331:/var/log/xen# cat qemu-dm-RHELhvm.log Using config file "/etc/xen/RHELhvm-dm". Started domain RHELhvm-dm (id=8) Using config file "/etc/xen/RHELhvm-dm". Unexpected error: <type 'exceptions.OSError'>
Quoted 5 lines
Please report to [email protected] Traceback (most recent call last):
File "/usr/sbin/xm", line 7, in <module>
main.main(sys.argv)
File "usr/lib/python2.5/site-packages/xen/xm/main.py", line
2980, in main
Quoted 2 lines
_, rc = _run_cmd(cmd, cmd_name, args)File "
usr/lib/python2.5/site-packages/xen/xm/main.py", line
3004, in
Quoted 4 lines
_run_cmd
return True, cmd(args)
File "<string>", line 1, in <lambda>
File "usr/lib/python2.5/site-packages/xen/xm/main.py", line
1360, in
Quoted 3 lines
xm_importcommand
cmd.main([command] + args)
File "usr/lib/python2.5/site-packages/xen/xm/create.py", line
1370, in
Quoted 3 lines
main
do_console(sxp.child_value(config, 'name', -1))File "
usr/lib/python2.5/site-packages/xen/xm/create.py", line
1397, in
Quoted 3 lines
do_console
(p, rv) =os.waitpid(cpid, os.WNOHANG)
OSError: [Errno 10] No child processes
If this is the only error you get, it seems to be unrelated to stubdoms. Could you please try to start a stubdom without autoconnect (no -c)?
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel