Skip to contentSkip to conversation

pypxeboot bootloader

PATCH In review Stephen Childs 6 participants 12 messages Feb 5, 2007 – May 7, 2007

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 pypxeboot bootloader

As promised on Friday here is the patch for the pypxeboot bootloader. It would be great if someone could try it out and give me some feedback.

Stephen

Signature · 4 lines
Dr. Stephen Childs,
Research Fellow, EGEE Project,    phone:                    +353-1-8961797
Computer Architecture Group,      email:        Stephen.Childs @ cs.tcd.ie
Trinity College Dublin, Ireland   web: http://www.cs.tcd.ie/Stephen.Childs

# HG changeset patch
# User [email protected]
# Date 1170673641 0
# Node ID 7f1a38c5c08659ae123e5f94696cbca19e4e10fb # Parent 01ec7dba9ff805a5c74a0318997b747d3e3e3327 Added pypxeboot bootloader for simulating PXE boot for DomUs.

tools/pypxeboot/README new file +44 -0
diff -r 01ec7dba9ff8 -r 7f1a38c5c086 tools/pypxeboot/README
@@ -0,0 +1,44 @@
+pypxeboot is a bootloader for xen that simulates PXE behaviour. It runs on 
+Domain 0 as part of the domain creation process and downloads boot information
+from a previously-configured PXELinux server using TFTP.
+
+pypxeboot requires the following external programs:
+
+1) A patched version of udhcp 0.9.8 (http://udhcp.busybox.net/) that supports 
+a user-specified MAC address. The patch is udhcp_usermac.patch, available
+in this distribution. There is also a script needed to output information
+received from the DHCP server. This is called outputpy.udhcp.sh and
+should be installed at /usr/share/udhcpc/
+
+2) The tftp client program (http://www.kernel.org/pub/software/network/tftp/). 
+RPMs are also available from the DAG repository at
+(http://dag.wieers.com/rpm/packages/tftp/)
+
+To use pypxeboot, add the following lines to your Xen domain configuration
+file:
+
+bootloader="/usr/bin/pypxeboot"
+bootargs=vif[0]
+
+If the pxelinux.cfg entry is set to localboot you should see output like this:
+
+[root@tg23 pypxeboot]# xm create  cagnode50-slc308
+Using config file "/etc/xen/cagnode50-slc308".
+pypxeboot: requesting info for MAC address AA:00:86:e2:35:72
+pypxeboot: getting cfg for IP 134.226.53.114 (86E23572) from server 192.168.12.1
+pypxeboot: dropping to pygrub for local boot
+Going to boot Scientific Linux CERN Xen DomU-xenU (2.4.21-47.0.1.EL.cernxenU)
+  kernel: /vmlinuz-2.4.21-47.0.1.EL.cernxenU
+  initrd: /initrd-2.4.21-47.0.1.EL.cernxenU.img
+
+and something like this if the pxelinux.cfg entry specifies a network boot:
+
+[root@tg23 pypxeboot]# xm create  cagnode50-slc308
+Using config file "/etc/xen/cagnode50-slc308".
+pypxeboot: requesting info for MAC address AA:00:86:e2:35:72
+pypxeboot: getting cfg for IP 134.226.53.114 (86E23572) from server 192.168.12.1
+pypxeboot: downloading initrd using cmd: tftp -c get 192.168.12.1:slc308_i386_xen/initrd.img
+pypxeboot: downloading kernel using cmd: tftp -c get 192.168.12.1:slc308_i386_xen/vmlinuz
+Started domain cagnode50
+
+The kernel and initrd on the tftp server need to be XenLinux images.
tools/pypxeboot/outputpy.udhcp.sh new file +42 -0

42 lines not on this page · raw message

tools/pypxeboot/pypxeboot new file +202 -0

202 lines not on this page · raw message

tools/pypxeboot/udhcp_usermac.patch new file +107 -0

107 lines not on this page · raw message

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw

Hi,

Quoted 2 lines · Stephen Childs

As promised on Friday here is the patch for the pypxeboot bootloader. It would be great if someone could try it out and give me some feedback.

First impressions: this looks really useful but has a few rough edges that need to be knocked off. (I haven't had a chance to try this on a real machine yet.)

tools/pypxeboot/pypxeboot line 106

Quoted 8 lines
tools/pypxeboot/pypxeboot new file +8 -0
+for line in udhcplines:
+    s = line.strip()
+    f = s.split()
+
+    if s[0]=='{' and s[-1]=='}':
+        dhcpinfo=eval(s, {"__builtins__" : {}})
+        for k in dhcpinfo:
+            dhcpinfo[k]=dhcpinfo[k].strip()

Executing the results of programs scares me. Could you just parse it?

tools/pypxeboot/pypxeboot line 113

Quoted 3 lines
tools/pypxeboot/pypxeboot new file +3 -0
+ipaddr=dhcpinfo['ip']
+ipaddrlist=ipaddr.split('.')
+hexip=commands.getstatusoutput("/usr/bin/gethostip -x "+ipaddr)[1]

This requires syslinux to be installed on the Xen host, and isn't hard to do by hand: "%2.2x%2.2x%2.2x%2.2x" % tuple(map (int, ipaddrlist)) (I'm sure more heavyweight python hackers can correct my style.)

tools/pypxeboot/pypxeboot line 122

Quoted 6 lines
tools/pypxeboot/pypxeboot new file +6 -0
+tmpdir="/var/lib/xen/"
+
+os.chdir(tmpdir)
+commandstr="tftp -c get "+servaddr+":pxelinux.cfg/"+hexip
+#print "running command "+commandstr
+getpxeres=commands.getstatusoutput(commandstr)

You'll need to be more careful with temporary file names -- here at least the filename is unique to the guest (we hope!) but for the kernel etc., two boots could clash. Also, you don't seem to remove the downloaded config files.

Cheers,

Tim.

Signature · 2 lines
Tim Deegan <[email protected]>, XenSource UK Limited
Registered office c/o EC2Y 5EB, UK; company number 05334508
#2 lore raw
Quoted 21 lines · Tim Deegan

Hi,

Quoted 2 lines · Stephen Childs

As promised on Friday here is the patch for the pypxeboot bootloader. It would be great if someone could try it out and give me some feedback.

First impressions: this looks really useful but has a few rough edges that need to be knocked off. (I haven't had a chance to try this on a real machine yet.)

tools/pypxeboot/pypxeboot line 106

Quoted 8 lines
tools/pypxeboot/pypxeboot new file +8 -0
+for line in udhcplines:
+    s = line.strip()
+    f = s.split()
+
+    if s[0]=='{' and s[-1]=='}':
+        dhcpinfo=eval(s, {"__builtins__" : {}})
+        for k in dhcpinfo:
+            dhcpinfo[k]=dhcpinfo[k].strip()

Executing the results of programs scares me. Could you just parse it?

Quoted 7 lines

tools/pypxeboot/pypxeboot line 113

Quoted 3 lines
tools/pypxeboot/pypxeboot new file +3 -0
+ipaddr=dhcpinfo['ip']
+ipaddrlist=ipaddr.split('.')
+hexip=commands.getstatusoutput("/usr/bin/gethostip -x "+ipaddr)[1]

This requires syslinux to be installed on the Xen host, and isn't hard to do by hand: "%2.2x%2.2x%2.2x%2.2x" % tuple(map (int, ipaddrlist)) (I'm sure more heavyweight python hackers can correct my style.)

(4 * "%2.2x") % tuple(map (int, ipaddrlist))

or

"".join(["%2.2x" % int(i) for i in ipaddrlist])

Both are pleasingly arcane...

Ewan.

#3 lore raw
In reply to Tim Deegan
Quoted 3 lines · Tim Deegan

First impressions: this looks really useful but has a few rough edges that need to be knocked off. (I haven't had a chance to try this on a real machine yet.)

I absolutely agree, this was the first working version and I was aware there were nasties in it.

Quoted 10 lines

tools/pypxeboot/pypxeboot line 106

Quoted 8 lines
tools/pypxeboot/pypxeboot new file +8 -0
+for line in udhcplines:
+    s = line.strip()
+    f = s.split()
+
+    if s[0]=='{' and s[-1]=='}':
+        dhcpinfo=eval(s, {"__builtins__" : {}})
+        for k in dhcpinfo:
+            dhcpinfo[k]=dhcpinfo[k].strip()

Executing the results of programs scares me. Could you just parse it?

Yes quite. Sorry that was cut-and-paste code so I didn't think about it enough. Will rewrite.

Quoted 5 lines

tools/pypxeboot/pypxeboot line 113

Quoted 3 lines
tools/pypxeboot/pypxeboot new file +3 -0
+ipaddr=dhcpinfo['ip']
+ipaddrlist=ipaddr.split('.')
+hexip=commands.getstatusoutput("/usr/bin/gethostip -x "+ipaddr)[1]

This requires syslinux to be installed on the Xen host, and isn't hard to do by hand: "%2.2x%2.2x%2.2x%2.2x" % tuple(map (int, ipaddrlist)) (I'm sure more heavyweight python hackers can correct my style.)

Yes of course I'll change that.

Quoted 11 lines

tools/pypxeboot/pypxeboot line 122

Quoted 6 lines
tools/pypxeboot/pypxeboot new file +6 -0
+tmpdir="/var/lib/xen/"
+
+os.chdir(tmpdir)
+commandstr="tftp -c get "+servaddr+":pxelinux.cfg/"+hexip
+#print "running command "+commandstr
+getpxeres=commands.getstatusoutput(commandstr)

You'll need to be more careful with temporary file names -- here at least the filename is unique to the guest (we hope!) but for the kernel etc., two boots could clash. Also, you don't seem to remove the downloaded config files.

Yes that all needs tidying up. Unfortunately the version of tftp I was using didn't actually support "get <remote-file> <local-file>" as advertised which I was initially going to use. I'll have to download to the filename supplied by the server then move to a tmpfile.

On the subject of removing files yes of course I'll remove the config files. Comparing with pygrub, that seems to leave kernels and initrds clogging up /var/lib/xen, and I suppose I'll end up doing the same. I suppose you could set up tmpwatch to clean it up, but it still seems a bit messy.

Expect a new patch soon ...

Stephen

Signature · 4 lines
Dr. Stephen Childs,
Research Fellow, EGEE Project,    phone:                    +353-1-8961797
Computer Architecture Group,      email:        Stephen.Childs @ cs.tcd.ie
Trinity College Dublin, Ireland   web: http://www.cs.tcd.ie/Stephen.Childs
#4 lore raw
Quoted 3 lines

Yes that all needs tidying up. Unfortunately the version of tftp I was using didn't actually support "get <remote-file> <local-file>" as advertised which I was initially going to use. I'll have to download

to

Quoted 1 line

the filename supplied by the server then move to a tmpfile.

tftp-hpa as supplied with FC3 seems to work: "tftp tftp-serv -c get foo /tmp/bar"

Incidentally, what do you do if the guest vif is not going to be on the same bridge that dom0 eth0 is on? In practice this likely matters more for the dhcp request than the tftp transfer.

Ian

#11 lore raw
In reply to Stephen Childs

Hi,

Quoted 3 lines · Stephen Childs

Yes that all needs tidying up. Unfortunately the version of tftp I was using didn't actually support "get <remote-file> <local-file>" as advertised which I was initially going to use.

:(

Quoted 2 lines

I'll have to download to
the filename supplied by the server then move to a tmpfile.

How about a wrapper function that mkdirs a new randomly named directory, does the tftp in there, and then moves the file to the destination path and rmdirs the temp dir?

Quoted 5 lines

On the subject of removing files yes of course I'll remove the config files. Comparing with pygrub, that seems to leave kernels and initrds clogging up /var/lib/xen, and I suppose I'll end up doing the same. I suppose you could set up tmpwatch to clean it up, but it still seems a bit messy.

The kernel and initrd files are actually deleted by Xend after it has used them to boot the domain. Somewhere way out on my todo/wish-list is plumbing the kernel/initrd through pipes and buffers, or open fds on unlinked files or some such...

Quoted 1 line

Expect a new patch soon ...

Great; thanks for that.

Cheers,

Tim.

Signature · 2 lines
Tim Deegan <[email protected]>, XenSource UK Limited
Registered office c/o EC2Y 5EB, UK; company number 05334508
#5 lore raw
In reply to Ian Pratt
Quoted 2 lines · Ian Pratt

tftp-hpa as supplied with FC3 seems to work: "tftp tftp-serv -c get foo /tmp/bar"

Yes and tftp version 0.43 from http://www.kernel.org/pub/software/network/tftp/ works too. I was using a much older one before so everything should be OK as long as people use a recent version. I've put it in the README.

I've attached a fresh patch that addresses this and the other issues you raised, Tim.

Quoted 3 lines

Incidentally, what do you do if the guest vif is not going to be on the same bridge that dom0 eth0 is on? In practice this likely matters more for the dhcp request than the tftp transfer.

Could you explain what the problem would be? Do you mean that the DHCP server might not be set up to serve requests from the dom0's subnet? To be honest I hadn't thought that much about more complex setups -- as it is there are embedded assumptions about the way pxelinux is set up.

Stephen

Signature · 4 lines
Dr. Stephen Childs,
Research Fellow, EGEE Project,    phone:                    +353-1-8961797
Computer Architecture Group,      email:        Stephen.Childs @ cs.tcd.ie
Trinity College Dublin, Ireland   web: http://www.cs.tcd.ie/Stephen.Childs

# HG changeset patch
# User [email protected]
# Date 1170952399 0
# Node ID 4cf16fd716cf1dff0859afda632f6cf922c35dfa # Parent 780f097b54c5f9161f7c6cf3f86b2bb72cc43587 pypxeboot, a PXE-like bootloader for getting configuration and kernel images for Domain Us.

tools/pypxeboot/README new file +44 -0
diff -r 780f097b54c5 -r 4cf16fd716cf tools/pypxeboot/README
@@ -0,0 +1,44 @@
+pypxeboot is a bootloader for xen that simulates PXE behaviour. It runs on 
+Domain 0 as part of the domain creation process and downloads boot information
+from a previously-configured PXELinux server using TFTP.
+
+pypxeboot requires the following external programs:
+
+1) A patched version of udhcp 0.9.8 (http://udhcp.busybox.net/) that supports 
+a user-specified MAC address. The patch is udhcp_usermac.patch, available
+in this distribution. There is also a script needed to output information
+received from the DHCP server. This is called outputpy.udhcp.sh and
+should be installed at /usr/share/udhcpc/
+
+2) The tftp client program (source and RPMs available at
+http://www.kernel.org/pub/software/network/tftp/). N.B. pypxeboot
+has been tested with version 0.43 of tftp.
+
+To use pypxeboot, add the following lines to your Xen domain configuration
+file:
+
+bootloader="/usr/bin/pypxeboot"
+bootargs=vif[0]
+
+If the pxelinux.cfg entry is set to localboot you should see output like this:
+
+[root@tg23 pypxeboot]# xm create  cagnode50-slc308
+Using config file "/etc/xen/cagnode50-slc308".
+pypxeboot: requesting info for MAC address AA:00:86:e2:35:72
+pypxeboot: getting cfg for IP 134.226.53.114 (86E23572) from server 192.168.12.1
+pypxeboot: dropping to pygrub for local boot
+Going to boot Scientific Linux CERN Xen DomU-xenU (2.4.21-47.0.1.EL.cernxenU)
+  kernel: /vmlinuz-2.4.21-47.0.1.EL.cernxenU
+  initrd: /initrd-2.4.21-47.0.1.EL.cernxenU.img
+
+and something like this if the pxelinux.cfg entry specifies a network boot:
+
+[root@tg23 pypxeboot]# xm create  cagnode50-slc308
+Using config file "/etc/xen/cagnode50-slc308".
+pypxeboot: requesting info for MAC address AA:00:86:e2:35:72
+pypxeboot: getting cfg for IP 134.226.53.114 (86E23572) from server 192.168.12.1
+pypxeboot: downloading initrd using cmd: tftp -c get 192.168.12.1:slc308_i386_xen/initrd.img
+pypxeboot: downloading kernel using cmd: tftp -c get 192.168.12.1:slc308_i386_xen/vmlinuz
+Started domain cagnode50
+
+The kernel and initrd on the tftp server need to be XenLinux images.
tools/pypxeboot/outputpy.udhcp.sh new file +44 -0

44 lines not on this page · raw message

tools/pypxeboot/pypxeboot new file +209 -0

209 lines not on this page · raw message

tools/pypxeboot/udhcp_usermac.patch new file +107 -0

107 lines not on this page · raw message

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#6 lore raw
Quoted 1 line
Quoted 1 line

Incidentally, what do you do if the guest vif is not going to be on

the

Quoted 1 line
Quoted 1 line

same bridge that dom0 eth0 is on? In practice this likely matters

more

Quoted 4 lines
Quoted 1 line

for the dhcp request than the tftp transfer.

Could you explain what the problem would be? Do you mean that the DHCP server might not be set up to serve requests from the dom0's subnet?

To be

Quoted 1 line

honest I hadn't thought that much about more complex setups -- as it

is

Quoted 1 line

there are embedded assumptions about the way pxelinux is set up.

I think we should at least try and send the dhcp request out on the bridge that the guest's first VIF is going to attach to -- it might be on an entirely different VLAN or even physical network from dom0.

Possibly it will work just to set the bridge as the interface that the raw socket is bound to when sending the dhcp request.

Ian

#7 lore raw
In reply to Stephen Childs
Quoted 2 lines · Stephen Childs

I've attached a fresh patch that addresses this and the other issues you raised, Tim.

Thanks. That looks much better -- I think we should use mkstemp for the config files too, if possible.

Is udhcp still actively maintained? I wonder if we can get the mac-address patch in upstream; otherwise we might have to bring the client source into our tree.

The other option (and it may end up being necesary to address Ian's point about physical NICs) would be to steal the guest's vif for the dhcp'ing, since we know the guest won't be using it yet. I haven't looked in detail at the plumbing of that, though.

Cheers,

Tim.

Signature · 2 lines
Tim Deegan <[email protected]>, XenSource UK Limited
Registered office c/o EC2Y 5EB, UK; company number 05334508
#8 lore raw
Quoted 5 lines · Tim Deegan
Quoted 1 line · Stephen Childs

I've attached a fresh patch that addresses this and the other issues you raised, Tim.

Thanks. That looks much better -- I think we should use mkstemp for the config files too, if possible.

OK I will try and have a look at that (or you're welcome to!)

Quoted 2 lines

Is udhcp still actively maintained? I wonder if we can get the
mac-address patch in upstream; otherwise we might have to bring the client source into our tree.

I've had no response to a patch I submitted and the last release was in 2002 so maybe we will have to do that for now.

Quoted 4 lines

The other option (and it may end up being necesary to address Ian's point about physical NICs) would be to steal the guest's vif for the dhcp'ing, since we know the guest won't be using it yet. I haven't looked in detail at the plumbing of that, though.

That sounds like a better solution in the long-term. Hopefully the existing approach already provides enough functionality to be useful to people in the short-term though.

Stephen

Signature · 4 lines
Dr. Stephen Childs,
Research Fellow, EGEE Project,    phone:                    +353-1-8961797
Computer Architecture Group,      email:        Stephen.Childs @ cs.tcd.ie
Trinity College Dublin, Ireland   web: http://www.cs.tcd.ie/Stephen.Childs
#9 lore raw
In reply to Stephen Childs

First off, thanks for all the work, Stephen. This is a useful feature.

I'm trying to test it, without much success. I've followed the instructions on
https://www.cs.tcd.ie/Stephen.Childs/pypxeboot/ but when I xm create the machine, it just hangs on: Using config file "/etc/xen/testU1.cfg".

I don't even get any output if I put in frivolous print statements in pypxeboot before anything else happens.

How does xm actually invoke bootloaders? What might be happening here?

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#10 lore raw
Quoted 8 lines

First off, thanks for all the work, Stephen. This is a useful feature.

I'm trying to test it, without much success. I've followed the instructions on
https://www.cs.tcd.ie/Stephen.Childs/pypxeboot/ but when I xm create the
machine, it just hangs on: Using config file "/etc/xen/testU1.cfg".

Quoted 4 lines

I don't even get any output if I put in frivolous print statements in pypxeboot before anything else happens.

How does xm actually invoke bootloaders? What might be happening here?

Not sure why its not working, but it would be good to get this patch dusted off and made a bit more general.

The biggest problem with it is that it assumes that dom0's eth0 interface is on the same bridged network as where the guest's virtual NIC will end up. This often won't be the case.

We could fix this quite easily, using the '-i' option to udhcp to point to the backend vif.

NB: Stephen: at one point does pypxeboot run? After the vifX.Y is created and put on the right bridge? If so, this should be a trivial change.

The other problem is that we then use dom0's default IP address to do the tftp. Again, this would best be done using the guest's IP. The best way of fixing this is probably to assign the backend vif with the guest's IP returned from udhcp (i.e. actually allow udhcp to configure an address on the vifX.Y interface). We can the get tftp to bind the outgoing socket it creates to that local address when doing the transfer. After completing, we should be sure to set the vif's IP address back to 0.0.0.0.

Anyone up for making these modifications? I'd really like to see this patch in mainline Xen.

Best,
Ian