Skip to contentSkip to conversation

VM-Tool: C-based Xen management tools

PATCH In review Anthony Liguori 5 participants 13 messages Feb 11, 2005 – Feb 20, 2005

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 VM-Tool: C-based Xen management tools

We've implemented a set of basic management tools for Xen. The tools are written in C and are currently capable of creating and shutting down domains, connecting to a domain's console, enumerating domains, and attaching VBDs and VIFs to a domain.

The tools are still very rough. This version is pretty much a proof-of-concept. It should not be used in a production environment. In fact, you shouldn't use them with any domains that are important to you. While I've never seen any data corruption caused by the tools there's just not been enough testing yet.

These tools are a product of about a weeks work. There are some missing features including save/restore, migration, and usb/pci virtualation. The reason is that we wanted to get the communities feedback before we went too far.

The tools are not meant to replace Xend but rather provide an alternative toolchain for those that need a reduced management foot print.

The plan is to continue cleaning up the code and submit again to the list as a patch to xen-unstable. We've talked to Mike Wray and we are both eager to make sure that these tools and Xend cooperate as much as possible in the long term.

You can get a copy of the tools at the following URL. xen-unstable is required to use them (they rely on xcs). Feedback is greatly appreciated :-) Nothing is really set in stone.

http://www.cs.utexas.edu/users/aliguori/vm-tools-0.0.1.tar.gz

Thanks,
Anthony Liguori

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

#1 lore raw
Quoted 1 line · Anthony Liguori

We've implemented a set of basic management tools for Xen. The tools are written in C and are currently capable of creating and shutting down domains, connecting to a domain's console, enumerating domains, and attaching VBDs and VIFs to a domain.

Hey, Anthony, thanks for getting this out!. I've started to take a look at it - won't be able to test it for a bit since I'm running on -testing for a while - I'll have feedback next week.

Quoted 1 line

The tools are still very rough. This version is pretty much a proof-of-concept. It should not be used in a production environment. In fact, you shouldn't use them with any domains that are important to you. While I've never seen any data corruption caused by the tools there's just not been enough testing yet.

Can you tell us on what you tested, and which configurations, so far?

Quoted 1 line

These tools are a product of about a weeks work. There are some missing features including save/restore, migration, and usb/pci virtualation. The reason is that we wanted to get the communities feedback before we went too far.

Are you going to be doing the USB and PCI stuff as well? Will you need any help on that?

Quoted 1 line

You can get a copy of the tools at the following URL. xen-unstable is required to use them (they rely on xcs). Feedback is greatly appreciated :-) Nothing is really set in stone.

So there will be no effort to backport, correct?

thanks,
Nivedita

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

#2 lore raw
Quoted 3 lines · Nivedita Singhvi

The tools are still very rough. This version is pretty much a proof-of-concept. It should not be used in a production environment. In fact, you shouldn't use them with any domains that are important to you. While I've never seen any data corruption caused by the tools there's just not been enough testing yet.

Can you tell us on what you tested, and which configurations, so far?

Sure, I've tested on Fedora Core 3 and on Gentoo both running various copies of xen-unstable (ranging from a few weeks ago to a day ago). There are a number of others (Mike, Woody, Dan) that have tested on other platforms.

By tested, I mean was able to create and destroy domains with various arrangements of block devices and network devices (0, 1, 2, etc, etc.). I've tested using a custom busybox based ramdisk and ttylinux as DomU. Other DomU configuration should also work (although haven't been tested).

Quoted 2 lines

Are you going to be doing the USB and PCI stuff as well? Will you need any help on that?

Yes, the goal was to minimize the amount of features to the least amount required before release. We're very interested in hearing feedback on what the tools should look like.

USB and PCI support isn't that hard. Doing in a way that people feel is intuitive is another matter. Feed-back on what sort of interfaces these tools should have is extremely useful.

Quoted 1 line

So there will be no effort to backport, correct?

That depends on demand. The tools could be modified to open the controls channels directly instead of going through xcs, but I'd rather encourage an xcs based world where Xend and vm-tools can live together. Of course, if someone was willing to backport the old Xend to xcs... :-)

Thanks,
Anthony Liguori

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

#3 lore raw

Nice work!

Quoted 10 lines
Quoted 2 lines

Are you going to be doing the USB and PCI stuff as well? Will you need any help on that?

Yes, the goal was to minimize the amount of features to the least amount required before release. We're very interested in hearing feedback on what the tools should look like.

USB and PCI support isn't that hard. Doing in a way that people feel is intuitive is another matter. Feed-back on what sort of interfaces these tools should have is extremely useful.

PCI should be fairly trivial at the moment since it just requires a hypercall. USB is very similar to Vifs and Vbds from the control protocol PoV.

As far as making the interface intuitive... I think it's reasonably sane to specify this stuff the way the current tools do, although arguably it would be useful to have some sort of abstraction so that users don't have to think about PCI config space / USB port IDs.

Cheers,
Mark

Quoted 11 lines
Quoted 1 line

So there will be no effort to backport, correct?

That depends on demand. The tools could be modified to open the controls channels directly instead of going through xcs, but I'd rather encourage an xcs based world where Xend and vm-tools can live together. Of course, if someone was willing to backport the old Xend to xcs... :-)

Thanks,
Anthony Liguori

Quoted 9 lines

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xen-devel

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

#4 lore raw
Quoted 3 lines · Mark Williamson

Nice work!

Thanks :-)

Quoted 3 lines

As far as making the interface intuitive... I think it's reasonably sane to specify this stuff the way the current tools do, although arguably it would be useful to have some sort of abstraction so that users don't have to think about PCI config space / USB port IDs.

Yes, PCI wouldn't be too hard to abstract. Abstracting USB worries me a bit though. On the one hand, you want to abstract at the device level. However, not all USB devices make it easy to be uniquely identified (too much optional information). This is a problem the devfs/udev guys have faced.

It's even hairer with virtualization though. Say you had two harddrives that were USB and not uniquely identifiable. You want to assign each harddrive to a separate VM. If you unplug them and swap ports, and you're using the old configurations, you really want that each VM to see the harddrive they previously saw. Swapping them could lead to confusion and even worse, security problems.

Not sure the best solution here. One of the reasons I've held off on USB.

Regards,

Quoted 17 lines

Cheers,
Mark

Quoted 12 lines
Quoted 3 lines

So there will be no effort to backport, correct?

That depends on demand. The tools could be modified to open the controls channels directly instead of going through xcs, but I'd rather encourage an xcs based world where Xend and vm-tools can live together. Of course, if someone was willing to backport the old Xend to xcs... :-)

Thanks,
Anthony Liguori

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

#5 lore raw
Quoted 2 lines

Yes, PCI wouldn't be too hard to abstract. Abstracting USB worries me a bit though. On the one hand, you want to abstract at the device level.

That would be a neat feature. It'd require changes to the backend device driver to be a bit smarter about grabbing devices, as well as frontend changes to communicate device identities the backend...

Quoted 6 lines

It's even hairer with virtualization though. Say you had two harddrives that were USB and not uniquely identifiable. You want to assign each harddrive to a separate VM. If you unplug them and swap ports, and you're using the old configurations, you really want that each VM to see the harddrive they previously saw. Swapping them could lead to confusion and even worse, security problems.

The current implementation sidesteps all these by specifying port IDs. e.g. port 1 on my root hub always belongs to VM1 (when running), port 2 on hub 3 always belongs to VM2 (when running), etc.

Quoted 1 line

Not sure the best solution here. One of the reasons I've held off on USB.

For comparison, do you know what abstraction other systems do? Do they assign ports or associate devices with VMs?

The latter is definitely an attractive option... If we added support for both ways of doing things, paranoid people (or people with weird devices!) could use the port specifiers, whilst others used device IDs.

What do you think?

Cheers,
Mark

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

#6 lore raw
Quoted 3 lines · Mark Williamson

The current implementation sidesteps all these by specifying port IDs. e.g. port 1 on my root hub always belongs to VM1 (when running), port 2 on hub 3 always belongs to VM2 (when running), etc.

Yeah, I think we'll always have to support this as a fallback option.

Quoted 7 lines
Quoted 3 lines

Not sure the best solution here. One of the reasons I've held off on USB.

For comparison, do you know what abstraction other systems do? Do they assign ports or associate devices with VMs?

Not sure. The only thing I know that does USB virtualization is VMware and I've not played with that feature (I think it's a relatively new feature).

udev/namedev have to deal with this to ensure that devices have a unique (and consistent) /dev entry. It currently uses a tiered approach to determine exactly what the device is. It's not perfect but it works reasonably well. The best link I could find explaining it with a quick google is:

http://www.gentoo.org/doc/en/udev-guide.xml

A GUI would be really handy for PCI/USB because it would provide an easy way to pick devices by name (instead of specifying an series of identifiers/bus/etc in the config).

Quoted 3 lines

The latter is definitely an attractive option... If we added support for both ways of doing things, paranoid people (or people with weird devices!) could use the port specifiers, whilst others used device IDs.

I agree completely. I'm currently adding proper documentation to vm-tools but after that's done I'll add USB virtualization and try to implement a mechanism to select by device ID and push out another version to the list.

Regards,

Signature · 10 lines
Anthony Liguori
[email protected]



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
#7 lore raw
Quoted 6 lines
Quoted 2 lines

For comparison, do you know what abstraction other systems do? Do they assign ports or associate devices with VMs?

Not sure. The only thing I know that does USB virtualization is VMware and I've not played with that feature (I think it's a relatively new feature).

I think they've been able to do some kind of USB passthrough for a while (not sure how long) but the support for isochronous is only available in their current beta.

Quoted 7 lines

udev/namedev have to deal with this to ensure that devices have a unique (and consistent) /dev entry. It currently uses a tiered approach to determine exactly what the device is. It's not perfect but it works reasonably well. The best link I could find explaining it with a quick google is:

http://www.gentoo.org/doc/en/udev-guide.xml

I'll check it out, thanks.

Thinking about it, we could keep the device identification logic in userspace, which would be cleaner. The tools could then just tell the backend to claim the device on a given port. There'd be a few driver tweaks required to make it all work together but this may well be a cleaner approach.

Quoted 3 lines

A GUI would be really handy for PCI/USB because it would provide an easy way to pick devices by name (instead of specifying an series of identifiers/bus/etc in the config).

Yes, I thought so to. A general admin GUI is on our todo list locally but AFAIK nobody is working on it right now. Every so often I poke the XenSV web interface to make sure it's still working.

Quoted 4 lines

I agree completely. I'm currently adding proper documentation to vm-tools but after that's done I'll add USB virtualization and try to implement a mechanism to select by device ID and push out another version to the list.

Cool!

Cheers,
Mark

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

#8 lore raw
Quoted 4 lines · Mark Williamson

Thinking about it, we could keep the device identification logic in userspace, which would be cleaner. The tools could then just tell the backend to claim the device on a given port. There'd be a few driver tweaks required to make it all work together but this may well be a cleaner approach.

Yes, this is how I also envisioned it would work. I think the approach has many advantages in general.

Quoted 3 lines

Yes, I thought so to. A general admin GUI is on our todo list locally but AFAIK nobody is working on it right now. Every so often I poke the XenSV web interface to make sure it's still working.

I'm trying to build a library around vm-tools so that doing something like writing a GUI would be quite trivial. At some point, I may try to throw together a GUI but I figured that one of the distros would probably beat me to it :-)

Regards,

Signature · 13 lines
Anthony Liguori
Linux Technology Center (LTC) - IBM Austin
E-mail: [email protected]
Phone: (512) 838-1208




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
#9 lore raw
In reply to Anthony Liguori
Quoted 1 line · Anthony Liguori

We've implemented a set of basic management tools for Xen. The tools are written in C and are currently capable of creating and shutting down domains, connecting to a domain's console, enumerating domains, and attaching VBDs and VIFs to a domain.

hi,
these tools work fine for me as long as I am trying to use the network. Has anyone had any luck using these along with the bridging scripts, or with alternative scripts that do the same thing?

Jacob

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

#10 lore raw
Quoted 2 lines · Jacob Gorm Hansen

hi,
these tools work fine for me as long as I am trying to use the network. Has anyone had any luck using these along with the bridging scripts, or with alternative scripts that do the same thing?

I've manually gotten networking going invoking the Xend network script by hands. I will automate this process eventually. There's just a number of other things I'm working on right now.

Regards,
Anthony Liguori

Quoted 13 lines

Jacob

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xen-devel

Signature · 10 lines
Anthony Liguori
[email protected]



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
#11 lore raw
Quoted 8 lines · Anthony Liguori
Quoted 2 lines · Jacob Gorm Hansen

hi,
these tools work fine for me as long as I am trying to use the network. Has anyone had any luck using these along with the bridging scripts, or with alternative scripts that do the same thing?

I've manually gotten networking going invoking the Xend network script by hands. I will automate this process eventually. There's just a number of other things I'm working on right now.

So that would be:

- /etc/xen/scripts/network start
- nics=1 in the config-file
- vm create config-file
- /etc/xen/scripts/vif-bridge up vif=vif1.0 domain=Domain-1 mac=... bridge=xen-br0

Or am I missing something? The above does not seem to work for me.

thanks,
Jacob

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

#12 lore raw
Quoted 9 lines · Jacob Gorm Hansen

So that would be:

- /etc/xen/scripts/network start
- nics=1 in the config-file
- vm create config-file
- /etc/xen/scripts/vif-bridge up vif=vif1.0 domain=Domain-1 mac=... bridge=xen-br0

Or am I missing something? The above does not seem to work for me.

I think I did something else. I think the network script takes more parameters. The way I figured out what to do was to annotate all of xend scripts and watch to see what it did.

I'll post a HOWTO (or perhaps take a stab at automating it) on Monday using the new 0.0.2 release. My local system doesn't have enough memory to create anything other than a simple busybox domain.

Regards,

Quoted 3 lines

thanks,
Jacob

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click