UML is now self-hosting!
UML is now able to run nested inside itself. This works as of UML 2.4.18-21, which isn't released yet, but will be soon. See the log below for the gory details, and also see http://user-mode-linux.sf.net/nesting.html for how to do it yourself.
This is a sign of UML maturity rather than any new magic functionality having been added. UML is a demanding process, so even though it uses only the Linux system call interface, it takes some maturity for a Linux kernel to host it.
The missing pieces were a couple of signal delivery bugs that were still lurking. Once these were fixed, UML booted right up.
Jeff
Here's the log of the first nested boot.
Notes:
'usermode:~#' is the UML shell prompt.
/dev/ubd/1 has been attached to a tomsrtbt image on the host ~/linux/2.4/nest/linux is the specially build nested UML.
Also note the back-to-back UML shutdowns at the very end.
usermode:~# scp [email protected]:linux/2.4/nest/linux . [email protected]'s password: scp: warning: Executing scp1 compatibility. linux 100% |*****************************| 8227 KB 00:00 ETA usermode:~# ls -l linux -rwxr--r-- 1 root root 8424800 May 3 21:09 linux usermode:~# ./linux ubd0=/dev/ubd/1 tracing thread pid = 146 Linux version 2.4.18-21um ([email protected]) (gcc version 2.96 20000731 (Red Hat Linux 7.1 2.96-81)) #1 Wed May 1 21:07:32 EDT 2002 On node 0 totalpages: 8192 zone(0): 0 pages. zone(1): 8192 pages. zone(2): 0 pages. Kernel command line: ubd0=/dev/ubd/1 root=/dev/ubd0 Calibrating delay loop... 707.26 BogoMIPS Memory: 32244k available Dentry-cache hash table entries: 4096 (order: 3, 32768 bytes) Inode-cache hash table entries: 2048 (order: 2, 16384 bytes) Mount-cache hash table entries: 512 (order: 0, 4096 bytes) Buffer-cache hash table entries: 1024 (order: 0, 4096 bytes) Page-cache hash table entries: 8192 (order: 3, 32768 bytes) Checking for host processor cmov support...Reading /proc/cpuinfo failed, errno = 2 Yes Checking for host processor xmm support...Reading /proc/cpuinfo failed, errno = 2 No Checking that ptrace can change system call numbers...OK Checking that host ptys support output SIGIO...No, enabling workaround POSIX conformance testing by UNIFIX Linux NET4.0 for Linux 2.4 Based upon Swansea University Computer Society NET3.039 Initializing RT netlink socket Starting kswapd VFS: Diskquotas version dquot_6.4.0 initialized devfs: v1.10 (20020120) Richard Gooch ([email protected]) devfs: boot_options: 0x1 JFFS version 1.0, (C) 1999, 2000 Axis Communications AB JFFS2 version 2.1. (C) 2001 Red Hat, Inc., designed by Axis Communications AB. pty: 256 Unix98 ptys configured block: 64 slots per queue, batch=16 RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256). loop: loaded (max 8 devices) PPP generic driver version 2.4.1 Universal TUN/TAP device driver 1.4 (C)1999-2001 Maxim Krasnyansky NET4: Linux TCP/IP 1.0 for NET4.0 IP Protocols: ICMP, UDP, TCP IP: routing cache hash table of 512 buckets, 4Kbytes TCP: Hash tables configured (established 2048 bind 2048) NET4: Unix domain sockets 1.0/SMP for Linux NET4.0. Initializing stdio console driver Initializing software serial port version 1 mconsole (version 1) initialized on /root/.uml/yfWc2E/mconsole Partition check: ubda: unknown partition table UML Audio Relay: May 1 2002 21:17:10 VFS: Mounted root (ext2 filesystem) readonly. Mounted devfs on /dev INIT: version 2.84 booting EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended INIT: Entering runlevel: 5
Welcome to the uml version of Tom's root/boot.
This is a customized version, so the notice below is displayed at Tom's request.
*******************************************************************************
* If you base something on it, use any of the scripts, distribute binaries or *
* libraries from it, or distribute customized versions of it: You must credit *
* tomsrtbt and include a pointer to http://www.toms.net/rb/ and [email protected], *
* and include this notice verbatim. Copyright Tom Oehser 1999. This notice in *
* no way supercedes or nullifies any other protections on the component parts *
* such as the BSD and GPL copyrights which apply to practically everything!!! *
* Within these strictures you may redistribute, incorporate, copy, modify, or *
* do anything else to it or with it that you like. Tomsrtbt has no warranties *
* not even implied fitness or usefulness. If it breaks you keep both pieces. *
*******************************************************************************
What you have is...
3c589_cs advansys agetty aha152x aha152x_cs aha1542 aic7xxx ash awk badblocks bdflush buildit.s busLogic busybox bzip2 cardbus cardmgr cat ce ce.help chain.b chattr chgrp chmod chown chroot clear clone.s cmp common config cp cpio cut date dd ddate debugfs df dhcpcd dirname dmesg ds du dumpe2fs e2fsck eata echo echo.c elvis emacs ex extend false fdflush fdformat fdisk fdomain filesize find findsuper fmt fsck.ext2 fsck.msdos fstab grep group gzip halt head hexdump hexedit host.conf hostname hosts i82365 ifconfig ifport ile image init init.old inittab insmod install.s ioctl.save issue kill killall5 ksyms ld ld-linux length less libc libcom_err libe2p libext2fs libss libtermcap libuuid lilo lilo.conf ln loadkeys login losetup ls lsattr man mawk md5sum memtest miterm mkdir mkdosfs mke2fs mkfifo mkfs.minix mklost+found mknod mkswap mnsed more more.help mount mt mtab mv nc ncr53c8xx network networks nmclan_cs ntfs passwd pax pcmcia pcmcia_core pcnet_cs ping plip ppa printf profile protocols ps pwd qlogic_cs qlogicfas rc.0 rc.6 rc.M rc.S rc.custom rc.custom.gz rc.custom~ reboot rescuept reset resolv.conf rm rmdir rmmod route rsh rshd script scsi scsi_info seagate sed serial serial_cs services setserial settings.s sh shared shutdown slattach sleep snarf sort split stty swapoff swapon sync tail tar tcic tee telnet telnetd termcap test tomcr.txt tomshexd tomsrtbt.FAQ touch true tune2fs umount undeb undeb-- unpack.s unrpm-- update utmp vi vi.help view wc wtmplock
...Login as root.
ttys/0 tomsrtbt login: root
Password: ile rev.2.01
Today is Pungenday, the 50th day of Discord in the YOLD 3168 Celebrate Discoflux
# ps uax 1 S (init) init 2 S (keventd) 3 S (ksoftirqd_CPU0) 4 S (kswapd) 5 S (bdflush) 6 S (kupdated) 7 S (mtdblockd) 55 S (ile) ile /bin/sh -c . /etc/profile -si 59 S (sh) /bin/sh -c . /etc/profile -si 61 S (ps) /bin/sh /usr/bin/ps uax 62 R (ps) /bin/sh /usr/bin/ps uax 63 R (sed) sed -e s/\ / /g # halt INIT: Switching to runlevel: 0 INIT: Sending processes the TERM signal
halt
System halted.
nbd: module cleaned up.
usermode:~# halt
Broadcast message from root (vc/0) Fri May 3 21:24:09 2002...
The system is going down for system halt NOW !!
INIT: Switching to runlevel: 0
INIT: Sending processes the TERM signal
INIT: Sending processes the KILL signal
Stopping internet superserver: inetd.
Stopping OpenBSD Secure Shell server: sshd.
Saving the System Clock time to the Hardware Clock...
hwclock: Can't open /dev/tty1, errno=2: No such file or directory. hwclock is unable to get I/O port access: the iopl(3) call failed. Hardware Clock updated to Fri May 3 21:25:17 CEST 2002. Stopping portmap daemon: portmap.
Stopping NFS kernel daemon: mountd nfsd.
Unexporting directories for NFS kernel daemon...done. Stopping NFS common utilities: statd.
Stopping system log daemon: klogd syslogd.
Sending all processes the TERM signal... done.
Sending all processes the KILL signal... done.
Saving random seed... done.
Unmounting remote filesystems... done.
Deconfiguring network interfaces: done.
Deactivating swap... done.
Unmounting local filesystems... done.
* route del -host 192.168.0.253 dev tap0
* bash -c echo 0 > /proc/sys/net/ipv4/conf/tap0/proxy_arp
* arp -i eth0 -d 192.168.0.253 pub
Power down.
nbd: module cleaned up.
um 1012:
Quoted 2 lines · Some moron who can't keep track of his own patches
This works as of UML 2.4.18-21, which isn't released yet, but will be
soon.
Correction - 2.4.18-22, which has just been released.
Jeff
Congratulations!
[Reminds me of the good old times 30 years ago - had a tower of three virtual machines on top of a real PDP 8/I. Now that you can run UML under UML, can you run UML under UML under UML?]
Quoted 6 lines · In message <[email protected]>, > : Guest section DW writes:
Congratulations!
[Reminds me of the good old times 30 years ago - had a tower of three virtual machines on top of a real PDP 8/I. Now that you can run UML under UML, can you run UML under UML under UML?]
Fun stuff! With PTX we were doing something very similar near the end of our days with PTX:
PTX could run Linux Binaries
PTX could run a System 390 emulator (Flex/ES ?)
PTX could *almost* run VMWare (might be able to run Win4Lin or Boochs...) PTX could sever as a Citrix (Windows NT) server
Picture Windows running in VMWare, talking to an OS/390 emulator on the same hardware. You might have been able to run Linux on 390, as well as VM/SP or whatever...
Add on all the other Linux emulators and you had quite a few applications you could run on a single platform, all able to talk to each other. ;-)
Customers wanted to run legacy OS/390 apps that they had lost the binaries for, with a fast, modern database (Oracle or DB2) running at native speed, with either Linux or Windows applications. Add UML and you can do development and client support like System 390 can do with Linux and you have an interesting (if a bit perverted ;-) world.
Sick and twisted...
gerrit
Quoted 1 line · [email protected] said:
Congratulations!
Thanks!
Quoted 2 lines
Now that you can run UML
under UML, can you run UML under UML under UML?]
Heh, I haven't tried it. Feel free, it should work now.
Quoted 5 lines · [email protected] said:
Customers wanted to run legacy OS/390 apps that they had lost the binaries for, with a fast, modern database (Oracle or DB2) running at native speed, with either Linux or Windows applications. Add UML and you can do development and client support like System 390 can do with Linux and you have an interesting (if a bit perverted ;-) world.
Yup. And that sort of thing doesn't come close to reaching the potential of a virtual OS.
A couple of the intermediate-term things I'm most interested in are
embedding UML in things like Apache to provide a standard internal development and execution environment
spreading a SMP UML instance across multiple hosts
Jeff
Quoted 1 line · Gerrit Huizenga
PTX could *almost* run VMWare (might be able to run Win4Lin or Boochs...)
Umm, you have ported the VMWare and Win4Lin kernel modules? For Win4Lin I could almost image it as it is ported UnixWare code..
Quoted 5 lines · In message <[email protected]>, > : Christoph Hellwig writes:
Quoted 1 line · Gerrit Huizenga
PTX could *almost* run VMWare (might be able to run Win4Lin or Boochs...)
Umm, you have ported the VMWare and Win4Lin kernel modules? For Win4Lin I could almost image it as it is ported UnixWare code..
Nope - that's where the part of the *almost* comes from. There were a few other things we didn't get to and the only time I remember someone trying it was when we still didn't have modify_ldt() ported, so that also broke. VMWare wasn't actually a goal - on the same hardware we could statically partition nodes and some nodes could natively run NT while some ran PTX. Now those same nodes can also run Linux as well. And we quit putting major development effort into PTX about two years ago, so we'll probably never find out just how close we were on VMWare.
gerrit
Quoted 3 lines · Jeff Dike
A couple of the intermediate-term things I'm most interested in are
embedding UML in things like Apache to provide a standard internal development and execution environment
How would that benefit Apache, and why does is have any use for an imbedded kernel?
Quoted 2 lines
spreading a SMP UML instance across multiple hosts
How would this be better than MOSIX, or other clustering solutions?
Any URLs you may have on this would be quite helpful.
Mike
Quoted 5 lines
How would this be better than MOSIX, or other clustering solutions?
Any URLs you may have on this would be quite helpful.
uh-huh, you miss the pt maybe? uml offers a great testing, debugging, developing platform. the whole idea is to replace the real thing (say like kernel devel) with UML and your qn is more like why cant we use the real thing itself....:)
Vikram
Quoted 10 lines · Vikram
Quoted 5 lines
How would this be better than MOSIX, or other clustering solutions?
Any URLs you may have on this would be quite helpful.
uh-huh, you miss the pt maybe? uml offers a great testing, debugging, developing platform. the whole idea is to replace the real thing (say like kernel devel) with UML and your qn is more like why cant we use the real thing itself....:)
If you want to test clustering (or "UML SMP over several seperate hosts" -JDike) with UML, why not just create a UML kernel with the clustering support (ie, MOSIX) in that UML kernel?
Really, I'm just asking what the benifit is to use UML for clustering as oposed to MOSIX. I can think of one, testing NUMA without special hardware...
Quoted 2 lines · [email protected] said:
How would that benefit Apache, and why does is have any use for an imbedded kernel?
See http://user-mode-linux.sourceforge.net/slides/wvu2002/wvu2002.htm, which are the slides from a talk I gave at WVU and MorLUG in which I explained this more clearly than I had managed before. The section that's relevant here starts at http://user-mode-linux.sourceforge.net/slides/wvu2002/img9.htm
Quoted 1 line
How would this be better than MOSIX, or other clustering solutions?
MOSIX (or Compaq's SSI) would certainly be a way of doing it. It happens that there's a particularly simple way of doing it with UML. You'd partition UML's 'physical' memory between the hosts, and use the fact that those pages are really virtual to fault them between hosts as needed. This would perform particularly badly, but its simplicity appeals to me.
Jeff
Quoted 8 lines · Jeff Dike
Quoted 1 line · [email protected] said:
How would this be better than MOSIX, or other clustering solutions?
MOSIX (or Compaq's SSI) would certainly be a way of doing it. It happens that there's a particularly simple way of doing it with UML. You'd partition UML's 'physical' memory between the hosts, and use the fact that those pages are really virtual to fault them between hosts as needed. This would perform particularly badly, but its simplicity appeals to me.
See http://www.bitmover.com/cc-pitch/ for some more on this idea. I think the UML approach would be very cool.
Signature · 2 lines
--- Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Quoted 2 lines · [email protected] said:
See http://www.bitmover.com/cc-pitch/ for some more on this idea. I think the UML approach would be very cool.
Actually, what I outlined is different from your CC-smp. A lot simpler and a lot less likely to be practical :-)
http://www.bitmover.com/cc-pitch/ also deserves to be looked at along with MOSIX and Compaq's SSI project for UML (and physical) clustering.
Jeff
Quoted 5 lines · Jeff Dike
MOSIX (or Compaq's SSI) would certainly be a way of doing it. It happens that there's a particularly simple way of doing it with UML. You'd partition UML's 'physical' memory between the hosts, and use the fact that those pages are really virtual to fault them between hosts as needed. This would perform particularly badly, but its simplicity appeals to me.
An interesting and simple approach indeed; but spreading an instance across multiple nodes is nowhere as simple as it seems; where do you keep OS data, IO access, scheduling decisions, inter-node communication in the first place, how to deal with node failure etc...
However, I believe it could potentially be implemented cleaner than currently with the Compaq SSI stuff, because the encapsulation is better etc; but I have been known to be wrong ;-)
It would certainly be very interesting. If you _really_ want to open this can of worms, you should consider joining linux-cluster mailing list for this, or the Open Clustering Framework list (because you are going to stumble into the madness which is "interoperability and lack of standards" here).
Sincerely,
Lars Marowsky-Brée <[email protected]>
Signature · 2 lines
Immortality is an adequate definition of high availability for me. --- Gregory F. Pfister
Quoted 2 lines · [email protected] said:
but spreading an instance across multiple nodes is nowhere as simple as it seems;
It is if you want to be sufficiently stupid about it :-)
Quoted 1 line
where do you keep OS data,
It gets faulted from host to host as needed.
Quoted 1 line
IO access,
This scheme (and any clustering scheme, I think) would need back channels for one node to access the devices of another
Quoted 1 line
scheduling decisions,
This machine thinks it's a normal SMP box, so scheduling happens as normal
Quoted 2 lines
inter-node communication in the first place, how to deal with node failure etc...
Maybe I'm not familiar enough with the clustering world, but I was under the impression that with a normal SSI cluster, the nodes are like CPUs in an SMP box - if one fails, the whole thing dies. In other words, that SSI clustering and HA clustering are pretty disjoint.
Jeff
Quoted 3 lines · Jeff Dike
Quoted 2 lines
but spreading an instance across multiple nodes is nowhere as simple as it seems;
It is if you want to be sufficiently stupid about it :-)
Ugh.
Quoted 2 lines
Quoted 1 line
where do you keep OS data,
It gets faulted from host to host as needed.
Ugh Ugh Ugh. You need coherency algorithms for these.
Quoted 3 lines
Quoted 1 line
IO access,
This scheme (and any clustering scheme, I think) would need back channels for one node to access the devices of another
Right. You need communication services and coherency algorithms for these ;-)
Quoted 2 lines
Quoted 1 line
scheduling decisions,
This machine thinks it's a normal SMP box, so scheduling happens as normal
Ugh ugh ugh. Too many page faults; you need a scheduler capable of keeping node affinity.
Quoted 6 lines
Quoted 2 lines
inter-node communication in the first place, how to deal with node failure etc...
Maybe I'm not familiar enough with the clustering world, but I was under the impression that with a normal SSI cluster, the nodes are like CPUs in an SMP box - if one fails, the whole thing dies. In other words, that SSI clustering and HA clustering are pretty disjoint.
No. In the optimal case, nothing dies. ;-) In the less than optimal case, only the processes affected directly by the failure die (the processes which had dirty pages there etc).
In the really useless case, the entire cluster goes down like on a SMP box or an unpartitioned CC-NUMA.
"In search of clusters" is definetely highly recommended reading. It is very entertaining, too.
Sincerely,
Lars Marowsky-Brée <[email protected]>
Signature · 2 lines
Immortality is an adequate definition of high availability for me. --- Gregory F. Pfister
Quoted 5 lines · [email protected] said:
Ugh.
Ugh Ugh Ugh.
Ugh ugh ugh. Too many page faults;
You will note that the only favorable adjective I've used in describing this scheme is 'easy' :-)
The too many page faults is covered by my 'performance will suck'...
Jeff
Quoted 5 lines · Lars Marowsky-Bree
Quoted 2 lines
This machine thinks it's a normal SMP box, so scheduling happens as normal
Ugh ugh ugh. Too many page faults; you need a scheduler capable of keeping node affinity.
O(1)?
Thought it did. Might have to teach the load balancer to be a bit more clever...
Rob
Hi!
Quoted 6 lines
Yup. And that sort of thing doesn't come close to reaching the potential of a virtual OS.
A couple of the intermediate-term things I'm most interested in are
embedding UML in things like Apache to provide a standard internal development and execution environment
This is little perverted, right? What is it good for?
Quoted 1 line
spreading a SMP UML instance across multiple hosts
Now this looks very interesting to me.
Pavel
Signature · 2 lines
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt, details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.
Quoted 3 lines · [email protected] said:
Quoted 2 lines
embedding UML in things like Apache to provide a standard internal development and execution environment
This is little perverted, right?
Maybe :-)
Quoted 1 line
What is it good for?
Secure mod_perl - i.e. mod_perl in an Apache that's shared with other web sites
mod_perl in any language supported by Linux
Interactive debugging of your perl on live requests inside a live Apache
See http://user-mode-linux.sourceforge.net/slides/wvu2002/wvu2002.htm
The section that's relevant here starts at
http://user-mode-linux.sourceforge.net/slides/wvu2002/img9.htm
There are also some wackier possibilities which I'm not sure are terribly useful or practical, but would still be interesting to look at, such as:
Treating HTTP requests as processes, so
# ps uax ... apache 1120 0.0 0.6 4388 1536 ? S 13:46 0:00 GET / HTTP/1.0 ... # kill -9 1120 or # nice -20 1120
Treating HTTP requests as packets and using Netfilter to manipulate them.
Jeff