Wiki for LXC with Checkpoint/Restart

Hi,

We created following wiki to describe the process to build/test LXC with checkpoint/restart.

https://ckpt.wiki.kernel.org/index.php/Link-LXC-USERCR

The USERCR and LXC patches that are needed are in:

http://lxc.sourceforge.net/patches/lxc+usercr/usercr-patches/
http://lxc.sourceforge.net/patches/lxc+usercr/lxc-patches/

(and also referred to in the above wiki).

For now, the wiki identifies specific "commit-id"s to identify the commit with which the instructions in the wiki were tested. We hope to create more meaningful tags to the commits and if so, we will update the wiki with the tags.

Please let me know if you have any comments on the wiki or run into problems.

Thanks,

Sukadev

Commit IDs are ugly but unambiguous. I didn't have a problem just copy/pasting the commands.

I took the liberty of cleaning up a few stray '|' characters in the last code snippet and fixed up the deprecated old-style git invocations (e.g. "git-clone" becomes "git clone"). Not trying to be picky about the git thing, but the hyphenated commands don't work on newer distributions.

I ran into an issue trying to follow the instructions on a SLES11 x86_64 system. When building lxc I get:

gcc -fPIC -DPIC -I../../src -DLIBCR -I/home/nathanl/user-cr
-I/home/nathanl/user-cr/include -g -O2 -Wall
-shared /home/nathanl/user-cr/checkpoint.o /home/nathanl/user-cr/restart.o /home/nathanl/user-cr/libeclone.a -Wl,-soname,liblxc.so.0 -o liblxc.so liblxc_so-arguments.o liblxc_so-commands.o liblxc_so-start.o liblxc_so-stop.o liblxc_so-monitor.o liblxc_so-console.o liblxc_so-freezer.o liblxc_so-checkpoint.o liblxc_so-restart.o liblxc_so-error.o liblxc_so-parse.o liblxc_so-cgroup.o liblxc_so-utils.o liblxc_so-namespace.o liblxc_so-conf.o liblxc_so-confile.o liblxc_so-state.o liblxc_so-log.o liblxc_so-network.o liblxc_so-nl.o liblxc_so-rtnl.o liblxc_so-genl.o liblxc_so-mainloop.o liblxc_so-af_unix.o -lutil /usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: /home/nathanl/user-cr/checkpoint.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC

Not sure what to do about it immediately; user-cr/checkpoint.o should be built with -fPIC I guess?

Nathan Lynch [[email protected]] wrote:

| On Mon, 2010-04-12 at 15:27 -0700, Sukadev Bhattiprolu wrote:
| > Hi,
| > 
| > We created following wiki to describe the process to build/test LXC
| > with checkpoint/restart.
| > 
| > 	https://ckpt.wiki.kernel.org/index.php/Link-LXC-USERCR
| > 
| > The USERCR and LXC patches that are needed are in:
| > 
| >         http://lxc.sourceforge.net/patches/lxc+usercr/usercr-patches/
| >         http://lxc.sourceforge.net/patches/lxc+usercr/lxc-patches/
| > 
| > (and also referred to in the above wiki).
| > 
| > For now, the wiki identifies specific "commit-id"s to identify the
| > commit with which the instructions in the wiki were tested. We hope
| > to create more meaningful tags to the commits and if so, we will update
| > the wiki with the tags.
| 
| Commit IDs are ugly but unambiguous.  I didn't have a problem just
| copy/pasting the commands.
| 
| > Please let me know if you have any comments on the wiki or run into
| > problems. 
| 
| I took the liberty of cleaning up a few stray '|' characters in the last
| code snippet and fixed up the deprecated old-style git invocations (e.g.
| "git-clone" becomes "git clone").  Not trying to be picky about the git
| thing, but the hyphenated commands don't work on newer distributions.
| 
| I ran into an issue trying to follow the instructions on a SLES11 x86_64
| system.  When building lxc I get:
| 
| gcc -fPIC -DPIC -I../../src -DLIBCR -I/home/nathanl/user-cr
| -I/home/nathanl/user-cr/include -g -O2 -Wall
| -shared /home/nathanl/user-cr/checkpoint.o /home/nathanl/user-cr/restart.o /home/nathanl/user-cr/libeclone.a -Wl,-soname,liblxc.so.0  -o liblxc.so liblxc_so-arguments.o liblxc_so-commands.o liblxc_so-start.o liblxc_so-stop.o liblxc_so-monitor.o liblxc_so-console.o liblxc_so-freezer.o liblxc_so-checkpoint.o liblxc_so-restart.o liblxc_so-error.o liblxc_so-parse.o liblxc_so-cgroup.o liblxc_so-utils.o liblxc_so-namespace.o liblxc_so-conf.o liblxc_so-confile.o liblxc_so-state.o liblxc_so-log.o liblxc_so-network.o liblxc_so-nl.o liblxc_so-rtnl.o liblxc_so-genl.o liblxc_so-mainloop.o liblxc_so-af_unix.o -lutil
| /usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: /home/nathanl/user-cr/checkpoint.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
| 
| Not sure what to do about it immediately; user-cr/checkpoint.o should be
| built with -fPIC I guess?

Yes, I think thats what Michel or Daniel ran into. For some reason, I did not run into on Fedora11. Can you add -fPIC to the usercr CFLAGS and try ? If it works for you, I will try on Fedora11 too and we can add a patch to lxc.sf.net/patches/lxc+usercr/.

We may need to do something similar for any kernel patches that are needed on top of each "release" of the wiki.

Sukadev

Yes, adding -fPIC to CFLAGS in the user-cr Makefile and rebuilding allows the lxc build to succeed.

Okay, so after getting the kernel build issues worked around and documented, I've gotten to part 4 "Checkpoint/restart a simple LXC container" in the wiki. It works! A couple of minor issues with the instructions:

The full path to the rcfile needs to be supplied or lxc-execute errors out. Also, it seems that the above command needs its own terminal or it should be run in the background.

Is it intentional that this fails if /root/lxc-foo.ckpt already exists?

When building the kernel with the commit id specified in the wiki (0fdca57) I get:

ERROR: "ckpt_netdev_base" [net/ipv6/sit.ko] undefined!
ERROR: "ckpt_netdev_in_init_netns" [drivers/net/veth.ko] undefined!
ERROR: "ckpt_netdev_base" [drivers/net/veth.ko] undefined!
ERROR: "ckpt_netdev_base" [drivers/net/macvlan.ko] undefined! make[1]: *** [__modpost] Error 1
make: *** [modules] Error 2

NL> When building the kernel with the commit id specified in the wiki NL> (0fdca57) I get:

NL> ERROR: "ckpt_netdev_base" [net/ipv6/sit.ko] undefined! NL> ERROR: "ckpt_netdev_in_init_netns" [drivers/net/veth.ko] undefined! NL> ERROR: "ckpt_netdev_base" [drivers/net/veth.ko] undefined! NL> ERROR: "ckpt_netdev_base" [drivers/net/macvlan.ko] undefined! NL> make[1]: *** [__modpost] Error 1
NL> make: *** [modules] Error 2

Right and the fixes in my latest set should eliminate that :)

Dan Smith
IBM Linux Technology Center
email: danms-r/[email protected]

I expected as much, but the question is how should we address this kind of problem in the wiki? Are the patches in a publicly available branch?

NL> Are the patches in a publicly available branch?

Not that I'm aware of, because they're wrapped in with some functional changes and cleanups to the way the stuff is called. I suppose I could hack in some #ifdef's to the public bits to fix that, if needed.

Dan Smith
IBM Linux Technology Center
email: danms-r/[email protected]

Dan Smith [danms-r/[email protected]] wrote:

| NL> Are the patches in a publicly available branch?
| 
| Not that I'm aware of, because they're wrapped in with some functional
| changes and cleanups to the way the stuff is called.  I suppose I
| could hack in some #ifdef's to the public bits to fix that, if
| needed.

Do we run into the compile error only if some config tokens are selected ? Given that the instructions in the wiki don't use networking heavily, if we have a simple hack for the commit in the wiki, we could post a kernel patch to the lxc.sf.net directory.

| 
| -- 
| Dan Smith
| IBM Linux Technology Center
| email: danms-r/[email protected]

SB> Do we run into the compile error only if some config tokens are selected ? SB> Given that the instructions in the wiki don't use networking heavily, SB> if we have a simple hack for the commit in the wiki, we could post a SB> kernel patch to the lxc.sf.net directory.

Yeah, if you don't do CONFIG_IPV6=m you should be fine.

Dan Smith
IBM Linux Technology Center
email: danms-r/[email protected]