Skip to contentSkip to conversation

: First try at integrating LXC and USER-CR

RFC PATCH In review Sukadev Bhattiprolu 4 participants 17 messages Feb 24, 2010 – Mar 3, 2010 8 messages not in the archive Kbuild

lore
Share & tools
lore
#0 lore raw

cover : First try at integrating LXC and USER-CR

Following two sets of patches is an early attempt to integrate LXC and USER-CR.

Overview:

Have USER-CR export the core checkpoint and restart functionality into a library (/lib/libcheckpoint.a and <usercr.h>) and have LXC link with this library.

TODO:

1. For now, libcheckpoint.a implements only the restart functionality and so only lxc_restart command is implemented. Implementing the checkpoint functionality and lxc_checkpoint command can be done similarly and is hopefully easier than the restart functionality.

2. The restart() functionality in user-cr makes extensive use of global variables and debug code. The API must be extended to properly include these variables/debug code in the API.

Similarly, the 'struct restart_args' may need to be sanitized for use in a formal API.

3. lxc_restart command restarts entire containers only (specifically it simulates the --pidns --pids --mount-pty arguments to /bin/restart).

4. Link lxc_restart and lxc_checkpoint with the shared library liblxc.so (currently links statically)

5. ...

STATUS:
I was able to checkpoint/restart a simple '/bin/sleep 1000' LXC container, except for a cgroup naming issue after restart (see below).

STEPS:

1. [USER-CR] Build/install /lib/libcheckpoint.a, /usr/include/usercr.h

1.1 Apply the attached [user-cr] patches to the user-cr git tree
(I tested with following commit as base)

commit 67cfee9329670ab28eb1a52e94745252b614718f
Author: Oren Laadan <[email protected]>
Date: Mon Feb 22 18:00:06 2010 -0500

1.2 Build/install user-cr binaries/libraries/includes

   	$ make all
	$ make install

This should install /lib/libcheckpoint.a and /usr/include/usercr.h

2. [LXC] Build lxc_restart using USER-CR API (usercr.h, libcheckpoint.a)

2.1 Apply attached [lxc] patches to Daniel Lezcano's lxc.git tree (0.6.5)

2.2 Build lxc_restart (this uses static linking for now)

   	$ make -f Makefile2 lxc_restart

3. Create and checkpoint a simple LXC container

	$ lxc-execute --name foo --rcfile lxc-macvlan.conf -- /bin/sleep 1000
	$ lxc-freeze --name foo
	TODO: 
		lxc_checkpoint --name foo should checkpoint the container,
		For now, use "lxc-ps --name foo" to find pid of lxc-init and
		checkpoint using:

		$ /bin/checkpoint --output=/tmp/sleep.ckpt <pid-of-lxc-init>

	$ lxc-unfreeze --name foo

	$ lxc-stop --name foo

4. Restart a checkpointed LXC container

	$ ./lxc_restart --statefile /tmp/sleep.ckpt --name bar

# Test some common lxc commands after restart

	$ lxc-ps --name "bar/1"
	CONTAINER    PID TTY          TIME CMD
	bar/1       8511 ?        00:00:00 lxc-init
	bar/1       8512 ?        00:00:00 sleep
	$ lxc-freeze --name "bar/1"
	$ grep State /proc/8511/status 
	State:	D (disk sleep)
	$ grep State /proc/8512/status 
	State:	D (disk sleep)

NOTE: For some reason, the container name after restart is "bar/1"
instead of "bar". Due to this, when the lxc_restart is exiting, I get a "-EBUSY - failed to remove "/cgroup/bar" error. I need to fix this still.

A message that is not in the archive (sent off-list or not yet received): 20100224083452.GB18758-r/[email protected]

#2 lore raw

PATCH 1/4 Move common definitions to restart.h

From: Sukadev Bhattiprolu <[email protected]>
Date: Tue, 23 Feb 2010 15:25:31 -0800
Subject: [PATCH 1/4][user-cr] Move common definitions to restart.h

To make restart() a library interface, the main() and related code in restart.c should be moved to a separate file. To prepare for the move, move some common definitions to a new file, 'restart.h'

---
 restart.c |   44 ++------------------------------------------
 restart.h |   45 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+), 42 deletions(-)
 create mode 100644 restart.h
restart.c +2 -42
index f5d23bb..7140786 100644
@@ -39,6 +39,7 @@
 #include "eclone.h"
 #include "genstack.h"
 #include "compat.h"
+#include "restart.h"
 
 static char usage_str[] =
 "usage: restart [opts]\n"
@@ -128,12 +129,7 @@ static char usage_str[] =
 			printf(__VA_ARGS__);	\
 	} while(0)
 
-#define SIGNAL_ENTRY(signal)  { SIG ## signal, #signal }
-
-struct {
-	int signum;
-	char *sigstr;
-} signal_array[] = {
+struct signal_array signal_array[] = {
 	{ 0, "NONE" },
 	SIGNAL_ENTRY(ALRM),
 	SIGNAL_ENTRY(HUP),
@@ -196,7 +192,6 @@ inline static int restart(pid_t pid, int fd, unsigned long flags, int logfd)
 	return syscall(__NR_restart, pid, fd, flags, logfd);
 }
 
-#define BUFSIZE  (4 * 4096)
 
 struct hashent {
 	long key;
@@ -205,29 +200,6 @@ struct hashent {
 };
 
 struct task;
-struct ckpt_ctx;
-
-struct task {
-	int flags;		/* state and (later) actions */
-
-	struct task *children;	/* pointers to first child, next and prev */
-	struct task *next_sib;	/*   sibling, and the creator of a process */
-	struct task *prev_sib;
-	struct task *creator;
-
-	struct task *phantom;	/* pointer to place-holdler task (if any) */
-
-	pid_t pid;		/* process IDs, our bread-&-butter */
-	pid_t ppid;
-	pid_t tgid;
-	pid_t sid;
-	
-	pid_t rpid;		/* [restart without vpids] actual (real) pid */
-
-	struct ckpt_ctx *ctx;	/* points back to the c/r context */
-
-	pid_t real_parent;	/* pid of task's real parent */
-};
 
 /* zero_task represents creator of root_task (all pids 0) */
 struct task zero_task;
@@ -360,18 +332,6 @@ struct args {
 	int keep_lsm;
 };
 
-#define CKPT_COND_PIDZERO  0x1
-#define CKPT_COND_MNTPROC  0x2
-#define CKPT_COND_MNTPTY   0x4
-
-#define CKPT_COND_NONE     0
-#define CKPT_COND_ANY      ULONG_MAX
-
-/* default for skip/warn/fail */
-#define CKPT_COND_WARN     (CKPT_COND_MNTPROC | \
-			    CKPT_COND_MNTPTY)
-#define CKPT_COND_FAIL     (CKPT_COND_NONE)
-
 static void usage(char *str)
 {
 	fprintf(stderr, "%s", str);
restart.h new file +45 -0

45 lines not on this page · raw message

Signature · 1 line
1.6.6.1

A message that is not in the archive (sent off-list or not yet received): 20100224083534.GC18758-r/[email protected]

#4 lore raw

PATCH 1/1 c/r: fix regression (in "fix scheduling in atomic while restoring ipc shm")

So... here's a simple fix to a silly regression - this should bring an end to my little fiasco.

---
 ipc/checkpoint_msg.c |    2 +-
 ipc/checkpoint_sem.c |    2 +-
 ipc/checkpoint_shm.c |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
ipc/checkpoint_msg.c +1 -1
index 073a893..0155c20 100644
@@ -359,7 +359,7 @@ int restore_ipc_msg(struct ckpt_ctx *ctx, struct ipc_namespace *ns)
 	 * ipc-ns, we will need to re-examine this.
 	 */
 
-	ipc = idr_find(&msg_ids->ipcs_idr, h->perms.id);
+	ipc = idr_find(&msg_ids->ipcs_idr, ipcid_to_idx(h->perms.id));
 	BUG_ON(!ipc);
 
 	msq = container_of(ipc, struct msg_queue, q_perm);
ipc/checkpoint_sem.c +1 -1
index 78c1932..2abfb5a 100644
@@ -216,7 +216,7 @@ int restore_ipc_sem(struct ckpt_ctx *ctx, struct ipc_namespace *ns)
 	 * ipc-ns, we will need to re-examine this.
 	 */
 
-	ipc = idr_find(&sem_ids->ipcs_idr, h->perms.id);
+	ipc = idr_find(&sem_ids->ipcs_idr, ipcid_to_idx(h->perms.id));
 	BUG_ON(!ipc);
 
 	sem = container_of(ipc, struct sem_array, sem_perm);
ipc/checkpoint_shm.c +1 -1
index 6329245..a07c051 100644
@@ -270,7 +270,7 @@ int restore_ipc_shm(struct ckpt_ctx *ctx, struct ipc_namespace *ns)
 	 * ipc-ns, we will need to re-examine this.
 	 */
 
-	ipc = idr_find(&shm_ids->ipcs_idr, h->perms.id);
+	ipc = idr_find(&shm_ids->ipcs_idr, ipcid_to_idx(h->perms.id));
 	BUG_ON(!ipc);
 
 	shp = container_of(ipc, struct shmid_kernel, shm_perm);
Signature · 1 line
1.6.3.3

A message that is not in the archive (sent off-list or not yet received): 1267574649-14269-1-git-send-email-orenl-eQaUEPhvms7ENvBUuze7eA@public.gmane.org

Tested-by
#6 lore raw

Re: [PATCH] c/r: fix regression (in "fix scheduling in atomic while restoring ipc shm")

Quoted 4 lines · Oren Laadan

So... here's a simple fix to a silly regression - this should bring an end to my little fiasco.

Quoted 47 lines
---
 ipc/checkpoint_msg.c |    2 +-
 ipc/checkpoint_sem.c |    2 +-
 ipc/checkpoint_shm.c |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
ipc/checkpoint_msg.c +1 -1
index 073a893..0155c20 100644
@@ -359,7 +359,7 @@ int restore_ipc_msg(struct ckpt_ctx *ctx, struct ipc_namespace *ns)
 	 * ipc-ns, we will need to re-examine this.
 	 */
 
-	ipc = idr_find(&msg_ids->ipcs_idr, h->perms.id);
+	ipc = idr_find(&msg_ids->ipcs_idr, ipcid_to_idx(h->perms.id));
 	BUG_ON(!ipc);
 
 	msq = container_of(ipc, struct msg_queue, q_perm);
ipc/checkpoint_sem.c +1 -1
index 78c1932..2abfb5a 100644
@@ -216,7 +216,7 @@ int restore_ipc_sem(struct ckpt_ctx *ctx, struct ipc_namespace *ns)
 	 * ipc-ns, we will need to re-examine this.
 	 */
 
-	ipc = idr_find(&sem_ids->ipcs_idr, h->perms.id);
+	ipc = idr_find(&sem_ids->ipcs_idr, ipcid_to_idx(h->perms.id));
 	BUG_ON(!ipc);
 
 	sem = container_of(ipc, struct sem_array, sem_perm);
ipc/checkpoint_shm.c +1 -1
index 6329245..a07c051 100644
@@ -270,7 +270,7 @@ int restore_ipc_shm(struct ckpt_ctx *ctx, struct ipc_namespace *ns)
 	 * ipc-ns, we will need to re-examine this.
 	 */
 
-	ipc = idr_find(&shm_ids->ipcs_idr, h->perms.id);
+	ipc = idr_find(&shm_ids->ipcs_idr, ipcid_to_idx(h->perms.id));
 	BUG_ON(!ipc);
 
 	shp = container_of(ipc, struct shmid_kernel, shm_perm);
Signature · 1 line
1.6.3.3
#7 lore raw

PATCH 2/4 Rename struct args to struct restart_args

In reply to a message not in the archive

From 8a6fad2170fbbeb7a5e44c32239f9703c0b7d2f9 Mon Sep 17 00:00:00 2001
From: Sukadev Bhattiprolu <[email protected]>
Date: Tue, 23 Feb 2010 15:33:04 -0800
Subject: [PATCH 2/4][user-cr] Rename struct args to struct restart_args

Rename struct args to struct restart_args and move to a new header file, usercr.h. usercr.h will become part of the external API for application checkpoint restart code.

Also move the bulk of the restart code into a separate function, app_restart() and have main() call the function.

TODO:
Cleanup 'struct restart_args' for a more consistent external API.

---
 restart.c |   77 ++++++++++++++++++++++++------------------------------------
 usercr.h  |   24 +++++++++++++++++++
 2 files changed, 55 insertions(+), 46 deletions(-)
 create mode 100644 usercr.h
restart.c +31 -46
index 7140786..b2281fb 100644
@@ -40,6 +40,7 @@
 #include "genstack.h"
 #include "compat.h"
 #include "restart.h"
+#include "usercr.h"
 
 static char usage_str[] =
 "usage: restart [opts]\n"
@@ -237,7 +238,7 @@ struct ckpt_ctx {
 	char container[BUFSIZE];
 	char tree[BUFSIZE];
 	char buf[BUFSIZE];
-	struct args *args;
+	struct restart_args *args;
 
 	char *freezer;
 };
@@ -310,28 +311,6 @@ struct pid_swap {
 	pid_t new;
 };
 
-struct args {
-	int self;
-	int pids;
-	int pidns;
-	int no_pidns;
-	int inspect;
-	char *root;
-	int wait;
-	int mntns;
-	int mnt_pty;
-	int show_status;
-	int copy_status;
-	char *freezer;
-	char *input;
-	int infd;
-	char *logfile;
-	int logfd;
-	long warn;
-	long fail;
-	int keep_lsm;
-};
-
 static void usage(char *str)
 {
 	fprintf(stderr, "%s", str);
@@ -382,7 +361,7 @@ static inline int ckpt_cond_fail(struct ckpt_ctx *ctx, long mask)
 	return (ctx->args->fail & mask);
 }
 
-static void parse_args(struct args *args, int argc, char *argv[])
+static void parse_args(struct restart_args *args, int argc, char *argv[])
 {
 	static struct option opts[] = {
 		{ "help",	no_argument,		NULL, 'h' },
@@ -712,71 +691,77 @@ static int freezer_register(struct ckpt_ctx *ctx, pid_t pid)
 
 int main(int argc, char *argv[])
 {
+	struct restart_args args;
+
+	parse_args(&args, argc, argv);
+
+	return app_restart(&args);
+}
+
+int app_restart(struct restart_args *args)
+{
 	struct ckpt_ctx ctx;
-	struct args args;
 	int ret;
 
 	memset(&ctx, 0, sizeof(ctx));
-
-	parse_args(&args, argc, argv);
-	ctx.args = &args;
+	ctx.args = args;
 
 	/* input file ? */
-	if (args.input) {
-		args.infd = open(args.input, O_RDONLY, 0);
-		if (args.infd < 0) {
+	if (args->input) {
+		args->infd = open(args->input, O_RDONLY, 0);
+		if (args->infd < 0) {
 			perror("open input file");
 			exit(1);
 		}
 	}
 
 	/* input file descriptor (default: stdin) */
-	if (args.infd >= 0) {
-		if (dup2(args.infd, STDIN_FILENO) < 0) {
+	if (args->infd >= 0) {
+		if (dup2(args->infd, STDIN_FILENO) < 0) {
 			perror("dup2 input file");
 			exit(1);
 		}
-		if (args.infd != STDIN_FILENO)
-			close(args.infd);
+		if (args->infd != STDIN_FILENO)
+			close(args->infd);
 	}
 
 	/* (optional) log file */
-	if (args.logfile) {
-		args.logfd = open(args.logfile,
+	if (args->logfile) {
+		args->logfd = open(args->logfile,
 				  O_RDWR | O_CREAT | O_EXCL, 0644);
-		if (args.logfd < 0) {
+		if (args->logfd < 0) {
 			perror("open log file");
 			exit(1);
 		}
 	}
 
 	/* output file descriptor (default: none) */
-	if (args.logfd < 0)
-		args.logfd = CHECKPOINT_FD_NONE;
+	if (args->logfd < 0)
+		args->logfd = CHECKPOINT_FD_NONE;
 
 	/* freezer preparation */
-	if (args.freezer && freezer_prepare(&ctx) < 0)
+	if (args->freezer && freezer_prepare(&ctx) < 0)
 		exit(1);
 
 	/* private mounts namespace ? */
-	if (args.mntns && unshare(CLONE_NEWNS | CLONE_FS) < 0) {
+	if (args->mntns && unshare(CLONE_NEWNS | CLONE_FS) < 0) {
 		perror("unshare");
 		exit(1);
 	}
 
 	/* chroot ? */
-	if (args.root && chroot(args.root) < 0) {
+	if (args->root && chroot(args->root) < 0) {
 		perror("chroot");
 		exit(1);
 	}
 
 	/* remount /dev/pts ? */
-	if (args.mnt_pty && ckpt_remount_devpts(&ctx) < 0)
+	if (args->mnt_pty && ckpt_remount_devpts(&ctx) < 0)
 		exit(1);
 
 	/* self-restart ends here: */
-	if (args.self) {
-		restart(getpid(), STDIN_FILENO, RESTART_TASKSELF, args.logfd);
+	if (args->self) {
+		restart(getpid(), STDIN_FILENO, RESTART_TASKSELF, args->logfd);
 		/* reach here if restart(2) failed ! */
 		perror("restart");
 		exit(1);
usercr.h new file +24 -0
new file mode 100644
index 0000000..71a36e6
@@ -0,0 +1,24 @@
+
+struct restart_args {
+	int self;
+	int pids;
+	int pidns;
+	int no_pidns;
+	int inspect;
+	char *root;
+	int wait;
+	int mntns;
+	int mnt_pty;
+	int show_status;
+	int copy_status;
+	char *freezer;
+	char *input;
+	int infd;
+	char *logfile;
+	int logfd;
+	long warn;
+	long fail;
+	int keep_lsm;
+};
+
+extern int app_restart(struct restart_args *args);
Signature · 1 line
1.6.6.1
#8 lore raw

PATCH 3/4 Move main() in restart.c to restart-main.c

In reply to a message not in the archive

From 6553b7da33f56e9cb6927e7469af22df56b7f55f Mon Sep 17 00:00:00 2001
From: Sukadev Bhattiprolu <[email protected]>
Date: Tue, 23 Feb 2010 15:55:34 -0800
Subject: [PATCH 3/4][user-cr] Move main() in restart.c to restart-main.c

This would enable us to put restart.o into a library.

TODO:
restart.c still has a whole bunch of external variables that would need to be addressed (maybe encapsulate in structure)

---
 Makefile       |    2 +
 restart-main.c |  312 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 restart.c      |  293 ----------------------------------------------------
 3 files changed, 314 insertions(+), 293 deletions(-)
 create mode 100644 restart-main.c
Makefile +2 -0
index b312358..4449081 100644
@@ -21,6 +21,8 @@ CFLAGS += -g $(WARNS) $(CKPT_INCLUDE) $(DEBUG)
 BIN_INSTALL_DIR = /bin
 LIB_INSTALL_DIR = /lib
 
+restart: restart.o restart-main.o
+
 ECLONE_PROGS = restart nsexec
 PROGS =	checkpoint ckptinfo $(ECLONE_PROGS)
 LIB_ECLONE = libeclone.a
restart-main.c new file +312 -0

312 lines not on this page · raw message

restart.c +0 -293

323 lines not on this page · raw message

Signature · 1 line
1.6.6.1
#9 lore raw

PATCH 4/4 Rename libeclone.a to libcheckpoint.a

In reply to a message not in the archive

From c4063a8976fd8eca9b8d62a12b95c3125c8471c7 Mon Sep 17 00:00:00 2001
From: Sukadev Bhattiprolu <[email protected]>
Date: Tue, 23 Feb 2010 16:25:36 -0800
Subject: [PATCH 4/4][user-cr] Rename libeclone.a to libcheckpoint.a

Export app_restart() and usercr.h to user in a new library libcheckpoint.a. Until eclone() makes it into glibc, include eclone() also in this new library and remove the libeclone.a.

TODO:
- cr-tests should be modified to look for libcheckpoint.a instead of libeclone.a

- libcheckpoint.a should eventually include the app_checkpoint() interface. I am guessing extracting the app_checkpoint() from checkpoint.c would be similar to (or somewhat easier than) the way we extracted app_restart() from restart.c

---
 Makefile  |   33 ++++++++++++++++++++-------------
 restart.c |    6 ++++++
 2 files changed, 26 insertions(+), 13 deletions(-)
Makefile +20 -13
index 4449081..090914a 100644
@@ -18,14 +18,14 @@ WARNS := -Wall -Wstrict-prototypes -Wno-trigraphs
 CFLAGS += -g $(WARNS) $(CKPT_INCLUDE) $(DEBUG)
 
 # install dir
+INC_INSTALL_DIR = /usr/include
 BIN_INSTALL_DIR = /bin
 LIB_INSTALL_DIR = /lib
 
-restart: restart.o restart-main.o
-
 ECLONE_PROGS = restart nsexec
 PROGS =	checkpoint ckptinfo $(ECLONE_PROGS)
-LIB_ECLONE = libeclone.a
+LIB_CHECKPOINT = libcheckpoint.a
+INC_CHECKPOINT = usercr.h
 
 # other cleanup
 OTHER = ckptinfo_types.c
@@ -34,32 +34,37 @@ LDLIBS = -lm
 
 .PHONY: all distclean clean headers install
 
-all: $(PROGS)
+all: $(PROGS) $(LIB_CHECKPOINT)
 	@make -C test
 
-$(LIB_ECLONE):
-	ar ruv $(LIB_ECLONE) $^
+restart: restart-main.o
+
+$(PROGS): $(LIB_CHECKPOINT) 
+
+$(LIB_CHECKPOINT):
+	ar ruv $(LIB_CHECKPOINT) $^
 
 # restart needs to be thread-safe
 restart: CFLAGS += -D__REENTRANT -pthread
 
+$(LIB_CHECKPOINT): restart.o
+
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
-$(ECLONE_PROGS): $(LIB_ECLONE) 
 $(ECLONE_PROGS): CFLAGS += -DARCH_HAS_ECLONE
-$(LIB_ECLONE): clone_$(SUBARCH).o genstack.o
+$(LIB_CHECKPOINT): clone_$(SUBARCH).o genstack.o
 endif
 
 # on powerpc, need also assembly file
 ifeq ($(SUBARCH),ppc)
 CFLAGS += -m32
 ASFLAGS += -m32
-$(LIB_ECLONE): clone_$(SUBARCH)_.o
+$(LIB_CHECKPOINT): clone_$(SUBARCH)_.o
 endif
 ifeq ($(SUBARCH),ppc64)
 CFLAGS += -m64
 ASFLAGS += -m64
-$(LIB_ECLONE): clone_$(SUBARCH)_.o
+$(LIB_CHECKPOINT): clone_$(SUBARCH)_.o
 endif
 
 # ckptinfo dependencies
@@ -71,8 +76,10 @@ ckptinfo_types.c: $(CKPT_HEADERS) ckptinfo.py
 install:
 	@echo /usr/bin/install -m 755 checkpoint restart nsexec ckptinfo $(BIN_INSTALL_DIR)
 	@/usr/bin/install -m 755 checkpoint restart ckptinfo nsexec $(BIN_INSTALL_DIR)
-	@echo /usr/bin/install -m 755 $(LIB_ECLONE) $(LIB_INSTALL_DIR)
-	@/usr/bin/install -m 755 $(LIB_ECLONE) $(LIB_INSTALL_DIR)
+	@echo /usr/bin/install -m 755 $(INC_CHECKPOINT) $(INC_INSTALL_DIR)
+	@/usr/bin/install -m 755 $(INC_CHECKPOINT) $(INC_INSTALL_DIR)
+	@echo /usr/bin/install -m 755 $(LIB_CHECKPOINT) $(LIB_INSTALL_DIR)
+	@/usr/bin/install -m 755 $(LIB_CHECKPOINT) $(LIB_INSTALL_DIR)
 
 $(CKPT_HEADERS): %:
 	./scripts/extract-headers.sh -s $(KERNELSRC) -o ./include
@@ -83,5 +90,5 @@ distclean: clean
 	@rm -f $(CKPT_HEADERS)
 
 clean:
-	@rm -f $(PROGS) $(LIB_ECLONE) $(OTHER) *~ *.o headers.h
+	@rm -f $(PROGS) $(LIB_CHECKPOINT) $(OTHER) *~ *.o headers.h
 	@make -C test clean
restart.c +6 -0
index b1518f2..a163862 100644
@@ -530,6 +530,12 @@ int app_restart(struct restart_args *args)
 		ret = ckpt_coordinator(&ctx);
 	}
 
+	/*
+	 * Hack for LXC - which currently only uses pidns == 1, pid0 == 1;
+	 */
+	if (ret >= 0)
+		ret = global_child_pid;
+
 	return ret;
 }
 
Signature · 1 line
1.6.6.1

A message that is not in the archive (sent off-list or not yet received): 20100224083726.GF18758-r/[email protected]

#11 lore raw

Re: [PATCH 4/4][user-cr] Rename libeclone.a to libcheckpoint.a

Quoted 117 lines · Sukadev Bhattiprolu
Quoted 2 lines

From c4063a8976fd8eca9b8d62a12b95c3125c8471c7 Mon Sep 17 00:00:00 2001

From: Sukadev Bhattiprolu <[email protected]>
Date: Tue, 23 Feb 2010 16:25:36 -0800
Subject: [PATCH 4/4][user-cr] Rename libeclone.a to libcheckpoint.a

Export app_restart() and usercr.h to user in a new library libcheckpoint.a. Until eclone() makes it into glibc, include eclone() also in this new library and remove the libeclone.a.

TODO:
- cr-tests should be modified to look for libcheckpoint.a instead of libeclone.a

- libcheckpoint.a should eventually include the app_checkpoint() interface. I am guessing extracting the app_checkpoint() from checkpoint.c would be similar to (or somewhat easier than) the way we extracted app_restart() from restart.c

---
 Makefile  |   33 ++++++++++++++++++++-------------
 restart.c |    6 ++++++
 2 files changed, 26 insertions(+), 13 deletions(-)
Makefile +20 -13
index 4449081..090914a 100644
@@ -18,14 +18,14 @@ WARNS := -Wall -Wstrict-prototypes -Wno-trigraphs
 CFLAGS += -g $(WARNS) $(CKPT_INCLUDE) $(DEBUG)
 
 # install dir
+INC_INSTALL_DIR = /usr/include
 BIN_INSTALL_DIR = /bin
 LIB_INSTALL_DIR = /lib
 
-restart: restart.o restart-main.o
-
 ECLONE_PROGS = restart nsexec
 PROGS =	checkpoint ckptinfo $(ECLONE_PROGS)
-LIB_ECLONE = libeclone.a
+LIB_CHECKPOINT = libcheckpoint.a
+INC_CHECKPOINT = usercr.h
 
 # other cleanup
 OTHER = ckptinfo_types.c
@@ -34,32 +34,37 @@ LDLIBS = -lm
 
 .PHONY: all distclean clean headers install
 
-all: $(PROGS)
+all: $(PROGS) $(LIB_CHECKPOINT)
 	@make -C test
 
-$(LIB_ECLONE):
-	ar ruv $(LIB_ECLONE) $^
+restart: restart-main.o
+
+$(PROGS): $(LIB_CHECKPOINT) 
+
+$(LIB_CHECKPOINT):
+	ar ruv $(LIB_CHECKPOINT) $^
 
 # restart needs to be thread-safe
 restart: CFLAGS += -D__REENTRANT -pthread
 
+$(LIB_CHECKPOINT): restart.o
+
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
-$(ECLONE_PROGS): $(LIB_ECLONE) 
 $(ECLONE_PROGS): CFLAGS += -DARCH_HAS_ECLONE
-$(LIB_ECLONE): clone_$(SUBARCH).o genstack.o
+$(LIB_CHECKPOINT): clone_$(SUBARCH).o genstack.o
 endif
 
 # on powerpc, need also assembly file
 ifeq ($(SUBARCH),ppc)
 CFLAGS += -m32
 ASFLAGS += -m32
-$(LIB_ECLONE): clone_$(SUBARCH)_.o
+$(LIB_CHECKPOINT): clone_$(SUBARCH)_.o
 endif
 ifeq ($(SUBARCH),ppc64)
 CFLAGS += -m64
 ASFLAGS += -m64
-$(LIB_ECLONE): clone_$(SUBARCH)_.o
+$(LIB_CHECKPOINT): clone_$(SUBARCH)_.o
 endif
 
 # ckptinfo dependencies
@@ -71,8 +76,10 @@ ckptinfo_types.c: $(CKPT_HEADERS) ckptinfo.py
 install:
 	@echo /usr/bin/install -m 755 checkpoint restart nsexec ckptinfo $(BIN_INSTALL_DIR)
 	@/usr/bin/install -m 755 checkpoint restart ckptinfo nsexec $(BIN_INSTALL_DIR)
-	@echo /usr/bin/install -m 755 $(LIB_ECLONE) $(LIB_INSTALL_DIR)
-	@/usr/bin/install -m 755 $(LIB_ECLONE) $(LIB_INSTALL_DIR)
+	@echo /usr/bin/install -m 755 $(INC_CHECKPOINT) $(INC_INSTALL_DIR)
+	@/usr/bin/install -m 755 $(INC_CHECKPOINT) $(INC_INSTALL_DIR)
+	@echo /usr/bin/install -m 755 $(LIB_CHECKPOINT) $(LIB_INSTALL_DIR)
+	@/usr/bin/install -m 755 $(LIB_CHECKPOINT) $(LIB_INSTALL_DIR)
 
 $(CKPT_HEADERS): %:
 	./scripts/extract-headers.sh -s $(KERNELSRC) -o ./include
@@ -83,5 +90,5 @@ distclean: clean
 	@rm -f $(CKPT_HEADERS)
 
 clean:
-	@rm -f $(PROGS) $(LIB_ECLONE) $(OTHER) *~ *.o headers.h
+	@rm -f $(PROGS) $(LIB_CHECKPOINT) $(OTHER) *~ *.o headers.h
 	@make -C test clean
restart.c +0 -0
index b1518f2..a163862 100644

restart.c int app_restart(struct restart_args *args) · line 538

Quoted 9 lines
restart.c +6 -0
@@ -530,6 +530,12 @@ int app_restart(struct restart_args *args)
 		ret = ckpt_coordinator(&ctx);
 	}
 
+	/*
+	 * Hack for LXC - which currently only uses pidns == 1, pid0 == 1;
+	 */
+	if (ret >= 0)
+		ret = global_child_pid;
+

Hmm? Maybe I'm looking at the wrong context, but isn't this specifically the existing pidns case? Should the second block ("new pidns with init") be the one to execute if your comment is correct?

Quoted 5 lines
restart.c +0 -0
 	return ret;
 }
 
Signature · 1 line
1.6.6.1

A message that is not in the archive (sent off-list or not yet received): 20100224151919.GB6425-r/[email protected]

#13 lore raw

Re: [PATCH 4/4][user-cr] Rename libeclone.a to libcheckpoint.a

Serge E. Hallyn [serue-r/[email protected]] wrote:

| > diff --git a/restart.c b/restart.c
| > index b1518f2..a163862 100644
| > --- a/restart.c
| > +++ b/restart.c
| > @@ -530,6 +530,12 @@ int app_restart(struct restart_args *args)
| >  		ret = ckpt_coordinator(&ctx);
| >  	}
| > 
| > +	/*
| > +	 * Hack for LXC - which currently only uses pidns == 1, pid0 == 1;
| > +	 */
| > +	if (ret >= 0)
| > +		ret = global_child_pid;
| > +
| 
| Hmm?  Maybe I'm looking at the wrong context, but isn't this specifically
| the existing pidns case?

Yes it is.

| Should the second block ("new pidns with init") | be the one to execute if your comment is correct?

Oh, you mean I could just return the global_child_pid from within the "new pid ns with init" block ?. I could do that for now.

BTW, from a long-term API pov, should app_restart():

- support the other two types of restart ("new pidns without init" and "subtree in current pidns"). LXC may not use those, but any reason for the API to not support those cases ?

- If the API supports them, should app_restart() always return the pid of the coordinator (i.e global_child_pid). That way the caller knows the root of the restarted process-tree ?

- Also, like the kernel's API, how about app_checkpoint() and app_restart() take a logfp/logfd and we send all the debugging output to that file ?

Thanks,

Sukadev

| 
| >  	return ret;
| >  }
| > 
| > -- 
| > 1.6.6.1
#14 lore raw

Re: [PATCH 4/4][user-cr] Rename libeclone.a to libcheckpoint.a

In reply to a message not in the archive
Quoted 8 lines · Sukadev Bhattiprolu

From c4063a8976fd8eca9b8d62a12b95c3125c8471c7 Mon Sep 17 00:00:00 2001
From: Sukadev Bhattiprolu <[email protected]>
Date: Tue, 23 Feb 2010 16:25:36 -0800
Subject: [PATCH 4/4][user-cr] Rename libeclone.a to libcheckpoint.a

Export app_restart() and usercr.h to user in a new library libcheckpoint.a. Until eclone() makes it into glibc, include eclone() also in this new library and remove the libeclone.a.

I thought libeclone.a is also used by the test suite ?

Oren.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#16 lore raw

Re: [PATCH 4/4][user-cr] Rename libeclone.a to libcheckpoint.a

Quoted 8 lines · Oren Laadan [[email protected]] wrote:
Quoted 5 lines · Sukadev Bhattiprolu

From c4063a8976fd8eca9b8d62a12b95c3125c8471c7 Mon Sep 17 00:00:00 2001
From: Sukadev Bhattiprolu <[email protected]>
Date: Tue, 23 Feb 2010 16:25:36 -0800
Subject: [PATCH 4/4][user-cr] Rename libeclone.a to libcheckpoint.a

Quoted 5 lines
Quoted 3 lines

Export app_restart() and usercr.h to user in a new library libcheckpoint.a. Until eclone() makes it into glibc, include eclone() also in this new library and remove the libeclone.a.

I thought libeclone.a is also used by the test suite ?

Yes and I have a todo in the patchset to fix cr-tests if/when we do end up changing the name.

libeclone.a looks like a short-term name (if/when eclone() is merged in libc, the library is no longer required).

libcheckpoint.a (or equivalent) has more long term need. If we are having a libcheckpoint, no point keeping libeclone around too ?

Sukadev

#17 lore raw
In reply to a message not in the archive
Quoted 31 lines · Sukadev Bhattiprolu

Following two sets of patches is an early attempt to integrate LXC and USER-CR.

Overview:

Have USER-CR export the core checkpoint and restart functionality into a library (/lib/libcheckpoint.a and <usercr.h>) and have LXC link with this library.

TODO:

1. For now, libcheckpoint.a implements only the restart functionality and so only lxc_restart command is implemented. Implementing the checkpoint functionality and lxc_checkpoint command can be done similarly and is hopefully easier than the restart functionality.

2. The restart() functionality in user-cr makes extensive use of global variables and debug code. The API must be extended to properly include these variables/debug code in the API.

Similarly, the 'struct restart_args' may need to be sanitized for use in a formal API.

3. lxc_restart command restarts entire containers only (specifically it simulates the --pidns --pids --mount-pty arguments to /bin/restart).

4. Link lxc_restart and lxc_checkpoint with the shared library liblxc.so (currently links statically)

Quoted 6 lines

5. ...

STATUS:
I was able to checkpoint/restart a simple '/bin/sleep 1000' LXC container, except for a cgroup naming issue after restart (see below).

Very nice, Suka.

-serge

#18 lore raw
In reply to a message not in the archive
Quoted 9 lines · Sukadev Bhattiprolu

Following two sets of patches is an early attempt to integrate LXC and USER-CR.

Overview:

Have USER-CR export the core checkpoint and restart functionality into a library (/lib/libcheckpoint.a and<usercr.h>) and have LXC link with this library.

suka,

if these patches were accepted in the user-cr git tree, it would be easier for us to follow with your experiments and move forward on some more tests.

cheers,

C.

#19 lore raw
In reply to a message not in the archive

Suka,

This is good stuff: convert restart to a library and intergate with LXC is excellent target.

We need to give a thought to the API design and the sort of functionality that the library will offer. I'd prefer to get a better view of the whole plan before committing to any incremental step.

For example:

* what prefix will the library calls have ? e.g. cr_checkpoint(), cr_restart() etc. Could also be checkpoint_... or ckpt_... In any case, must be consistent and unique, and also agree with the library name: libcheckpoint.a and usercr.h doesn't make sense to me.

* what arguments are really necessary in struct restart_args ? for example, a ->logfd is necessary, but ->logfile (pathname) is not (the caller should already open the file)

* the library should provide an api to initialize the default args (e.g. the logfd should be -1 by default), e.g. cr_init_restart_args()

* similarly, only constants and macros relevant to a caller should be exposed, not internal data structures or macros.

* verbosity, debugging, warn and fail: currently that can be configurable to some extent; but we should never impose on the caller a perror() - instead the caller should pass FILE * for stdout and stderr (if NULL - the library will keep silent).

* probably makes sense to add interface to freeze a process tree identified by its root pid, and thaw a process tree (or a cgroup).

* does the mnt-ns manipulation belong to cr_restart() ? perhaps make it a separate api in the library ?

* we need to change the way restart detects errors and cleans up from relying on signals to, e.g. on pipes. I've been wanting to do this for the longest time, and it will eliminate most (if not all) of the global variables.

Oren.

Quoted 105 lines · Sukadev Bhattiprolu

Following two sets of patches is an early attempt to integrate LXC and USER-CR.

Overview:

Have USER-CR export the core checkpoint and restart functionality into a library (/lib/libcheckpoint.a and <usercr.h>) and have LXC link with this library.

TODO:

1. For now, libcheckpoint.a implements only the restart functionality and so only lxc_restart command is implemented. Implementing the checkpoint functionality and lxc_checkpoint command can be done similarly and is hopefully easier than the restart functionality.

2. The restart() functionality in user-cr makes extensive use of global variables and debug code. The API must be extended to properly include these variables/debug code in the API.

Similarly, the 'struct restart_args' may need to be sanitized for use in a formal API.

3. lxc_restart command restarts entire containers only (specifically it simulates the --pidns --pids --mount-pty arguments to /bin/restart).

4. Link lxc_restart and lxc_checkpoint with the shared library liblxc.so (currently links statically)

5. ...

STATUS:
I was able to checkpoint/restart a simple '/bin/sleep 1000' LXC container, except for a cgroup naming issue after restart (see below).

STEPS:

1. [USER-CR] Build/install /lib/libcheckpoint.a, /usr/include/usercr.h

1.1 Apply the attached [user-cr] patches to the user-cr git tree
(I tested with following commit as base)

commit 67cfee9329670ab28eb1a52e94745252b614718f
Author: Oren Laadan <[email protected]>
Date: Mon Feb 22 18:00:06 2010 -0500

1.2 Build/install user-cr binaries/libraries/includes

   	$ make all
	$ make install

This should install /lib/libcheckpoint.a and /usr/include/usercr.h

2. [LXC] Build lxc_restart using USER-CR API (usercr.h, libcheckpoint.a)

2.1 Apply attached [lxc] patches to Daniel Lezcano's lxc.git tree (0.6.5)

2.2 Build lxc_restart (this uses static linking for now)

   	$ make -f Makefile2 lxc_restart

3. Create and checkpoint a simple LXC container

	$ lxc-execute --name foo --rcfile lxc-macvlan.conf -- /bin/sleep 1000
	$ lxc-freeze --name foo
	TODO: 
		lxc_checkpoint --name foo should checkpoint the container,
		For now, use "lxc-ps --name foo" to find pid of lxc-init and
		checkpoint using:

		$ /bin/checkpoint --output=/tmp/sleep.ckpt <pid-of-lxc-init>

	$ lxc-unfreeze --name foo

	$ lxc-stop --name foo

4. Restart a checkpointed LXC container

	$ ./lxc_restart --statefile /tmp/sleep.ckpt --name bar

# Test some common lxc commands after restart

	$ lxc-ps --name "bar/1"
	CONTAINER    PID TTY          TIME CMD
	bar/1       8511 ?        00:00:00 lxc-init
	bar/1       8512 ?        00:00:00 sleep
	$ lxc-freeze --name "bar/1"
	$ grep State /proc/8511/status 
	State:	D (disk sleep)
	$ grep State /proc/8512/status 
	State:	D (disk sleep)

NOTE: For some reason, the container name after restart is "bar/1"
instead of "bar". Due to this, when the lxc_restart is exiting, I get a "-EBUSY - failed to remove "/cgroup/bar" error. I need to fix this still.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#21 lore raw
Quoted 10 lines · Oren Laadan [[email protected]] wrote:

Suka,

This is good stuff: convert restart to a library and intergate with LXC is excellent target.

We need to give a thought to the API design and the sort of functionality that the library will offer. I'd prefer to get a better view of the whole plan before committing to any incremental step.

Quoted 7 lines

For example:

* what prefix will the library calls have ? e.g. cr_checkpoint(), cr_restart() etc. Could also be checkpoint_... or ckpt_... In any case, must be consistent and unique, and also agree with the library name: libcheckpoint.a and usercr.h doesn't make sense to me.

Agree. I have been worried about publishing an interface and getting stuck with a bad interface. But I have also been trying to prototype interface with LIBLXC available so did not yet spend lot of time on the interface.

I have been meaning to add a disclaimer to usercr.h that the structures will change without regard to backward compatibility

libcheckpoint.a looked like a reasonable name but for the header, we already have a checkpoint.h from the kernel :-) Open to new/better names/prefixes for the library calls.

Anyway, these names are just for the prototype and to start a discussion, My guess is that we are not ready to publish a formal interface at this point, but we can start moving towards that.

My current pass has been to eliminate all global references in app_restart() and app_checkpoint() (like perror(), stderr, stdout etc). while preserving the current behavior. Once we have an interface that LIBLXC can use, maybe we could focus more on stabilizing the interface ? It seemed to hard to do that with all the globals in the code.

Quoted 4 lines

* what arguments are really necessary in struct restart_args ? for example, a ->logfd is necessary, but ->logfile (pathname) is not (the caller should already open the file)

I have several small patches to clean this up. Was hoping to post later today. They remove ->logfile, ->input, ->output fromt he structures and just leave the relevant fds.

Quoted 11 lines

* the library should provide an api to initialize the default args (e.g. the logfd should be -1 by default), e.g. cr_init_restart_args()

* similarly, only constants and macros relevant to a caller should be exposed, not internal data structures or macros.

* verbosity, debugging, warn and fail: currently that can be configurable to some extent; but we should never impose on the caller a perror() - instead the caller should pass FILE * for stdout and stderr (if NULL - the library will keep silent).

Yes, my current patchset uses a restart_args->ulogfd for the user-space error messages. The restart-main.c sets this field to fileno(stderr).

Quoted 3 lines

* probably makes sense to add interface to freeze a process tree identified by its root pid, and thaw a process tree (or a cgroup).

True. For the near-term, LIBLXC has a way to freeze/thaw the containers

Quoted 3 lines

* does the mnt-ns manipulation belong to cr_restart() ? perhaps make it a separate api in the library ?

Good point. We need to think about that.

Quoted 5 lines

* we need to change the way restart detects errors and cleans up from relying on signals to, e.g. on pipes. I've been wanting to do this for the longest time, and it will eliminate most (if not all) of the global variables.

Ok.

A message that is not in the archive (sent off-list or not yet received): 20100227001002.GA22965-r/[email protected]

#23 lore raw
Quoted 24 lines · Sukadev Bhattiprolu
Quoted 10 lines · Oren Laadan [[email protected]] wrote:

Suka,

This is good stuff: convert restart to a library and intergate with LXC is excellent target.

We need to give a thought to the API design and the sort of functionality that the library will offer. I'd prefer to get a better view of the whole plan before committing to any incremental step.

Quoted 7 lines

For example:

* what prefix will the library calls have ? e.g. cr_checkpoint(), cr_restart() etc. Could also be checkpoint_... or ckpt_... In any case, must be consistent and unique, and also agree with the library name: libcheckpoint.a and usercr.h doesn't make sense to me.

Agree. I have been worried about publishing an interface and getting stuck with a bad interface. But I have also been trying to prototype interface with LIBLXC available so did not yet spend lot of time on the interface.

Quoted 6 lines

I have been meaning to add a disclaimer to usercr.h that the structures will change without regard to backward compatibility

libcheckpoint.a looked like a reasonable name but for the header, we already have a checkpoint.h from the kernel :-) Open to new/better names/prefixes for the library calls.

we have sys/checkpoint.h, so it makes sense to have checkpoint.h :)

Quoted 12 lines

Anyway, these names are just for the prototype and to start a discussion, My guess is that we are not ready to publish a formal interface at this point, but we can start moving towards that.

My current pass has been to eliminate all global references in app_restart() and app_checkpoint() (like perror(), stderr, stdout etc). while preserving the current behavior. Once we have an interface that LIBLXC can use, maybe we could focus more on stabilizing the interface ? It seemed to hard to do that with all the globals in the code.

Quoted 7 lines
Quoted 3 lines

* what arguments are really necessary in struct restart_args ? for example, a ->logfd is necessary, but ->logfile (pathname) is not (the caller should already open the file)

I have several small patches to clean this up. Was hoping to post later today. They remove ->logfile, ->input, ->output fromt he structures and just leave the relevant fds.

What I suggest is to avoid unnecessary work and fixes, it's worth our while to discuss what could be the api and functionality even before getting into the implementation details.etc.

Also, two important points that I had meant to mention are:

* versioning: we should provision for api extensions, e.g. in the struct restart_args, or at the very least a way to figure out the version.

* hooks / plugins: allow the user to request hooks or plugins, for example after checkpoint completes but before applications are thawed, or in restart after the tree is created but before tasks call sys_restart(). another example - handling the mount- namespace (and remounting the pty) could be a plugin.

Oren.

Quoted 33 lines
Quoted 11 lines

* the library should provide an api to initialize the default args (e.g. the logfd should be -1 by default), e.g. cr_init_restart_args()

* similarly, only constants and macros relevant to a caller should be exposed, not internal data structures or macros.

* verbosity, debugging, warn and fail: currently that can be configurable to some extent; but we should never impose on the caller a perror() - instead the caller should pass FILE * for stdout and stderr (if NULL - the library will keep silent).

Yes, my current patchset uses a restart_args->ulogfd for the user-space error messages. The restart-main.c sets this field to fileno(stderr).

Quoted 2 lines

* probably makes sense to add interface to freeze a process tree identified by its root pid, and thaw a process tree (or a cgroup).

True. For the near-term, LIBLXC has a way to freeze/thaw the containers

Quoted 2 lines

* does the mnt-ns manipulation belong to cr_restart() ? perhaps make it a separate api in the library ?

Good point. We need to think about that.

Quoted 4 lines

* we need to change the way restart detects errors and cleans up from relying on signals to, e.g. on pipes. I've been wanting to do this for the longest time, and it will eliminate most (if not all) of the global variables.

Ok.

#24 lore raw
In reply to a message not in the archive

Suka,

To make these patches available for those who want to try out lxc + c/r, I pulled them to a separate branch: ckpt-v19-suka.

(The last two did not apply cleanly because of recent changes to the Makefile so I merged them manually - let me know if it breaks).

Oren

Quoted 105 lines · Sukadev Bhattiprolu

Following two sets of patches is an early attempt to integrate LXC and USER-CR.

Overview:

Have USER-CR export the core checkpoint and restart functionality into a library (/lib/libcheckpoint.a and <usercr.h>) and have LXC link with this library.

TODO:

1. For now, libcheckpoint.a implements only the restart functionality and so only lxc_restart command is implemented. Implementing the checkpoint functionality and lxc_checkpoint command can be done similarly and is hopefully easier than the restart functionality.

2. The restart() functionality in user-cr makes extensive use of global variables and debug code. The API must be extended to properly include these variables/debug code in the API.

Similarly, the 'struct restart_args' may need to be sanitized for use in a formal API.

3. lxc_restart command restarts entire containers only (specifically it simulates the --pidns --pids --mount-pty arguments to /bin/restart).

4. Link lxc_restart and lxc_checkpoint with the shared library liblxc.so (currently links statically)

5. ...

STATUS:
I was able to checkpoint/restart a simple '/bin/sleep 1000' LXC container, except for a cgroup naming issue after restart (see below).

STEPS:

1. [USER-CR] Build/install /lib/libcheckpoint.a, /usr/include/usercr.h

1.1 Apply the attached [user-cr] patches to the user-cr git tree
(I tested with following commit as base)

commit 67cfee9329670ab28eb1a52e94745252b614718f
Author: Oren Laadan <[email protected]>
Date: Mon Feb 22 18:00:06 2010 -0500

1.2 Build/install user-cr binaries/libraries/includes

   	$ make all
	$ make install

This should install /lib/libcheckpoint.a and /usr/include/usercr.h

2. [LXC] Build lxc_restart using USER-CR API (usercr.h, libcheckpoint.a)

2.1 Apply attached [lxc] patches to Daniel Lezcano's lxc.git tree (0.6.5)

2.2 Build lxc_restart (this uses static linking for now)

   	$ make -f Makefile2 lxc_restart

3. Create and checkpoint a simple LXC container

	$ lxc-execute --name foo --rcfile lxc-macvlan.conf -- /bin/sleep 1000
	$ lxc-freeze --name foo
	TODO: 
		lxc_checkpoint --name foo should checkpoint the container,
		For now, use "lxc-ps --name foo" to find pid of lxc-init and
		checkpoint using:

		$ /bin/checkpoint --output=/tmp/sleep.ckpt <pid-of-lxc-init>

	$ lxc-unfreeze --name foo

	$ lxc-stop --name foo

4. Restart a checkpointed LXC container

	$ ./lxc_restart --statefile /tmp/sleep.ckpt --name bar

# Test some common lxc commands after restart

	$ lxc-ps --name "bar/1"
	CONTAINER    PID TTY          TIME CMD
	bar/1       8511 ?        00:00:00 lxc-init
	bar/1       8512 ?        00:00:00 sleep
	$ lxc-freeze --name "bar/1"
	$ grep State /proc/8511/status 
	State:	D (disk sleep)
	$ grep State /proc/8512/status 
	State:	D (disk sleep)

NOTE: For some reason, the container name after restart is "bar/1"
instead of "bar". Due to this, when the lxc_restart is exiting, I get a "-EBUSY - failed to remove "/cgroup/bar" error. I need to fix this still.