Skip to contentSkip to conversation

Enable linking with LIBLXC

PATCH In review Sukadev Bhattiprolu 3 participants 18 messages Mar 19, 2010 – Mar 30, 2010 2 messages not in the archive Kbuild

lore
Share & tools
lore
#0 lore raw

cover Enable linking with LIBLXC

Patches to enable linking LIBLXC with USERCR.

Patches 1 through 9 are simply rebased patches due to the dropping of one of the earlier patches (Oren had nacked one patch in my earlier set. See

https://lists.linux-foundation.org/pipermail/containers/2010-March/023564.html

Patches 10 through 14 are additional changes needed to get LXC checkpoint and restart to work.

[PATCH 01/14] Add app_restart_args->debug [PATCH 02/14] Add app_restart_args->verbose [PATCH 03/14] Add app_restart_args->ulogfd [PATCH 04/14] Add app_restart_args->uerrfd [PATCH 05/14] Define INIT_SIGNAL_ARRAY
[PATCH 06/14] Create common.h
[PATCH 07/14] Create app-checkpoint.h
[PATCH 08/14] restart: Move main() to restart-main.c [PATCH 09/14] checkpoint: Move main() to checkpoint-main.c

[PATCH 10/14] Have app_restart() return pid [PATCH 11/14] restart: Define process_args() [PATCH 12/14] app_restart(): mnt-pty implies mntns [PATCH 13/14] restart: Move args checking to app_restart() [PATCH 14/14] Minimize unshare() calls

With these patches, USERCR exports app_checkpoint() and app_restart() functions in checkpoint.o and restart.o. LXC links directly with these .o files to checkpoint/restart its containers.

A message that is not in the archive (sent off-list or not yet received): 20100319062659.GA23838-r/[email protected]

#2 lore raw

PATCH 1/14 Add app_restart_args->debug

From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 17:46:31 -0800
Subject: [PATCH 01/14][user-cr] Add app_restart_args->debug

Make 'debug' a field in struct app_restart_args so other callers of app_restart() can also control the debug output of app_restart().

---
 restart.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
restart.c +4 -1
index 0c74bb6..365b7ad 100644
@@ -386,6 +386,7 @@ struct app_restart_args {
 	int infd;
 	int klogfd;
 	long warn;
+	int debug;
 	long fail;
 	int keep_lsm;
 };
@@ -581,7 +582,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 			args->copy_status = 1;
 			break;
 		case 'd':
-			global_debug = 1;
+			global_debug = args->debug = 1;
 			break;
 		case 'F':
 			args->freezer = optarg;
@@ -815,6 +816,8 @@ int app_restart(struct app_restart_args *args)
 	memset(&ctx, 0, sizeof(ctx));
 	ctx.args = args;
 
+	global_debug = args->debug;
+
 	/* input file descriptor (default: stdin) */
 	if (args->infd >= 0) {
 		if (dup2(args->infd, STDIN_FILENO) < 0) {
Signature · 1 line
1.6.0.4
#3 lore raw

PATCH 2/14 Add app_restart_args->verbose

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 17:53:27 -0800
Subject: [PATCH 02/14][user-cr] Add app_restart_args->verbose

Make 'verbose' a field in struct app_restart_args so other callers of app_restart() can also control the verbose output of app_restart().

---
 restart.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
restart.c +3 -1
index 365b7ad..9d16a4d 100644
@@ -387,6 +387,7 @@ struct app_restart_args {
 	int klogfd;
 	long warn;
 	int debug;
+	int verbose;
 	long fail;
 	int keep_lsm;
 };
@@ -513,7 +514,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 		case 'h':
 			usage(usage_str);
 		case 'v':
-			global_verbose = 1;
+			global_verbose = args->verbose = 1;
 			break;
 		case 5:  /* --inspect */
 			args->inspect = 1;
@@ -817,6 +818,7 @@ int app_restart(struct app_restart_args *args)
 	ctx.args = args;
 
 	global_debug = args->debug;
+	global_verbose = args->verbose;
 
 	/* input file descriptor (default: stdin) */
 	if (args->infd >= 0) {
Signature · 1 line
1.6.0.4
#4 lore raw

PATCH 3/14 Add app_restart_args->ulogfd

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 22:33:34 -0800
Subject: [PATCH 03/14][user-cr] Add app_restart_args->ulogfd

Add a 'ulogfd' field in struct app_restart_args so other callers of app_restart() can also choose a file descriptor for the log messages from app_restart().

---
 restart.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
restart.c +3 -0
index 9d16a4d..5ae5ec4 100644
@@ -385,6 +385,7 @@ struct app_restart_args {
 	char *freezer;
 	int infd;
 	int klogfd;
+	int ulogfd;
 	long warn;
 	int debug;
 	int verbose;
@@ -497,6 +498,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 	args->wait = 1;
 	args->infd = -1;
 	args->klogfd = -1;
+	args->ulogfd = fileno(stdout);
 	args->warn = CKPT_COND_WARN;
 	args->fail = CKPT_COND_FAIL;
 	no_pidns = 0;
@@ -819,6 +821,7 @@ int app_restart(struct app_restart_args *args)
 
 	global_debug = args->debug;
 	global_verbose = args->verbose;
+	global_ulogfd = args->ulogfd;
 
 	/* input file descriptor (default: stdin) */
 	if (args->infd >= 0) {
Signature · 1 line
1.6.0.4
#5 lore raw

PATCH 4/14 Add app_restart_args->uerrfd

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 22:36:31 -0800
Subject: [PATCH 04/14][user-cr] Add app_restart_args->uerrfd

Add a 'uerrfd' field in struct app_restart_args so other callers of app_restart() can also choose a file descriptor for the error messages from app_restart().

---
 restart.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
restart.c +3 -0
index 5ae5ec4..b526bbd 100644
@@ -386,6 +386,7 @@ struct app_restart_args {
 	int infd;
 	int klogfd;
 	int ulogfd;
+	int uerrfd;
 	long warn;
 	int debug;
 	int verbose;
@@ -499,6 +500,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 	args->infd = -1;
 	args->klogfd = -1;
 	args->ulogfd = fileno(stdout);
+	args->uerrfd = fileno(stderr);
 	args->warn = CKPT_COND_WARN;
 	args->fail = CKPT_COND_FAIL;
 	no_pidns = 0;
@@ -822,6 +824,7 @@ int app_restart(struct app_restart_args *args)
 	global_debug = args->debug;
 	global_verbose = args->verbose;
 	global_ulogfd = args->ulogfd;
+	global_uerrfd = args->uerrfd;
 
 	/* input file descriptor (default: stdin) */
 	if (args->infd >= 0) {
Signature · 1 line
1.6.0.4
#6 lore raw

PATCH 5/14 Define INIT_SIGNAL_ARRAY

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 21:15:25 -0800
Subject: [PATCH 05/14][user-cr] Define INIT_SIGNAL_ARRAY

Code to initialize the signal_array[] table will need to be shared by more than one file. To enable this sharing with, define INIT_SIGNAL_ARRAY which can later be used in many places.

---
 restart.c |   76 ++++++++++++++++++++++++++++++++-----------------------------
 1 files changed, 40 insertions(+), 36 deletions(-)
restart.c +40 -36
index b526bbd..4cd0bad 100644
@@ -157,46 +157,50 @@ static inline void ckpt_msg(int fd, char *format, ...)
 			ckpt_msg(global_ulogfd, __VA_ARGS__);	\
 	} while(0)
 
-#define SIGNAL_ENTRY(signal)  { SIG ## signal, #signal }
-
-struct {
+struct signal_array {
 	int signum;
 	char *sigstr;
-} signal_array[] = {
-	{ 0, "NONE" },
-	SIGNAL_ENTRY(ALRM),
-	SIGNAL_ENTRY(HUP),
-	SIGNAL_ENTRY(INT),
-	SIGNAL_ENTRY(KILL),
-	SIGNAL_ENTRY(PIPE),
-	SIGNAL_ENTRY(POLL),
-	SIGNAL_ENTRY(PROF),
-	SIGNAL_ENTRY(TERM),
-	SIGNAL_ENTRY(USR1),
-	SIGNAL_ENTRY(USR2),
-	SIGNAL_ENTRY(VTALRM),
-	SIGNAL_ENTRY(STKFLT),
-	SIGNAL_ENTRY(PWR),
-	SIGNAL_ENTRY(WINCH),
-	SIGNAL_ENTRY(CHLD),
-	SIGNAL_ENTRY(URG),
-	SIGNAL_ENTRY(TTIN),
-	SIGNAL_ENTRY(TTOU),
-	SIGNAL_ENTRY(STOP),
-	SIGNAL_ENTRY(CONT),
-	SIGNAL_ENTRY(ABRT),
-	SIGNAL_ENTRY(FPE),
-	SIGNAL_ENTRY(ILL),
-	SIGNAL_ENTRY(QUIT),
-	SIGNAL_ENTRY(SEGV),
-	SIGNAL_ENTRY(TRAP),
-	SIGNAL_ENTRY(SYS),
-	SIGNAL_ENTRY(BUS),
-	SIGNAL_ENTRY(XCPU),
-	SIGNAL_ENTRY(XFSZ),
-	{ -1, "LAST" },
 };
 
+#define SIGNAL_ENTRY(signal)  { SIG ## signal, #signal }
+
+#define INIT_SIGNAL_ARRAY { 	\
+	{ 0, "NONE" },		\
+	SIGNAL_ENTRY(ALRM),	\
+	SIGNAL_ENTRY(HUP),	\
+	SIGNAL_ENTRY(INT),	\
+	SIGNAL_ENTRY(KILL),	\
+	SIGNAL_ENTRY(PIPE),	\
+	SIGNAL_ENTRY(POLL),	\
+	SIGNAL_ENTRY(PROF),	\
+	SIGNAL_ENTRY(TERM),	\
+	SIGNAL_ENTRY(USR1),	\
+	SIGNAL_ENTRY(USR2),	\
+	SIGNAL_ENTRY(VTALRM),	\
+	SIGNAL_ENTRY(STKFLT),	\
+	SIGNAL_ENTRY(PWR),	\
+	SIGNAL_ENTRY(WINCH),	\
+	SIGNAL_ENTRY(CHLD),	\
+	SIGNAL_ENTRY(URG),	\
+	SIGNAL_ENTRY(TTIN),	\
+	SIGNAL_ENTRY(TTOU),	\
+	SIGNAL_ENTRY(STOP),	\
+	SIGNAL_ENTRY(CONT),	\
+	SIGNAL_ENTRY(ABRT),	\
+	SIGNAL_ENTRY(FPE),	\
+	SIGNAL_ENTRY(ILL),	\
+	SIGNAL_ENTRY(QUIT),	\
+	SIGNAL_ENTRY(SEGV),	\
+	SIGNAL_ENTRY(TRAP),	\
+	SIGNAL_ENTRY(SYS),	\
+	SIGNAL_ENTRY(BUS),	\
+	SIGNAL_ENTRY(XCPU),	\
+	SIGNAL_ENTRY(XFSZ),	\
+	{ -1, "LAST" },		\
+}
+
+static struct signal_array signal_array[] = INIT_SIGNAL_ARRAY;
+
 static char *sig2str(int sig)
 {
 	int i = 0;
Signature · 1 line
1.6.0.4
#7 lore raw

PATCH 6/14 Create common.h

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 21:30:50 -0800
Subject: [PATCH 06/14][user-cr] Create common.h

Code in common.h can be shared by checkpoint.c and restart.c for now.

---
 Makefile     |    4 ++
 checkpoint.c |   31 +---------------
 common.h     |  112 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 restart.c    |  110 +--------------------------------------------------------
 4 files changed, 119 insertions(+), 138 deletions(-)
 create mode 100644 common.h
Makefile +4 -0
index 64b5f73..406f685 100644
@@ -13,6 +13,8 @@ CKPT_HEADERS = include/linux/checkpoint.h \
 		include/linux/checkpoint_hdr.h \
 		include/asm/checkpoint_hdr.h
 
+CR_OBJS = checkpoint.o restart.o
+
 # detect architecture (for eclone)
 SUBARCH ?= $(patsubst i%86,x86_32,$(shell uname -m))
 
@@ -50,6 +52,8 @@ $(LIB_ECLONE):
 # restart needs to be thread-safe
 restart: CFLAGS += -D__REENTRANT -pthread
 
+$(CR_OBJS): common.h
+
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
 $(ECLONE_PROGS): $(LIB_ECLONE) 
checkpoint.c +2 -29
index 86608b1..6e91149 100644
@@ -21,6 +21,8 @@
 
 #include <linux/checkpoint.h>
 
+#include "common.h"
+
 static char usage_str[] =
 "usage: ckpt [opts] PID\n"
 "  'checkpoint' takes a checkpoint of the task indicated by PID, and all\n"
@@ -55,35 +57,6 @@ inline static int checkpoint(pid_t pid, int fd, unsigned long flags, int logfd)
 	return syscall(__NR_checkpoint, pid, fd, flags, logfd);
 }
 
-#define BUFSIZE  (4 * 4096)
-static inline void ckpt_msg(int fd, char *format, ...)
-{
-	va_list ap;
-	char *bufp;
-	if (fd < 0)
-		return;
-
-	va_start(ap, format);
-
-	bufp = malloc(BUFSIZE);
-	if(bufp) {
-		vsnprintf(bufp, BUFSIZE, format, ap);
-		write(fd, bufp, strlen(bufp));
-	}
-	free(bufp);
-
-	va_end(ap);
-}
-
-#define ckpt_err(...)				\
-	ckpt_msg(global_uerrfd, __VA_ARGS__)
-
-#define ckpt_perror(s)                                                  \
-	do {                                                            \
-		ckpt_msg(global_uerrfd, s);                             \
-		ckpt_msg(global_uerrfd, ": %s\n", strerror(errno));     \
-	} while (0)
-
 static void usage(char *str)
 {
 	ckpt_err("%s", str);
common.h new file +112 -0

112 lines not on this page · raw message

restart.c +1 -109

128 lines not on this page · raw message

Signature · 1 line
1.6.0.4
#8 lore raw

PATCH 7/14 Create app-checkpoint.h

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 21:51:27 -0800
Subject: [PATCH 07/14][user-cr] Create app-checkpoint.h

Create a new header file, app-checkpoint.h that can be used to export the app_checkpoint() and app_restart() functionality to other users (such as LXC).

For now, leave 'global_send_sigint' as an external variable. We should eventually implement an API that gives callers better control of how to handle signals during restart.

Changelog[v2]:
- (In response to Oren Laadan's comment): Leave 'global_send_sigint' as an extern and add a TODO to remove later.

---
 Makefile         |    2 +-
 app-checkpoint.h |   42 ++++++++++++++++++++++++++++++++++++++++++
 checkpoint.c     |    9 +--------
 restart.c        |   31 +++++++------------------------
 4 files changed, 51 insertions(+), 33 deletions(-)
 create mode 100644 app-checkpoint.h
Makefile +1 -1
index 406f685..639d42f 100644
@@ -52,7 +52,7 @@ $(LIB_ECLONE):
 # restart needs to be thread-safe
 restart: CFLAGS += -D__REENTRANT -pthread
 
-$(CR_OBJS): common.h
+$(CR_OBJS): common.h app-checkpoint.h
 
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
app-checkpoint.h new file +42 -0

42 lines not on this page · raw message

checkpoint.c +1 -8
index 6e91149..291cb36 100644
@@ -21,6 +21,7 @@
 
 #include <linux/checkpoint.h>
 
+#include "app-checkpoint.h"
 #include "common.h"
 
 static char usage_str[] =
@@ -44,14 +45,6 @@ static char usage_str[] =
 
 static int global_uerrfd = -1;
 
-struct app_checkpoint_args {
-	int outfd;
-	int logfd;
-	int uerrfd;
-	int container;
-	int verbose;
-};
-
 inline static int checkpoint(pid_t pid, int fd, unsigned long flags, int logfd)
 {
 	return syscall(__NR_checkpoint, pid, fd, flags, logfd);
restart.c +7 -24
index 7c9db80..bbc5ef3 100644
@@ -40,6 +40,7 @@
 #include "eclone.h"
 #include "genstack.h"
 #include "compat.h"
+#include "app-checkpoint.h"
 #include "common.h"
 
 static char usage_str[] =
@@ -216,9 +217,14 @@ static int global_verbose;
 static pid_t global_child_pid;
 static int global_child_status;
 static int global_child_collected;
-static int global_send_sigint = -1;
 static int global_sent_sigint;
 
+/*
+ * TODO: Implement an API to let callers choose if/how an interrupt be sent
+ * 	 and remove global_send_sigint.
+ */
+int global_send_sigint = -1;
+
 static int ckpt_remount_proc(struct ckpt_ctx *ctx);
 static int ckpt_remount_devpts(struct ckpt_ctx *ctx);
 
@@ -279,29 +285,6 @@ struct pid_swap {
 	pid_t new;
 };
 
-struct app_restart_args {
-	int self;
-	int pids;
-	int pidns;
-	int inspect;
-	char *root;
-	int wait;
-	int mntns;
-	int mnt_pty;
-	int show_status;
-	int copy_status;
-	char *freezer;
-	int infd;
-	int klogfd;
-	int ulogfd;
-	int uerrfd;
-	long warn;
-	int debug;
-	int verbose;
-	long fail;
-	int keep_lsm;
-};
-
 static void usage(char *str)
 {
 	ckpt_err("%s", str);
Signature · 1 line
1.6.0.4
#9 lore raw

PATCH 8/14 restart: Move main() to restart-main.c

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 21:46:14 -0800
Subject: [PATCH 08/14][user-cr] restart: Move main() to restart-main.c

---
 Makefile       |    4 +-
 restart-main.c |  350 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 restart.c      |  331 -----------------------------------------------------
 3 files changed, 353 insertions(+), 332 deletions(-)
 create mode 100644 restart-main.c
Makefile +3 -1
index 639d42f..7e1d701 100644
@@ -13,7 +13,7 @@ CKPT_HEADERS = include/linux/checkpoint.h \
 		include/linux/checkpoint_hdr.h \
 		include/asm/checkpoint_hdr.h
 
-CR_OBJS = checkpoint.o restart.o
+CR_OBJS = checkpoint.o restart.o restart-main.o
 
 # detect architecture (for eclone)
 SUBARCH ?= $(patsubst i%86,x86_32,$(shell uname -m))
@@ -54,6 +54,8 @@ restart: CFLAGS += -D__REENTRANT -pthread
 
 $(CR_OBJS): common.h app-checkpoint.h
 
+restart: restart.o restart-main.o
+
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
 $(ECLONE_PROGS): $(LIB_ECLONE) 
restart-main.c new file +350 -0

350 lines not on this page · raw message

restart.c +0 -331

361 lines not on this page · raw message

Signature · 1 line
1.6.0.4
#10 lore raw

PATCH 9/14 checkpoint: Move main() to checkpoint-main.c

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 22:28:34 -0800
Subject: [PATCH 09/14][user-cr] checkpoint: Move main() to checkpoint-main.c

---
 Makefile          |    4 +-
 checkpoint-main.c |  171 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 checkpoint.c      |  157 ------------------------------------------------
 3 files changed, 174 insertions(+), 158 deletions(-)
 create mode 100644 checkpoint-main.c
Makefile +3 -1
index 7e1d701..6c9ff93 100644
@@ -13,7 +13,7 @@ CKPT_HEADERS = include/linux/checkpoint.h \
 		include/linux/checkpoint_hdr.h \
 		include/asm/checkpoint_hdr.h
 
-CR_OBJS = checkpoint.o restart.o restart-main.o
+CR_OBJS = checkpoint.o checkpoint-main.o restart.o restart-main.o
 
 # detect architecture (for eclone)
 SUBARCH ?= $(patsubst i%86,x86_32,$(shell uname -m))
@@ -56,6 +56,8 @@ $(CR_OBJS): common.h app-checkpoint.h
 
 restart: restart.o restart-main.o
 
+checkpoint: checkpoint.o checkpoint-main.o
+
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
 $(ECLONE_PROGS): $(LIB_ECLONE) 
checkpoint-main.c new file +171 -0

171 lines not on this page · raw message

checkpoint.c +0 -157

172 lines not on this page · raw message

Signature · 1 line
1.6.0.4
#11 lore raw

PATCH 10/14 Have app_restart() return pid

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Fri, 5 Mar 2010 10:24:53 -0800
Subject: [PATCH 10/14][user-cr] Have app_restart() return pid

On success, return the pid of a the root of the restarted process tree. This would enable callers to manage the restarted process-tree as a cgroup.

---
 restart-main.c |    7 ++++++-
 restart.c      |    6 ++++++
 2 files changed, 12 insertions(+), 1 deletions(-)
restart-main.c +6 -1
index d3a1782..237742e 100644
@@ -332,6 +332,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 
 int main(int argc, char *argv[])
 {
+	int ret;
 	struct app_restart_args args;
 
 	/*
@@ -345,6 +346,10 @@ int main(int argc, char *argv[])
 
 	parse_args(&args, argc, argv);
 
-	return app_restart(&args);
+	ret = app_restart(&args);
+	if (ret > 0)
+		ret = 0;
+
+	return ret;
 }
 
restart.c +6 -0
index 0118ad9..b6ee23c 100644
@@ -490,6 +490,12 @@ int app_restart(struct app_restart_args *args)
 		ret = ckpt_coordinator(&ctx);
 	}
 
+	/*
+	 * On success, return pid of root of the restart process tree.
+	 */
+	if (ret >= 0)
+		ret = global_child_pid;
+
 	return ret;
 }
 
Signature · 1 line
1.6.0.4
#12 lore raw

PATCH 11/14 restart: Define process_args()

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Sat, 6 Mar 2010 13:14:02 -0800
Subject: [PATCH 11/14][user-cr] restart: Define process_args()

Move the code that validates/processes the arguments into a separate function.

---
 restart.c |   27 ++++++++++++++++++++-------
 1 files changed, 20 insertions(+), 7 deletions(-)
restart.c +20 -7
index b6ee23c..139f6db 100644
@@ -379,14 +379,12 @@ static int freezer_register(struct ckpt_ctx *ctx, pid_t pid)
 	return ret;
 }
 
-int app_restart(struct app_restart_args *args)
+/*
+ * Validate the specified arguments and initialize globals based on the
+ * arguments. Return 0 on success.
+ */
+int process_args(struct app_restart_args *args)
 {
-	struct ckpt_ctx ctx;
-	int ret;
-
-	memset(&ctx, 0, sizeof(ctx));
-	ctx.args = args;
-
 	global_debug = args->debug;
 	global_verbose = args->verbose;
 	global_ulogfd = args->ulogfd;
@@ -406,6 +404,21 @@ int app_restart(struct app_restart_args *args)
 	if (args->klogfd < 0)
 		args->klogfd = CHECKPOINT_FD_NONE;
 
+	return 0;
+}
+
+int app_restart(struct app_restart_args *args)
+{
+	struct ckpt_ctx ctx;
+	int ret;
+
+	memset(&ctx, 0, sizeof(ctx));
+	ctx.args = args;
+
+	ret = process_args(args);
+	if (ret < 0)
+		return ret;
+
 	/* freezer preparation */
 	if (args->freezer && freezer_prepare(&ctx) < 0)
 		exit(1);
Signature · 1 line
1.6.0.4
#13 lore raw

PATCH 12/14 app_restart(): mnt-pty implies mntns

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Sat, 6 Mar 2010 12:49:40 -0800
Subject: [PATCH 12/14][user-cr] app_restart(): mnt-pty implies mntns

If the mount_pty is set, mntns must be implied, otherwise the newinstance mount of devpts will make the ptys in the original pts instance inaccessible.

parse_args() does this correctly, but move the code into app_restart() so other callers of app_restart() also get this behavior.

---
 restart-main.c |    4 ----
 restart.c      |    3 +++
 2 files changed, 3 insertions(+), 4 deletions(-)
restart-main.c +0 -4
index 237742e..36318a4 100644
@@ -324,10 +324,6 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 			exit(1);
 		}
 	}
-
-
-	if (args->mnt_pty)
-		args->mntns = 1;
 }
 
 int main(int argc, char *argv[])
restart.c +3 -0
index 139f6db..d7b5b72 100644
@@ -404,6 +404,9 @@ int process_args(struct app_restart_args *args)
 	if (args->klogfd < 0)
 		args->klogfd = CHECKPOINT_FD_NONE;
 
+	if (args->mnt_pty)
+		args->mntns = 1;
+
 	return 0;
 }
 
Signature · 1 line
1.6.0.4
#14 lore raw

PATCH 13/14 restart: Move args checking to app_restart()

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Sat, 6 Mar 2010 13:06:35 -0800
Subject: [PATCH 13/14][user-cr] restart: Move args checking to app_restart()

parse_args() validates the specified arguments to ensure they are consistent. Move this code into app_restart() so that these checks apply to other callers of app_restart() also.

---
 restart-main.c |   33 +--------------------------------
 restart.c      |   36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 32 deletions(-)
restart-main.c +1 -32
index 36318a4..681ff8e 100644
@@ -261,38 +261,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 	if (no_pidns)
 		args->pidns = 0;
 
-#ifndef CLONE_NEWPID
-	if (args->pidns) {
-		ckpt_err("This version of restart was compiled without "
-		       "support for --pidns.\n");
-		exit(1);
-	}
-#endif
-
-#ifndef CHECKPOINT_DEBUG
-	if (global_debug) {
-		ckpt_err("This version of restart was compiled without "
-		       "support for --debug.\n");
-		exit(1);
-	}
-#endif
-
-	if (args->pidns)
-		args->pids = 1;
-
-#if 0   /* Defered until __NR_eclone makes it to standard headers */
-#ifndef __NR_eclone
-	if (args->pids) {
-		ckpt_err("This version of restart was compiled without "
-		       "support for --pids.\n");
-		exit(1);
-	}
-#endif
-#endif
-
-	if (args->self &&
-	    (args->pids || args->pidns || no_pidns ||
-	     args->show_status || args->copy_status || args->freezer)) {
+	if (args->self && no_pidns) {
 		ckpt_err("Invalid mix of --self with multiprocess options\n");
 		exit(1);
 	}
restart.c +36 -0
index d7b5b72..b2708c3 100644
@@ -407,6 +407,42 @@ int process_args(struct app_restart_args *args)
 	if (args->mnt_pty)
 		args->mntns = 1;
 
+#ifndef CLONE_NEWPID
+	if (args->pidns) {
+		ckpt_err("This version of restart was compiled without "
+		       "support for --pidns.\n");
+		exit(1);
+	}
+#endif
+
+#ifndef CHECKPOINT_DEBUG
+	if (global_debug) {
+		ckpt_err("This version of restart was compiled without "
+		       "support for --debug.\n");
+		exit(1);
+	}
+#endif
+
+	if (args->pidns)
+		args->pids = 1;
+
+#if 0   /* Defered until __NR_eclone makes it to standard headers */
+#ifndef __NR_eclone
+	if (args->pids) {
+		ckpt_err("This version of restart was compiled without "
+		       "support for --pids.\n");
+		exit(1);
+	}
+#endif
+#endif
+
+	if (args->self &&
+	    (args->pids || args->pidns || args->show_status ||
+	     args->copy_status || args->freezer)) {
+		ckpt_err("Invalid mix of --self with multiprocess options\n");
+		exit(1);
+	}
+
 	return 0;
 }
 
Signature · 1 line
1.6.0.4
#15 lore raw

PATCH 14/14 Minimize unshare() calls

In reply to a message not in the archive

From: Sukadev Bhattiprolu <[email protected]>
Date: Mon, 8 Mar 2010 12:03:46 -0800
Subject: [PATCH 14/14][user-cr] Minimize unshare() calls

We currently have a few unshare() calls at different points in the code. While these don't affect the restart application itself, the excess calls create additional levels in the cgroup hierarchy, which can surprise the administrator (or other users of the hierarchy such as LXC.

Rather than several unshare() calls, can we instead specify the appropriate clone_flags while creating the coordinator/root process of the application tree ? When this root process is created it can remount /proc, remount devpts, chroot() etc if necessary.

Note that for "new-container with init" and "subtree restart", the first process is also the root of the application process tree.

In the case of "new-container without init", the coordinator process which acts as the container-init can do the setup.

In case of self-restart, the main process itself can do the unshare.

This patch has been very gently tested :-) but wanted to get more feedback on the direction and see if there is an easier way.

Changelog[v2]:
- Remove rather than '#ifdef 0' some unshare() calls

---
 restart.c |   80 +++++++++++++++++++++++++++++++++++++++++++++---------------
 1 files changed, 60 insertions(+), 20 deletions(-)
restart.c +60 -20
index b2708c3..484668a 100644
@@ -127,6 +127,9 @@ struct task zero_task;
 #define TASK_SESSION	0x10	/* inherits creator's original sid */
 #define TASK_NEWPID	0x20	/* starts a new pid namespace */
 #define TASK_DEAD	0x40	/* dead task (dummy) */
+#define TASK_NEWROOT	0x80	/* task must chroot() */
+#define TASK_NEWPTS	0x100	/* remount devpts */
+#define TASK_NEWNS	0x200	/* unshare namespace/file-system */
 
 struct ckpt_ctx {
 	pid_t root_pid;
@@ -462,24 +465,24 @@ int app_restart(struct app_restart_args *args)
 	if (args->freezer && freezer_prepare(&ctx) < 0)
 		exit(1);
 
-	/* private mounts namespace ? */
-	if (args->mntns && unshare(CLONE_NEWNS | CLONE_FS) < 0) {
-		ckpt_perror("unshare");
-		exit(1);
-	}
+	/* self-restart ends here: */
+	if (args->self) {
+		/* private mounts namespace ? */
+		if (args->mntns && unshare(CLONE_NEWNS | CLONE_FS) < 0) {
+			ckpt_perror("unshare");
+			exit(1);
+		}
 
-	/* chroot ? */
-	if (args->root && chroot(args->root) < 0) {
-		ckpt_perror("chroot");
-		exit(1);
-	}
+		/* chroot ? */
+		if (args->root && chroot(args->root) < 0) {
+			ckpt_perror("chroot");
+			exit(1);
+		}
 
-	/* remount /dev/pts ? */
-	if (args->mnt_pty && ckpt_remount_devpts(&ctx) < 0)
-		exit(1);
+		/* remount /dev/pts ? */
+		if (args->mnt_pty && ckpt_remount_devpts(&ctx) < 0)
+			exit(1);
 
-	/* self-restart ends here: */
-	if (args->self) {
 		restart(getpid(), STDIN_FILENO, RESTART_TASKSELF, args->klogfd);
 		/* reach here if restart(2) failed ! */
 		ckpt_perror("restart");
@@ -524,10 +527,30 @@ int app_restart(struct app_restart_args *args)
 	if (ret < 0)
 		exit(1);
 
+	/*
+	 * Have the first child in the restarted process tree
+	 * setup devpts, root-dir and /proc if necessary, ...
+	 */
+	if (ctx.args->mnt_pty)
+		ctx.tasks_arr[0].flags |= TASK_NEWPTS;
+
+	if (ctx.args->mntns)
+		ctx.tasks_arr[0].flags |= TASK_NEWNS;
+
+	if (ctx.args->root)
+		ctx.tasks_arr[0].flags |= TASK_NEWROOT;
+
 	if (ctx.args->pidns && ctx.tasks_arr[0].pid != 1) {
 		ckpt_dbg("new pidns without init\n");
 		if (global_send_sigint == -1)
 			global_send_sigint = SIGINT;
+		/*
+		 * ...unless we have an explicit coordinator, in which case
+		 * the coordinator should set up the filesystems and
+		 * not the first process in the application process tree.
+		 */
+		ctx.tasks_arr[0].flags &= ~(TASK_NEWPTS | TASK_NEWROOT | \
+				TASK_NEWNS);
 		ret = ckpt_coordinator_pidns(&ctx);
 	} else if (ctx.args->pidns) {
 		ckpt_dbg("new pidns with init\n");
@@ -721,10 +744,6 @@ static int ckpt_probe_child(pid_t pid, char *str)
  */
 static int ckpt_remount_proc(struct ckpt_ctx *ctx)
 {
-	if (unshare(CLONE_NEWNS | CLONE_FS) < 0) {
-		ckpt_perror("unshare");
-		return -1;
-	}
 	/* this is unlikely, but we don't want to fail */
 	if (umount2("/proc", MNT_DETACH) < 0) {
 		if (ckpt_cond_fail(ctx, CKPT_COND_MNTPROC)) {
@@ -747,9 +766,18 @@ static int __ckpt_coordinator(void *arg)
 {
 	struct ckpt_ctx *ctx = (struct ckpt_ctx *) arg;
 
+	/* chroot ? */
+	if (ctx->args->root && chroot(ctx->args->root) < 0) {
+		ckpt_perror("chroot");
+		exit(1);
+	}
+
 	if (ckpt_remount_proc(ctx) < 0)
 		return -1;
 
+	if (ctx->args->mnt_pty && ckpt_remount_devpts(ctx) < 0)
+		return -1;
+
 	if (!ctx->args->wait)
 		close(ctx->pipe_coord[0]);
 
@@ -782,6 +810,7 @@ static int ckpt_coordinator_pidns(struct ckpt_ctx *ctx)
 	int copy, ret;
 	genstack stk;
 	void *sp;
+	unsigned long flags = SIGCHLD;
 
 	ckpt_dbg("forking coordinator in new pidns\n");
 
@@ -806,7 +835,9 @@ static int ckpt_coordinator_pidns(struct ckpt_ctx *ctx)
 	copy = ctx->args->copy_status;
 	ctx->args->copy_status = 1;
 
-	coord_pid = clone(__ckpt_coordinator, sp, CLONE_NEWPID|SIGCHLD, ctx);
+	flags |= CLONE_NEWPID|CLONE_NEWNS;
+
+	coord_pid = clone(__ckpt_coordinator, sp, flags, ctx);
 	genstack_release(stk);
 	if (coord_pid < 0) {
 		ckpt_perror("clone coordinator");
@@ -1614,10 +1645,16 @@ int ckpt_fork_stub(void *data)
 	struct task *task = (struct task *) data;
 	struct ckpt_ctx *ctx = task->ctx;
 
+	if ((task->flags & TASK_NEWROOT) && chroot(ctx->args->root) < 0)
+		return -1;
+
 	/* tasks with new pid-ns need new /proc mount */
 	if ((task->flags & TASK_NEWPID) && ckpt_remount_proc(ctx) < 0)
 		return -1;
 
+	if ((task->flags & TASK_NEWPTS) && ckpt_remount_devpts(ctx) < 0)
+		return -1;
+
 	/*
 	 * In restart into a new pid namespace (--pidns), coordinator
 	 * is the container init, hence if it terminated permatutely
@@ -1695,6 +1732,9 @@ static pid_t ckpt_fork_child(struct ckpt_ctx *ctx, struct task *child)
 		pid = 0;
 	}
 #endif
+	if (child->flags & TASK_NEWNS) {
+		flags |= CLONE_NEWNS;
+	}
 
 	if (child->flags & (TASK_SIBLING | TASK_THREAD))
 		child->real_parent = getppid();
Signature · 1 line
1.6.0.4

A message that is not in the archive (sent off-list or not yet received): 20100319063448.GN24844-r/[email protected]

#17 lore raw

Re: [PATCH 14/14][user-cr] Minimize unshare() calls

Quoted 19 lines · Sukadev Bhattiprolu

From: Sukadev Bhattiprolu <[email protected]>
Date: Mon, 8 Mar 2010 12:03:46 -0800
Subject: [PATCH 14/14][user-cr] Minimize unshare() calls

We currently have a few unshare() calls at different points in the code. While these don't affect the restart application itself, the excess calls create additional levels in the cgroup hierarchy, which can surprise the administrator (or other users of the hierarchy such as LXC.

Rather than several unshare() calls, can we instead specify the appropriate clone_flags while creating the coordinator/root process of the application tree ? When this root process is created it can remount /proc, remount devpts, chroot() etc if necessary.

Note that for "new-container with init" and "subtree restart", the first process is also the root of the application process tree.

Quoted 7 lines

In the case of "new-container without init", the coordinator process which acts as the container-init can do the setup.

In case of self-restart, the main process itself can do the unshare.

This patch has been very gently tested :-) but wanted to get more feedback on the direction and see if there is an easier way.

No objection from me on the direction or the code.

#18 lore raw

Re: [PATCH 14/14][user-cr] Minimize unshare() calls

In reply to a message not in the archive
Quoted 4 lines · Sukadev Bhattiprolu

From: Sukadev Bhattiprolu <[email protected]>
Date: Mon, 8 Mar 2010 12:03:46 -0800
Subject: [PATCH 14/14][user-cr] Minimize unshare() calls

Hmm, and yet, when I run cr_tests/simple.sh, restart apparently does not use CLONE_NEWNS, bc I then have to
umount /proc
mount -t proc proc /proc
to restore sanity.

Not sure why... restart -vd gives me

Using output dir ./cr_simple_N3wEpal
about to clone with 20020000
<3793>number of tasks: 1
<3793>total tasks (including ghosts): 1
<3793>====== TASKS
<3793> [0] pid 1 ppid 0 sid 0 creator 0
<3793>............
<3793>new pidns with init
<3793>forking child vpid 1 flags 0x21
<3793>forked child vpid 3795 (asked 1)
<1>root task pid 1
<1>pid 1: pid 1 sid 0 parent 0
<1>about to call sys_restart(), flags 0
<3794>====== PIDS ARRAY
<3794>[0] pid 1 ppid 0 sid 0 pgid 0
<3794>............
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 11161
<3794>c/r read input 0
Success
<3793>restart succeeded
<3793>SIGCHLD: already collected
<3793>task exited with status 0
<3793>c/r succeeded
PASS

Oh, there you go - I suppose restart.c:557 should have TASK_NEWNS added? Indeed that fixes that.

-serge

#19 lore raw
In reply to a message not in the archive

Suka,

Thanks for the patches - applied on ckpt-v20-dev.

I think it's time to look for a naming convention for the library and API ...

Oren

Quoted 32 lines · Sukadev Bhattiprolu

Patches to enable linking LIBLXC with USERCR.

Patches 1 through 9 are simply rebased patches due to the dropping of one of the earlier patches (Oren had nacked one patch in my earlier set. See

https://lists.linux-foundation.org/pipermail/containers/2010-March/023564.html

Patches 10 through 14 are additional changes needed to get LXC checkpoint and restart to work.

[PATCH 01/14] Add app_restart_args->debug [PATCH 02/14] Add app_restart_args->verbose [PATCH 03/14] Add app_restart_args->ulogfd [PATCH 04/14] Add app_restart_args->uerrfd [PATCH 05/14] Define INIT_SIGNAL_ARRAY
[PATCH 06/14] Create common.h
[PATCH 07/14] Create app-checkpoint.h
[PATCH 08/14] restart: Move main() to restart-main.c [PATCH 09/14] checkpoint: Move main() to checkpoint-main.c

[PATCH 10/14] Have app_restart() return pid [PATCH 11/14] restart: Define process_args() [PATCH 12/14] app_restart(): mnt-pty implies mntns [PATCH 13/14] restart: Move args checking to app_restart() [PATCH 14/14] Minimize unshare() calls

With these patches, USERCR exports app_checkpoint() and app_restart() functions in checkpoint.o and restart.o. LXC links directly with these .o files to checkpoint/restart its containers.