Enable linking with LIBLXC
cover Enable linking with LIBLXC
Patches to enable linking LIBLXC with USERCR.
Patches 1 through 9 are simply rebased patches due to the dropping of one of the earlier patches (Oren had nacked one patch in my earlier set. See
https://lists.linux-foundation.org/pipermail/containers/2010-March/023564.html
Patches 10 through 14 are additional changes needed to get LXC checkpoint and restart to work.
[PATCH 01/14] Add app_restart_args->debug [PATCH 02/14] Add app_restart_args->verbose [PATCH 03/14] Add app_restart_args->ulogfd [PATCH 04/14] Add app_restart_args->uerrfd [PATCH 05/14] Define INIT_SIGNAL_ARRAY
[PATCH 06/14] Create common.h
[PATCH 07/14] Create app-checkpoint.h
[PATCH 08/14] restart: Move main() to restart-main.c [PATCH 09/14] checkpoint: Move main() to checkpoint-main.c
[PATCH 10/14] Have app_restart() return pid [PATCH 11/14] restart: Define process_args() [PATCH 12/14] app_restart(): mnt-pty implies mntns [PATCH 13/14] restart: Move args checking to app_restart() [PATCH 14/14] Minimize unshare() calls
With these patches, USERCR exports app_checkpoint() and app_restart() functions in checkpoint.o and restart.o. LXC links directly with these .o files to checkpoint/restart its containers.
A message that is not in the archive (sent off-list or not yet received): 20100319062659.GA23838-r/[email protected]
PATCH 1/14 Add app_restart_args->debug
From: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 17:46:31 -0800
Subject: [PATCH 01/14][user-cr] Add app_restart_args->debug
Make 'debug' a field in struct app_restart_args so other callers of app_restart() can also control the debug output of app_restart().
--- restart.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
restart.c +4 -1
@@ -386,6 +386,7 @@ struct app_restart_args {int infd;int klogfd;long warn; +int debug;long fail;int keep_lsm; };
@@ -581,7 +582,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[]) args->copy_status =1 ;break ;case 'd' : - global_debug =1 ; + global_debug = args->debug =1 ;break ;case 'F' : args->freezer = optarg;
@@ -815,6 +816,8 @@ int app_restart(struct app_restart_args *args)memset (&ctx,0 ,sizeof (ctx)); ctx.args = args; + global_debug = args->debug; +/* input file descriptor (default: stdin) */ if (args->infd >=0 ) {if (dup2 (args->infd,STDIN_FILENO ) <0 ) {
Signature · 1 line
1.6.0.4
PATCH 2/14 Add app_restart_args->verbose
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 17:53:27 -0800
Subject: [PATCH 02/14][user-cr] Add app_restart_args->verbose
Make 'verbose' a field in struct app_restart_args so other callers of app_restart() can also control the verbose output of app_restart().
--- restart.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
restart.c +3 -1
@@ -387,6 +387,7 @@ struct app_restart_args {int klogfd;long warn;int debug; +int verbose;long fail;int keep_lsm; };
@@ -513,7 +514,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])case 'h' :usage (usage_str);case 'v' : - global_verbose =1 ; + global_verbose = args->verbose =1 ;break ;case 5 :/* --inspect */ args->inspect =1 ;
@@ -817,6 +818,7 @@ int app_restart(struct app_restart_args *args) ctx.args = args; global_debug = args->debug; + global_verbose = args->verbose;/* input file descriptor (default: stdin) */ if (args->infd >=0 ) {
Signature · 1 line
1.6.0.4
PATCH 3/14 Add app_restart_args->ulogfd
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 22:33:34 -0800
Subject: [PATCH 03/14][user-cr] Add app_restart_args->ulogfd
Add a 'ulogfd' field in struct app_restart_args so other callers of app_restart() can also choose a file descriptor for the log messages from app_restart().
--- restart.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
restart.c +3 -0
@@ -385,6 +385,7 @@ struct app_restart_args {char *freezer;int infd;int klogfd; +int ulogfd;long warn;int debug;int verbose;
@@ -497,6 +498,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[]) args->wait =1 ; args->infd =-1 ; args->klogfd =-1 ; + args->ulogfd =fileno (stdout); args->warn =CKPT_COND_WARN ; args->fail =CKPT_COND_FAIL ; no_pidns =0 ;
@@ -819,6 +821,7 @@ int app_restart(struct app_restart_args *args) global_debug = args->debug; global_verbose = args->verbose; + global_ulogfd = args->ulogfd;/* input file descriptor (default: stdin) */ if (args->infd >=0 ) {
Signature · 1 line
1.6.0.4
PATCH 4/14 Add app_restart_args->uerrfd
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 22:36:31 -0800
Subject: [PATCH 04/14][user-cr] Add app_restart_args->uerrfd
Add a 'uerrfd' field in struct app_restart_args so other callers of app_restart() can also choose a file descriptor for the error messages from app_restart().
--- restart.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
restart.c +3 -0
@@ -386,6 +386,7 @@ struct app_restart_args {int infd;int klogfd;int ulogfd; +int uerrfd;long warn;int debug;int verbose;
@@ -499,6 +500,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[]) args->infd =-1 ; args->klogfd =-1 ; args->ulogfd =fileno (stdout); + args->uerrfd =fileno (stderr); args->warn =CKPT_COND_WARN ; args->fail =CKPT_COND_FAIL ; no_pidns =0 ;
@@ -822,6 +824,7 @@ int app_restart(struct app_restart_args *args) global_debug = args->debug; global_verbose = args->verbose; global_ulogfd = args->ulogfd; + global_uerrfd = args->uerrfd;/* input file descriptor (default: stdin) */ if (args->infd >=0 ) {
Signature · 1 line
1.6.0.4
PATCH 5/14 Define INIT_SIGNAL_ARRAY
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 21:15:25 -0800
Subject: [PATCH 05/14][user-cr] Define INIT_SIGNAL_ARRAY
Code to initialize the signal_array[] table will need to be shared by more than one file. To enable this sharing with, define INIT_SIGNAL_ARRAY which can later be used in many places.
--- restart.c | 76 ++++++++++++++++++++++++++++++++----------------------------- 1 files changed, 40 insertions(+), 36 deletions(-)
restart.c +40 -36
@@ -157,46 +157,50 @@ static inline void ckpt_msg(int fd, char *format, ...)ckpt_msg (global_ulogfd, __VA_ARGS__); \ }while (0 ) -#define SIGNAL_ENTRY (signal) { SIG ## signal, #signal } - -struct { +struct signal_array {int signum;char *sigstr; -} signal_array[] = { - {0 ,"NONE" }, -SIGNAL_ENTRY (ALRM ), -SIGNAL_ENTRY (HUP ), -SIGNAL_ENTRY (INT ), -SIGNAL_ENTRY (KILL ), -SIGNAL_ENTRY (PIPE ), -SIGNAL_ENTRY (POLL ), -SIGNAL_ENTRY (PROF ), -SIGNAL_ENTRY (TERM ), -SIGNAL_ENTRY (USR1 ), -SIGNAL_ENTRY (USR2 ), -SIGNAL_ENTRY (VTALRM ), -SIGNAL_ENTRY (STKFLT ), -SIGNAL_ENTRY (PWR ), -SIGNAL_ENTRY (WINCH ), -SIGNAL_ENTRY (CHLD ), -SIGNAL_ENTRY (URG ), -SIGNAL_ENTRY (TTIN ), -SIGNAL_ENTRY (TTOU ), -SIGNAL_ENTRY (STOP ), -SIGNAL_ENTRY (CONT ), -SIGNAL_ENTRY (ABRT ), -SIGNAL_ENTRY (FPE ), -SIGNAL_ENTRY (ILL ), -SIGNAL_ENTRY (QUIT ), -SIGNAL_ENTRY (SEGV ), -SIGNAL_ENTRY (TRAP ), -SIGNAL_ENTRY (SYS ), -SIGNAL_ENTRY (BUS ), -SIGNAL_ENTRY (XCPU ), -SIGNAL_ENTRY (XFSZ ), - {-1 ,"LAST" }, }; +#define SIGNAL_ENTRY (signal) { SIG ## signal, #signal } + +#define INIT_SIGNAL_ARRAY { \ + { 0, "NONE" }, \ + SIGNAL_ENTRY(ALRM), \ + SIGNAL_ENTRY(HUP), \ + SIGNAL_ENTRY(INT), \ + SIGNAL_ENTRY(KILL), \ + SIGNAL_ENTRY(PIPE), \ + SIGNAL_ENTRY(POLL), \ + SIGNAL_ENTRY(PROF), \ + SIGNAL_ENTRY(TERM), \ + SIGNAL_ENTRY(USR1), \ + SIGNAL_ENTRY(USR2), \ + SIGNAL_ENTRY(VTALRM), \ + SIGNAL_ENTRY(STKFLT), \ + SIGNAL_ENTRY(PWR), \ + SIGNAL_ENTRY(WINCH), \ + SIGNAL_ENTRY(CHLD), \ + SIGNAL_ENTRY(URG), \ + SIGNAL_ENTRY(TTIN), \ + SIGNAL_ENTRY(TTOU), \ + SIGNAL_ENTRY(STOP), \ + SIGNAL_ENTRY(CONT), \ + SIGNAL_ENTRY(ABRT), \ + SIGNAL_ENTRY(FPE), \ + SIGNAL_ENTRY(ILL), \ + SIGNAL_ENTRY(QUIT), \ + SIGNAL_ENTRY(SEGV), \ + SIGNAL_ENTRY(TRAP), \ + SIGNAL_ENTRY(SYS), \ + SIGNAL_ENTRY(BUS), \ + SIGNAL_ENTRY(XCPU), \ + SIGNAL_ENTRY(XFSZ), \ + { -1, "LAST" }, \ +} + +static struct signal_array signal_array[] =INIT_SIGNAL_ARRAY ; +static char *sig2str (int sig) {int i =0 ;
Signature · 1 line
1.6.0.4
PATCH 6/14 Create common.h
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 21:30:50 -0800
Subject: [PATCH 06/14][user-cr] Create common.h
Code in common.h can be shared by checkpoint.c and restart.c for now.
--- Makefile | 4 ++ checkpoint.c | 31 +--------------- common.h | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ restart.c | 110 +-------------------------------------------------------- 4 files changed, 119 insertions(+), 138 deletions(-) create mode 100644 common.h
Makefile +4 -0
@@ -13,6 +13,8 @@ CKPT_HEADERS = include/linux/checkpoint.h \ include/linux/checkpoint_hdr.h \ include/asm/checkpoint_hdr.h +CR_OBJS = checkpoint.o restart.o + # detect architecture (for eclone) SUBARCH ?= $(patsubst i%86,x86_32,$(shell uname -m))
@@ -50,6 +52,8 @@ $(LIB_ECLONE): # restart needs to be thread-safe restart: CFLAGS += -D__REENTRANT -pthread +$(CR_OBJS): common.h + # eclone() is architecture specific ifneq ($(SUBARCH),) $(ECLONE_PROGS): $(LIB_ECLONE)
checkpoint.c +2 -29
@@ -21,6 +21,8 @@#include <linux/checkpoint.h> +#include "common.h" +static char usage_str[] ="usage: ckpt [opts] PID\n" " 'checkpoint' takes a checkpoint of the task indicated by PID, and all\n"
@@ -55,35 +57,6 @@ inline static int checkpoint(pid_t pid, int fd, unsigned long flags, int logfd)return syscall (__NR_checkpoint, pid, fd, flags, logfd); } -#define BUFSIZE (4 * 4096) -static inline void ckpt_msg (int fd,char *format, ...) -{ -va_list ap; -char *bufp; -if (fd <0 ) -return ; - -va_start (ap, format); - - bufp =malloc (BUFSIZE ); -if (bufp) { -vsnprintf (bufp,BUFSIZE , format, ap); -write (fd, bufp,strlen (bufp)); - } -free (bufp); - -va_end (ap); -} - -#define ckpt_err (...) \ - ckpt_msg(global_uerrfd, __VA_ARGS__) - -#define ckpt_perror (s) \ - do { \ - ckpt_msg(global_uerrfd, s); \ - ckpt_msg(global_uerrfd, ": %s\n", strerror(errno)); \ - } while (0) -static void usage (char *str) {ckpt_err ("%s" , str);
common.h new file +112 -0
112 lines not on this page · raw message
restart.c +1 -109
128 lines not on this page · raw message
Signature · 1 line
1.6.0.4
PATCH 7/14 Create app-checkpoint.h
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 21:51:27 -0800
Subject: [PATCH 07/14][user-cr] Create app-checkpoint.h
Create a new header file, app-checkpoint.h that can be used to export the app_checkpoint() and app_restart() functionality to other users (such as LXC).
For now, leave 'global_send_sigint' as an external variable. We should eventually implement an API that gives callers better control of how to handle signals during restart.
Changelog[v2]:
- (In response to Oren Laadan's comment): Leave 'global_send_sigint' as an extern and add a TODO to remove later.
--- Makefile | 2 +- app-checkpoint.h | 42 ++++++++++++++++++++++++++++++++++++++++++ checkpoint.c | 9 +-------- restart.c | 31 +++++++------------------------ 4 files changed, 51 insertions(+), 33 deletions(-) create mode 100644 app-checkpoint.h
Makefile +1 -1
@@ -52,7 +52,7 @@ $(LIB_ECLONE): # restart needs to be thread-safe restart: CFLAGS += -D__REENTRANT -pthread -$(CR_OBJS): common.h +$(CR_OBJS): common.h app-checkpoint.h # eclone() is architecture specific ifneq ($(SUBARCH),)
app-checkpoint.h new file +42 -0
42 lines not on this page · raw message
checkpoint.c +1 -8
@@ -21,6 +21,7 @@#include <linux/checkpoint.h> +#include "app-checkpoint.h" #include "common.h" static char usage_str[] =
@@ -44,14 +45,6 @@ static char usage_str[] =static int global_uerrfd =-1 ; -struct app_checkpoint_args { -int outfd; -int logfd; -int uerrfd; -int container; -int verbose; -}; -inline static int checkpoint (pid_t pid,int fd,unsigned long flags,int logfd) {return syscall (__NR_checkpoint, pid, fd, flags, logfd);
restart.c +7 -24
@@ -40,6 +40,7 @@#include "eclone.h" #include "genstack.h" #include "compat.h" +#include "app-checkpoint.h" #include "common.h" static char usage_str[] =
@@ -216,9 +217,14 @@ static int global_verbose;static pid_t global_child_pid;static int global_child_status;static int global_child_collected; -static int global_send_sigint =-1 ;static int global_sent_sigint; +/* +* TODO: Implement an API to let callers choose if/how an interrupt be sent +* and remove global_send_sigint. +*/ +int global_send_sigint =-1 ; +static int ckpt_remount_proc (struct ckpt_ctx *ctx);static int ckpt_remount_devpts (struct ckpt_ctx *ctx);
@@ -279,29 +285,6 @@ struct pid_swap {pid_t new; }; -struct app_restart_args { -int self; -int pids; -int pidns; -int inspect; -char *root; -int wait; -int mntns; -int mnt_pty; -int show_status; -int copy_status; -char *freezer; -int infd; -int klogfd; -int ulogfd; -int uerrfd; -long warn; -int debug; -int verbose; -long fail; -int keep_lsm; -}; -static void usage (char *str) {ckpt_err ("%s" , str);
Signature · 1 line
1.6.0.4
PATCH 8/14 restart: Move main() to restart-main.c
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 21:46:14 -0800
Subject: [PATCH 08/14][user-cr] restart: Move main() to restart-main.c
--- Makefile | 4 +- restart-main.c | 350 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ restart.c | 331 ----------------------------------------------------- 3 files changed, 353 insertions(+), 332 deletions(-) create mode 100644 restart-main.c
Makefile +3 -1
@@ -13,7 +13,7 @@ CKPT_HEADERS = include/linux/checkpoint.h \ include/linux/checkpoint_hdr.h \ include/asm/checkpoint_hdr.h -CR_OBJS = checkpoint.o restart.o +CR_OBJS = checkpoint.o restart.o restart-main.o # detect architecture (for eclone) SUBARCH ?= $(patsubst i%86,x86_32,$(shell uname -m))
@@ -54,6 +54,8 @@ restart: CFLAGS += -D__REENTRANT -pthread $(CR_OBJS): common.h app-checkpoint.h +restart: restart.o restart-main.o + # eclone() is architecture specific ifneq ($(SUBARCH),) $(ECLONE_PROGS): $(LIB_ECLONE)
restart-main.c new file +350 -0
350 lines not on this page · raw message
restart.c +0 -331
361 lines not on this page · raw message
Signature · 1 line
1.6.0.4
PATCH 9/14 checkpoint: Move main() to checkpoint-main.c
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Thu, 4 Mar 2010 22:28:34 -0800
Subject: [PATCH 09/14][user-cr] checkpoint: Move main() to checkpoint-main.c
--- Makefile | 4 +- checkpoint-main.c | 171 +++++++++++++++++++++++++++++++++++++++++++++++++++++ checkpoint.c | 157 ------------------------------------------------ 3 files changed, 174 insertions(+), 158 deletions(-) create mode 100644 checkpoint-main.c
Makefile +3 -1
@@ -13,7 +13,7 @@ CKPT_HEADERS = include/linux/checkpoint.h \ include/linux/checkpoint_hdr.h \ include/asm/checkpoint_hdr.h -CR_OBJS = checkpoint.o restart.o restart-main.o +CR_OBJS = checkpoint.o checkpoint-main.o restart.o restart-main.o # detect architecture (for eclone) SUBARCH ?= $(patsubst i%86,x86_32,$(shell uname -m))
@@ -56,6 +56,8 @@ $(CR_OBJS): common.h app-checkpoint.h restart: restart.o restart-main.o +checkpoint: checkpoint.o checkpoint-main.o + # eclone() is architecture specific ifneq ($(SUBARCH),) $(ECLONE_PROGS): $(LIB_ECLONE)
checkpoint-main.c new file +171 -0
171 lines not on this page · raw message
checkpoint.c +0 -157
172 lines not on this page · raw message
Signature · 1 line
1.6.0.4
PATCH 10/14 Have app_restart() return pid
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Fri, 5 Mar 2010 10:24:53 -0800
Subject: [PATCH 10/14][user-cr] Have app_restart() return pid
On success, return the pid of a the root of the restarted process tree. This would enable callers to manage the restarted process-tree as a cgroup.
--- restart-main.c | 7 ++++++- restart.c | 6 ++++++ 2 files changed, 12 insertions(+), 1 deletions(-)
restart-main.c +6 -1
@@ -332,6 +332,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])int main (int argc,char *argv[]) { +int ret;struct app_restart_args args;/*
@@ -345,6 +346,10 @@ int main(int argc, char *argv[])parse_args (&args, argc, argv); -return app_restart (&args); + ret =app_restart (&args); +if (ret >0 ) + ret =0 ; + +return ret; }
restart.c +6 -0
@@ -490,6 +490,12 @@ int app_restart(struct app_restart_args *args) ret =ckpt_coordinator (&ctx); } +/* +* On success, return pid of root of the restart process tree. +*/ +if (ret >=0 ) + ret = global_child_pid; +return ret; }
Signature · 1 line
1.6.0.4
PATCH 11/14 restart: Define process_args()
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Sat, 6 Mar 2010 13:14:02 -0800
Subject: [PATCH 11/14][user-cr] restart: Define process_args()
Move the code that validates/processes the arguments into a separate function.
--- restart.c | 27 ++++++++++++++++++++------- 1 files changed, 20 insertions(+), 7 deletions(-)
restart.c +20 -7
@@ -379,14 +379,12 @@ static int freezer_register(struct ckpt_ctx *ctx, pid_t pid)return ret; } -int app_restart (struct app_restart_args *args) +/* +* Validate the specified arguments and initialize globals based on the +* arguments. Return 0 on success. +*/ +int process_args (struct app_restart_args *args) { -struct ckpt_ctx ctx; -int ret; - -memset (&ctx,0 ,sizeof (ctx)); - ctx.args = args; - global_debug = args->debug; global_verbose = args->verbose; global_ulogfd = args->ulogfd;
@@ -406,6 +404,21 @@ int app_restart(struct app_restart_args *args)if (args->klogfd <0 ) args->klogfd =CHECKPOINT_FD_NONE ; +return 0 ; +} + +int app_restart (struct app_restart_args *args) +{ +struct ckpt_ctx ctx; +int ret; + +memset (&ctx,0 ,sizeof (ctx)); + ctx.args = args; + + ret =process_args (args); +if (ret <0 ) +return ret; +/* freezer preparation */ if (args->freezer &&freezer_prepare (&ctx) <0 )exit (1 );
Signature · 1 line
1.6.0.4
PATCH 12/14 app_restart(): mnt-pty implies mntns
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Sat, 6 Mar 2010 12:49:40 -0800
Subject: [PATCH 12/14][user-cr] app_restart(): mnt-pty implies mntns
If the mount_pty is set, mntns must be implied, otherwise the newinstance mount of devpts will make the ptys in the original pts instance inaccessible.
parse_args() does this correctly, but move the code into app_restart() so other callers of app_restart() also get this behavior.
--- restart-main.c | 4 ---- restart.c | 3 +++ 2 files changed, 3 insertions(+), 4 deletions(-)
restart-main.c +0 -4
@@ -324,10 +324,6 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])exit (1 ); } } - - -if (args->mnt_pty) - args->mntns =1 ; }int main (int argc,char *argv[])
restart.c +3 -0
@@ -404,6 +404,9 @@ int process_args(struct app_restart_args *args)if (args->klogfd <0 ) args->klogfd =CHECKPOINT_FD_NONE ; +if (args->mnt_pty) + args->mntns =1 ; +return 0 ; }
Signature · 1 line
1.6.0.4
PATCH 13/14 restart: Move args checking to app_restart()
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Sat, 6 Mar 2010 13:06:35 -0800
Subject: [PATCH 13/14][user-cr] restart: Move args checking to app_restart()
parse_args() validates the specified arguments to ensure they are consistent. Move this code into app_restart() so that these checks apply to other callers of app_restart() also.
--- restart-main.c | 33 +-------------------------------- restart.c | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 32 deletions(-)
restart-main.c +1 -32
@@ -261,38 +261,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])if (no_pidns) args->pidns =0 ; -#ifndef CLONE_NEWPID -if (args->pidns) { -ckpt_err ("This version of restart was compiled without " -"support for --pidns.\n" ); -exit (1 ); - } -#endif - -#ifndef CHECKPOINT_DEBUG -if (global_debug) { -ckpt_err ("This version of restart was compiled without " -"support for --debug.\n" ); -exit (1 ); - } -#endif - -if (args->pidns) - args->pids =1 ; - -#if 0 /* Defered until __NR_eclone makes it to standard headers */ -#ifndef __NR_eclone -if (args->pids) { -ckpt_err ("This version of restart was compiled without " -"support for --pids.\n" ); -exit (1 ); - } -#endif -#endif - -if (args->self && - (args->pids || args->pidns || no_pidns || - args->show_status || args->copy_status || args->freezer)) { +if (args->self && no_pidns) {ckpt_err ("Invalid mix of --self with multiprocess options\n" );exit (1 ); }
restart.c +36 -0
@@ -407,6 +407,42 @@ int process_args(struct app_restart_args *args)if (args->mnt_pty) args->mntns =1 ; +#ifndef CLONE_NEWPID +if (args->pidns) { +ckpt_err ("This version of restart was compiled without " +"support for --pidns.\n" ); +exit (1 ); + } +#endif + +#ifndef CHECKPOINT_DEBUG +if (global_debug) { +ckpt_err ("This version of restart was compiled without " +"support for --debug.\n" ); +exit (1 ); + } +#endif + +if (args->pidns) + args->pids =1 ; + +#if 0 /* Defered until __NR_eclone makes it to standard headers */ +#ifndef __NR_eclone +if (args->pids) { +ckpt_err ("This version of restart was compiled without " +"support for --pids.\n" ); +exit (1 ); + } +#endif +#endif + +if (args->self && + (args->pids || args->pidns || args->show_status || + args->copy_status || args->freezer)) { +ckpt_err ("Invalid mix of --self with multiprocess options\n" ); +exit (1 ); + } +return 0 ; }
Signature · 1 line
1.6.0.4
PATCH 14/14 Minimize unshare() calls
In reply to a message not in the archiveFrom: Sukadev Bhattiprolu <[email protected]>
Date: Mon, 8 Mar 2010 12:03:46 -0800
Subject: [PATCH 14/14][user-cr] Minimize unshare() calls
We currently have a few unshare() calls at different points in the code. While these don't affect the restart application itself, the excess calls create additional levels in the cgroup hierarchy, which can surprise the administrator (or other users of the hierarchy such as LXC.
Rather than several unshare() calls, can we instead specify the appropriate clone_flags while creating the coordinator/root process of the application tree ? When this root process is created it can remount /proc, remount devpts, chroot() etc if necessary.
Note that for "new-container with init" and "subtree restart", the first process is also the root of the application process tree.
In the case of "new-container without init", the coordinator process which acts as the container-init can do the setup.
In case of self-restart, the main process itself can do the unshare.
This patch has been very gently tested :-) but wanted to get more feedback on the direction and see if there is an easier way.
Changelog[v2]:
- Remove rather than '#ifdef 0' some unshare() calls
--- restart.c | 80 +++++++++++++++++++++++++++++++++++++++++++++--------------- 1 files changed, 60 insertions(+), 20 deletions(-)
restart.c +60 -20
@@ -127,6 +127,9 @@ struct task zero_task;#define TASK_SESSION 0x10/* inherits creator's original sid */ #define TASK_NEWPID 0x20/* starts a new pid namespace */ #define TASK_DEAD 0x40/* dead task (dummy) */ +#define TASK_NEWROOT 0x80/* task must chroot() */ +#define TASK_NEWPTS 0x100/* remount devpts */ +#define TASK_NEWNS 0x200/* unshare namespace/file-system */ struct ckpt_ctx {pid_t root_pid;
@@ -462,24 +465,24 @@ int app_restart(struct app_restart_args *args)if (args->freezer &&freezer_prepare (&ctx) <0 )exit (1 ); -/* private mounts namespace ? */ -if (args->mntns &&unshare (CLONE_NEWNS |CLONE_FS ) <0 ) { -ckpt_perror ("unshare" ); -exit (1 ); - } +/* self-restart ends here: */ +if (args->self) { +/* private mounts namespace ? */ +if (args->mntns &&unshare (CLONE_NEWNS |CLONE_FS ) <0 ) { +ckpt_perror ("unshare" ); +exit (1 ); + } -/* chroot ? */ -if (args->root &&chroot (args->root) <0 ) { -ckpt_perror ("chroot" ); -exit (1 ); - } +/* chroot ? */ +if (args->root &&chroot (args->root) <0 ) { +ckpt_perror ("chroot" ); +exit (1 ); + } -/* remount /dev/pts ? */ -if (args->mnt_pty &&ckpt_remount_devpts (&ctx) <0 ) -exit (1 ); +/* remount /dev/pts ? */ +if (args->mnt_pty &&ckpt_remount_devpts (&ctx) <0 ) +exit (1 ); -/* self-restart ends here: */ -if (args->self) {restart (getpid (),STDIN_FILENO ,RESTART_TASKSELF , args->klogfd);/* reach here if restart(2) failed ! */ ckpt_perror ("restart" );
@@ -524,10 +527,30 @@ int app_restart(struct app_restart_args *args)if (ret <0 )exit (1 ); +/* +* Have the first child in the restarted process tree +* setup devpts, root-dir and /proc if necessary, ... +*/ +if (ctx.args->mnt_pty) + ctx.tasks_arr[0 ].flags |=TASK_NEWPTS ; + +if (ctx.args->mntns) + ctx.tasks_arr[0 ].flags |=TASK_NEWNS ; + +if (ctx.args->root) + ctx.tasks_arr[0 ].flags |=TASK_NEWROOT ; +if (ctx.args->pidns && ctx.tasks_arr[0 ].pid !=1 ) {ckpt_dbg ("new pidns without init\n" );if (global_send_sigint ==-1 ) global_send_sigint =SIGINT ; +/* +* ...unless we have an explicit coordinator, in which case +* the coordinator should set up the filesystems and +* not the first process in the application process tree. +*/ + ctx.tasks_arr[0 ].flags &= ~(TASK_NEWPTS |TASK_NEWROOT | \ +TASK_NEWNS ); ret =ckpt_coordinator_pidns (&ctx); }else if (ctx.args->pidns) {ckpt_dbg ("new pidns with init\n" );
@@ -721,10 +744,6 @@ static int ckpt_probe_child(pid_t pid, char *str)*/ static int ckpt_remount_proc (struct ckpt_ctx *ctx) { -if (unshare (CLONE_NEWNS |CLONE_FS ) <0 ) { -ckpt_perror ("unshare" ); -return -1 ; - }/* this is unlikely, but we don't want to fail */ if (umount2 ("/proc" ,MNT_DETACH ) <0 ) {if (ckpt_cond_fail (ctx,CKPT_COND_MNTPROC )) {
@@ -747,9 +766,18 @@ static int __ckpt_coordinator(void *arg) {struct ckpt_ctx *ctx = (struct ckpt_ctx *) arg; +/* chroot ? */ +if (ctx->args->root &&chroot (ctx->args->root) <0 ) { +ckpt_perror ("chroot" ); +exit (1 ); + } +if (ckpt_remount_proc (ctx) <0 )return -1 ; +if (ctx->args->mnt_pty &&ckpt_remount_devpts (ctx) <0 ) +return -1 ; +if (!ctx->args->wait)close (ctx->pipe_coord[0 ]);
@@ -782,6 +810,7 @@ static int ckpt_coordinator_pidns(struct ckpt_ctx *ctx)int copy, ret;genstack stk;void *sp; +unsigned long flags =SIGCHLD ;ckpt_dbg ("forking coordinator in new pidns\n" );
@@ -806,7 +835,9 @@ static int ckpt_coordinator_pidns(struct ckpt_ctx *ctx) copy = ctx->args->copy_status; ctx->args->copy_status =1 ; - coord_pid =clone (__ckpt_coordinator, sp,CLONE_NEWPID |SIGCHLD , ctx); + flags |=CLONE_NEWPID |CLONE_NEWNS ; + + coord_pid =clone (__ckpt_coordinator, sp, flags, ctx);genstack_release (stk);if (coord_pid <0 ) {ckpt_perror ("clone coordinator" );
@@ -1614,10 +1645,16 @@ int ckpt_fork_stub(void *data)struct task *task = (struct task *) data;struct ckpt_ctx *ctx = task->ctx; +if ((task->flags &TASK_NEWROOT ) &&chroot (ctx->args->root) <0 ) +return -1 ; +/* tasks with new pid-ns need new /proc mount */ if ((task->flags &TASK_NEWPID ) &&ckpt_remount_proc (ctx) <0 )return -1 ; +if ((task->flags &TASK_NEWPTS ) &&ckpt_remount_devpts (ctx) <0 ) +return -1 ; +/* * In restart into a new pid namespace (--pidns), coordinator * is the container init, hence if it terminated permatutely
@@ -1695,6 +1732,9 @@ static pid_t ckpt_fork_child(struct ckpt_ctx *ctx, struct task *child) pid = 0; }#endif +if (child->flags &TASK_NEWNS ) { + flags |=CLONE_NEWNS ; + }if (child->flags & (TASK_SIBLING |TASK_THREAD )) child->real_parent =getppid ();
Signature · 1 line
1.6.0.4
A message that is not in the archive (sent off-list or not yet received): 20100319063448.GN24844-r/[email protected]
Serge E. Hallyn
Acked-byRe: [PATCH 14/14][user-cr] Minimize unshare() calls
Quoted 19 lines · Sukadev Bhattiprolu
From: Sukadev Bhattiprolu <[email protected]>
Date: Mon, 8 Mar 2010 12:03:46 -0800
Subject: [PATCH 14/14][user-cr] Minimizeunshare()callsWe currently have a few
unshare()calls at different points in the code. While these don't affect the restart application itself, the excess calls create additional levels in the cgroup hierarchy, which can surprise the administrator (or other users of the hierarchy such as LXC.Rather than several
unshare()calls, can we instead specify the appropriateclone_flagswhile creating the coordinator/root process of the application tree ? When this root process is created it can remount /proc, remount devpts,chroot()etc if necessary.Note that for "new-container with init" and "subtree restart", the first process is also the root of the application process tree.
Quoted 7 lines
In the case of "new-container without init", the coordinator process which acts as the container-init can do the setup.
In case of self-restart, the main process itself can do the unshare.
This patch has been very gently tested :-) but wanted to get more feedback on the direction and see if there is an easier way.
No objection from me on the direction or the code.
Re: [PATCH 14/14][user-cr] Minimize unshare() calls
In reply to a message not in the archiveQuoted 4 lines · Sukadev Bhattiprolu
From: Sukadev Bhattiprolu <[email protected]>
Date: Mon, 8 Mar 2010 12:03:46 -0800
Subject: [PATCH 14/14][user-cr] Minimizeunshare()calls
Hmm, and yet, when I run cr_tests/simple.sh, restart apparently does not use CLONE_NEWNS, bc I then have to
umount /proc
mount -t proc proc /proc
to restore sanity.
Not sure why... restart -vd gives me
Using output dir ./cr_simple_N3wEpal
about to clone with 20020000
<3793>number of tasks: 1
<3793>total tasks (including ghosts): 1
<3793>====== TASKS
<3793> [0] pid 1 ppid 0 sid 0 creator 0
<3793>............
<3793>new pidns with init
<3793>forking child vpid 1 flags 0x21
<3793>forked child vpid 3795 (asked 1)
<1>root task pid 1
<1>pid 1: pid 1 sid 0 parent 0
<1>about to call sys_restart(), flags 0
<3794>====== PIDS ARRAY
<3794>[0] pid 1 ppid 0 sid 0 pgid 0
<3794>............
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 11161
<3794>c/r read input 0
Success
<3793>restart succeeded
<3793>SIGCHLD: already collected
<3793>task exited with status 0
<3793>c/r succeeded
PASS
Oh, there you go - I suppose restart.c:557 should have TASK_NEWNS added? Indeed that fixes that.
-serge
Suka,
Thanks for the patches - applied on ckpt-v20-dev.
I think it's time to look for a naming convention for the library and API ...
Oren
Quoted 32 lines · Sukadev Bhattiprolu
Patches to enable linking LIBLXC with USERCR.
Patches 1 through 9 are simply rebased patches due to the dropping of one of the earlier patches (Oren had nacked one patch in my earlier set. See
https://lists.linux-foundation.org/pipermail/containers/2010-March/023564.html
Patches 10 through 14 are additional changes needed to get LXC checkpoint and restart to work.
[PATCH 01/14] Add
app_restart_args->debug[PATCH 02/14] Addapp_restart_args->verbose[PATCH 03/14] Addapp_restart_args->ulogfd[PATCH 04/14] Addapp_restart_args->uerrfd[PATCH 05/14] DefineINIT_SIGNAL_ARRAY
[PATCH 06/14] Createcommon.h
[PATCH 07/14] Createapp-checkpoint.h
[PATCH 08/14] restart: Movemain()torestart-main.c[PATCH 09/14] checkpoint: Movemain()tocheckpoint-main.c[PATCH 10/14] Have
app_restart()return pid [PATCH 11/14] restart: Defineprocess_args()[PATCH 12/14]app_restart(): mnt-pty implies mntns [PATCH 13/14] restart: Move args checking toapp_restart()[PATCH 14/14] Minimizeunshare()callsWith these patches, USERCR exports
app_checkpoint()andapp_restart()functions in checkpoint.o and restart.o. LXC links directly with these .o files to checkpoint/restart its containers.Signed-off-by Sukadev Bhattiprolu <[email protected]>