restart (mktree) program usage
I have a usage question on the 'restart' (formerly mktree) program.
In the following container c/r case:
- create a container
- log in to the container,
- restore filesystem(s) from snapshot
- restart application from checkpoint
On restart, suppose the user wants to restore the original pids. But he does not want to create a new pid-ns, (since he just created the container, and is sure the original pids are available).
To accomplish this the user has to specify the arguments in the following order right (since -pids implies --pidns).
restart --pids --no-pidns
IOW, the order of the arguments matters.
Would it be easier to understand if --pids did not imply --pidns ?
(Or in fact the reverse seems to make more sense -i.e --pidns implies --pids, with a new option, --no-pids if user absolutely hates the pids he was dealt before checkpoint :-)
$ restart don't create pid ns, don't try to restore pids
$ restart --pids try to restore original pids, don't create pid-ns
$ restart --pidns create new pid-namespace and restore original pids
$ restart --pidns --no-pids $ restart --no-pids --pidns create new pid-namespace, DO NOT restore pids
Or maybe drop implying and let user explicitly specify one, none or both of --pids, --pidns ?
Sukadev
A message that is not in the archive (sent off-list or not yet received): 20090909055636.GA27622-r/[email protected]
Quoted 8 lines · Sukadev Bhattiprolu
I have a usage question on the 'restart' (formerly mktree) program.
In the following container c/r case:
- create a container
- log in to the container,
- restore filesystem(s) from snapshot
- restart application from checkpoint
FWIW, I'd expect that future versions of 'restart' will be capable of doing this entire setup, (filesystem(s) included), as it matures.
Note that this use case that you suggest will only work to restart subtrees; it is unsuitable for full containers (with pids) because the pid of init (1) will already be in use.
Perhaps we should think of some "plugin" architecture for 'restart' that will allow the user to ask it to execute some work at between creating a new container and actually restarting into it ?
Quoted 5 lines
On restart, suppose the user wants to restore the original pids. But he does not want to create a new pid-ns, (since he just created the container, and is sure the original pids are available).
Quoted 8 lines
To accomplish this the user has to specify the arguments in the following order right (since -pids implies --pidns).
restart --pids --no-pidnsIOW, the order of the arguments matters.
Would it be easier to understand if --pids did not imply --pidns ?
Either that, or simply change the semantics such that the order does not matter:
Quoted 11 lines
(Or in fact the reverse seems to make more sense -i.e --pidns implies --pids, with a new option, --no-pids if user absolutely hates the pids he was dealt before checkpoint :-)
$ restart don't create pid ns, don't try to restore pids$ restart --pids try to restore original pids, don't create pid-ns
Quoted 6 lines
$ restart --pidns create new pid-namespace and restore original pids$ restart --pidns --no-pids $ restart --no-pids --pidns create new pid-namespace, DO NOT restore pids
Yes, something like that.
Oren.
Quoted 9 lines
Or maybe drop implying and let user explicitly specify one, none or both of --pids, --pidns ?
Sukadev
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
A message that is not in the archive (sent off-list or not yet received): [email protected]
Oren Laadan [[email protected]] wrote:
| | | Sukadev Bhattiprolu wrote: | > I have a usage question on the 'restart' (formerly mktree) program. | > | > In the following container c/r case: | > | > - create a container | > - log in to the container, | > - restore filesystem(s) from snapshot | > - restart application from checkpoint | | FWIW, I'd expect that future versions of 'restart' will be capable | of doing this entire setup, (filesystem(s) included), as it matures. | | Note that this use case that you suggest will only work to restart | subtrees; it is unsuitable for full containers (with pids) because | the pid of init (1) will already be in use.
True. But if originally the application was started as:
Create container
Login to contaienr
Set up filesystem
Start application
The application would not be using the pid 1 right - even if the application was started from an rc script in the container ?
| | Perhaps we should think of some "plugin" architecture for 'restart' | that will allow the user to ask it to execute some work at between | creating a new container and actually restarting into it ?
Yes, that would be really useful I think for things like restoring file system to its snapshot. Without that there is somewhat of an assymetry in starting an application in a container and restarting it from a checkpoint.
A message that is not in the archive (sent off-list or not yet received): 20090917013546.GA30161-r/[email protected]
Quoted 20 lines · Sukadev Bhattiprolu
Oren Laadan [[email protected]] wrote:
| | | Sukadev Bhattiprolu wrote: | > I have a usage question on the 'restart' (formerly mktree) program. | > | > In the following container c/r case: | > | > - create a container | > - log in to the container, | > - restore filesystem(s) from snapshot | > - restart application from checkpoint | | FWIW, I'd expect that future versions of 'restart' will be capable | of doing this entire setup, (filesystem(s) included), as it matures. | | Note that this use case that you suggest will only work to restart | subtrees; it is unsuitable for full containers (with pids) because | the pid of init (1) will already be in use.
Quoted 4 lines
True. But if originally the application was started as:
Create container
Login to contaienr
Actually, I'm not sure what you mean by "login to container" ?
Quoted 3 lines
Set up filesystem
Start application
[One way to solve this is - after the file systems are setup, to start a container inside that container :p ... or not].
Ok, let's assume that the application was started inside an existing container to begin with, but was checkpointed as a whole-container -- with the ancestor(s) processes there - then we basically have a checkpoint image that holds more processes than we really care about.
So what ? The simple fact is that the checkpoint image contains a task with pid 1. So it won't restart unless in a new container.
I'd suggest checkpointing only a subtree, but that may not be an option, e.g. if the application already has some orphan processes - unreachable under subtree !.
Or, I'd suggest to use a userspace tool to chop data away from the checkpoint image (e.g. remove processes ...). But if you remove the init process, you again face a problem with the orphans.
<warning> Here's a crazy idea: </warning>
Userspace maniuplation of the checkpoint image is a powerful tool. I can imagine, for instance, a flag RESTART_I_AM_FINE_THANK_YOU with which a process tells the restart to to let it be (like the current ghost tasks, but without exiting).
How is that useful ? combine that with some checkpoint image tweaks, and you can drop the init(1) task from the checkpoint image, and have the real init task in the new container participate in the restart without really restoring its state.... voila.
In fact, just like the proposed cradvise() would be able to tell the kernel to use a given resource instead of recreating it from the image, such a flag could tell the kernel to do so for processes.
Ok... a bit carried away. But maybe someone will find this idea not only cool, but also useful :)
[Or, you start a new container, setup file systems, and then restart into a new - nested - container :p ...]
Oren.
Quoted 12 lines
The application would not be using the pid 1 right - even if the application was started from an rc script in the container ?
| | Perhaps we should think of some "plugin" architecture for 'restart' | that will allow the user to ask it to execute some work at between | creating a new container and actually restarting into it ?Yes, that would be really useful I think for things like restoring file system to its snapshot. Without that there is somewhat of an assymetry in starting an application in a container and restarting it from a checkpoint.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 23 lines · Oren Laadan
Quoted 20 lines · Sukadev Bhattiprolu
Oren Laadan [[email protected]] wrote:
| | | Sukadev Bhattiprolu wrote: | > I have a usage question on the 'restart' (formerly mktree) program. | > | > In the following container c/r case: | > | > - create a container | > - log in to the container, | > - restore filesystem(s) from snapshot | > - restart application from checkpoint | | FWIW, I'd expect that future versions of 'restart' will be capable | of doing this entire setup, (filesystem(s) included), as it matures. | | Note that this use case that you suggest will only work to restart | subtrees; it is unsuitable for full containers (with pids) because | the pid of init (1) will already be in use.
Quoted 6 lines
Quoted 4 lines
True. But if originally the application was started as:
Create container
Login to contaienrActually, I'm not sure what you mean by "login to container" ?
I assume he was thinking of a system container created with liblxc or libvirt, and literally logging in on its console or over ssh.
...
Quoted 4 lines
Quoted 4 lines
Yes, that would be really useful I think for things like restoring file system to its snapshot. Without that there is somewhat of an assymetry in starting an application in a container and restarting it from a checkpoint.
Fine, they are different operations :)
-serge
A message that is not in the archive (sent off-list or not yet received): 20090917131843.GA29297-r/[email protected]
Serge E. Hallyn [serue-r/[email protected]] wrote:
| > > True. But if originally the application was started as: | > > | > > Create container | > > Login to contaienr | > | > Actually, I'm not sure what you mean by "login to container" ? | | I assume he was thinking of a system container created with liblxc | or libvirt, and literally logging in on its console or over ssh.
Yes, but...
| | ... | | > > Yes, that would be really useful I think for things like restoring file | > > system to its snapshot. Without that there is somewhat of an assymetry | > > in starting an application in a container and restarting it from a | > > checkpoint. | | Fine, they are different operations :)
...more than the assymetry, I guess I am not sure what is the "recommended" way to restart an application, while ensuring filesystem (and network) are correctly restored by user.
ns_exec --cpuim -- /usr/bin/restart --wait < checkpoint-image.1
assumes fs/network are setup. Hence the "login" to the container.
|
| -serge
A message that is not in the archive (sent off-list or not yet received): 20090917165311.GB13855-r/[email protected]
Quoted 22 lines · Sukadev Bhattiprolu
Serge E. Hallyn [serue-r/[email protected]] wrote:
| > > True. But if originally the application was started as: | > > | > > Create container | > > Login to contaienr | > | > Actually, I'm not sure what you mean by "login to container" ? | | I assume he was thinking of a system container created with liblxc | or libvirt, and literally logging in on its console or over ssh.Yes, but...
| | ... | | > > Yes, that would be really useful I think for things like restoring file | > > system to its snapshot. Without that there is somewhat of an assymetry | > > in starting an application in a container and restarting it from a | > > checkpoint. | | Fine, they are different operations :)
Quoted 7 lines
...more than the assymetry, I guess I am not sure what is the "recommended" way to restart an application, while ensuring filesystem (and network) are correctly restored by user.
ns_exec --cpuim -- /usr/bin/restart --wait < checkpoint-image.1assumes fs/network are setup. Hence the "login" to the container.
Right, this is stuff to discuss at the plumbers talk and BOF. How much of this do we want /usr/bin/restart to handle? It already does pidns, and IMO having it handle everything needed makes sense. The only problem is we can't have it handle filesystem yet because we haven't yet decided where/how to checkpont the fs :) I don't even mean mounts, I mean a btrfs snapshot or rsync. Do we want to create a 'container_checkpoint' program which wraps 'checkpoint' and does filesystem and network interface snapshotting?
I'm not asking for an answer in this thread, but for it to be discussed at the bof :)
-serge
Quoted 23 lines · Serge E. Hallyn
Quoted 22 lines · Oren Laadan
Quoted 20 lines · Sukadev Bhattiprolu
Oren Laadan [[email protected]] wrote:
| | | Sukadev Bhattiprolu wrote: | > I have a usage question on the 'restart' (formerly mktree) program. | > | > In the following container c/r case: | > | > - create a container | > - log in to the container, | > - restore filesystem(s) from snapshot | > - restart application from checkpoint | | FWIW, I'd expect that future versions of 'restart' will be capable | of doing this entire setup, (filesystem(s) included), as it matures. | | Note that this use case that you suggest will only work to restart | subtrees; it is unsuitable for full containers (with pids) because | the pid of init (1) will already be in use.
Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
True. But if originally the application was started as:
Create container
Login to contaienrActually, I'm not sure what you mean by "login to container" ?
I assume he was thinking of a system container created with liblxc or libvirt, and literally logging in on its console or over ssh.
This is exactly my point:
* If you checkpoint full container, you get the sshd (and init) as well, so you can't restore into an "existing" container, but create a new one.
* If you checkpoint subtree, you will miss orphans, and you will give up leak-detection.
I'd assume most users of this scenarios will prefer full container.
If you want to restart by first creating a new container, than set it up, then log-in to that container, and finally run restart - you will _have to_ modify the checkpoint image to remove unwanted parts, and adjust the remaining accordingly.
For instance, you will have to remove (or tell the kernel to skip) the data for the init task, and the ssh-server, and then change the ppid of the root of your subtree to be 1 instead of what it was (probably the old ssh-server), and so on.
It's doable, but I'm unsure if it's worth the extra work.
Oren.
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
Yes, that would be really useful I think for things like restoring file system to its snapshot. Without that there is somewhat of an assymetry in starting an application in a container and restarting it from a checkpoint.
Fine, they are different operations :)
-serge
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 37 lines · Oren Laadan
Quoted 23 lines · Serge E. Hallyn
Quoted 22 lines · Oren Laadan
Quoted 20 lines · Sukadev Bhattiprolu
Oren Laadan [[email protected]] wrote:
| | | Sukadev Bhattiprolu wrote: | > I have a usage question on the 'restart' (formerly mktree) program. | > | > In the following container c/r case: | > | > - create a container | > - log in to the container, | > - restore filesystem(s) from snapshot | > - restart application from checkpoint | | FWIW, I'd expect that future versions of 'restart' will be capable | of doing this entire setup, (filesystem(s) included), as it matures. | | Note that this use case that you suggest will only work to restart | subtrees; it is unsuitable for full containers (with pids) because | the pid of init (1) will already be in use.Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
True. But if originally the application was started as:
Create container
Login to contaienrActually, I'm not sure what you mean by "login to container" ?
I assume he was thinking of a system container created with liblxc or libvirt, and literally logging in on its console or over ssh.
This is exactly my point:
Quoted 8 lines
* If you checkpoint full container, you get the sshd (and init) as well, so you can't restore into an "existing" container, but create a new one.
* If you checkpoint subtree, you will miss orphans, and you will give up leak-detection.
I'd assume most users of this scenarios will prefer full container.
Yup, I think we are agreed.
-serge
Quoted 20 lines · Sukadev Bhattiprolu
Oren Laadan [[email protected]] wrote:
| | | Sukadev Bhattiprolu wrote: | > I have a usage question on the 'restart' (formerly mktree) program. | > | > In the following container c/r case: | > | > - create a container | > - log in to the container, | > - restore filesystem(s) from snapshot | > - restart application from checkpoint | | FWIW, I'd expect that future versions of 'restart' will be capable | of doing this entire setup, (filesystem(s) included), as it matures. | | Note that this use case that you suggest will only work to restart | subtrees; it is unsuitable for full containers (with pids) because | the pid of init (1) will already be in use.
Quoted 6 lines
True. But if originally the application was started as:
Create container
Login to contaienr
Set up filesystem
Start application
You should take a look at lxc, which is doing what you want.
Metacluster (IBM's proprietary C/R solution) is fully integrated with lxc containers and the current lxc development plans focus on externalizing some home made C/R plugins we've been doing for it. We should then spend some time on blcr.
And we expect you guys to work on oren's patchset ? but I guess that will all depend on how 'proprietary' is mktree, by 'proprietary' I mean dependent on the kernel patch.
Cheers,
C.