Skip to contentSkip to conversation

Checkpoint-restore tool v0.1

Announce Pavel Emelyanov 7 participants 23 messages Jul 23, 2012 – Sep 20, 2012 5 messages not in the archive cgroup

lore
Share & tools
lore

Pavel Emelyanov

author
#0 lore raw

Hi!

I'm glad to announce the first release of the checkpoint-restore tool.

This project is an attempt to implement the checkpoint-restore functionality for processes and containers without driving too much code into the kernel tree, but putting there "various oddball helper code" instead.

The tool can already be used for checkpointing and restoring various individual applications. And the greatest thing about this so far is that most of the below functionality has the required kernel support in the recently released v3.5!

So, we support now

* x86_64 architecture
* process' linkage
* process groups and sessions (without ttys though :\ )
* memory mappings of any kind (shared, file, etc.)
* threads
* open files (shared between tasks and partially opened-and-unlinked)
* pipes and fifos with data
* unix sockets with packet queues contents
* TCP and UDP sockets (TCP connections support exists, but needs polishing)
* inotifies, eventpoll and eventfd
* tasks' sigactions setup, credentials and itimers
* IPC, mount and PID namespaces

Though namespaces support is in there, we do not yet support an LXC container c/r, but we're close to it :)

I'd like to thank everyone who took part in new kernel APIs discussions, the feedback was great! Special thanks goes to Linus for letting the kernel parts in early, instead of making them sit out of tree till becoming stable enough.

Tarball with the tool sources is at

http://download.openvz.org/criu/crtools-0.1.tar.bz2

The git repo is at

http://git.criu.org/

And some sort of docs growing at

http://criu.org/

There are still things for which we don't have the kernel support merged (SysVIPC and various anon file descriptors, i.e. inotify, eventpoll, eventfd) yet. We have the kernel branch with the stuff applied available at

https://github.com/cyrillos/linux-2.6.git

Thanks,
Pavel

author
#1 lore raw

Fixing the bogus containers@ ml and adding cgroups@ one. Sorry :(

== Original message ==

Hi!

I'm glad to announce the first release of the checkpoint-restore tool.

This project is an attempt to implement the checkpoint-restore functionality for processes and containers without driving too much code into the kernel tree, but putting there "various oddball helper code" instead.

The tool can already be used for checkpointing and restoring various individual applications. And the greatest thing about this so far is that most of the below functionality has the required kernel support in the recently released v3.5!

So, we support now

* x86_64 architecture
* process' linkage
* process groups and sessions (without ttys though :\ )
* memory mappings of any kind (shared, file, etc.)
* threads
* open files (shared between tasks and partially opened-and-unlinked)
* pipes and fifos with data
* unix sockets with packet queues contents
* TCP and UDP sockets (TCP connections support exists, but needs polishing)
* inotifies, eventpoll and eventfd
* tasks' sigactions setup, credentials and itimers
* IPC, mount and PID namespaces

Though namespaces support is in there, we do not yet support an LXC container c/r, but we're close to it :)

I'd like to thank everyone who took part in new kernel APIs discussions, the feedback was great! Special thanks goes to Linus for letting the kernel parts in early, instead of making them sit out of tree till becoming stable enough.

Tarball with the tool sources is at

http://download.openvz.org/criu/crtools-0.1.tar.bz2

The git repo is at

http://git.criu.org/

And some sort of docs growing at

http://criu.org/

There are still things for which we don't have the kernel support merged (SysVIPC and various anon file descriptors, i.e. inotify, eventpoll, eventfd) yet. We have the kernel branch with the stuff applied available at

https://github.com/cyrillos/linux-2.6.git

Thanks,
Pavel

#2 lore raw
Quoted 58 lines · Pavel Emelyanov

Fixing the bogus containers@ ml and adding cgroups@ one. Sorry :(

== Original message ==

Hi!

I'm glad to announce the first release of the checkpoint-restore tool.

This project is an attempt to implement the checkpoint-restore functionality for processes and containers without driving too much code into the kernel tree, but putting there "various oddball helper code" instead.

The tool can already be used for checkpointing and restoring various individual applications. And the greatest thing about this so far is that most of the below functionality has the required kernel support in the recently released v3.5!

So, we support now

* x86_64 architecture
* process' linkage
* process groups and sessions (without ttys though :\ )
* memory mappings of any kind (shared, file, etc.)
* threads
* open files (shared between tasks and partially opened-and-unlinked)
* pipes and fifos with data
* unix sockets with packet queues contents
* TCP and UDP sockets (TCP connections support exists, but needs polishing)
* inotifies, eventpoll and eventfd
* tasks' sigactions setup, credentials and itimers
* IPC, mount and PID namespaces

Though namespaces support is in there, we do not yet support an LXC container c/r, but we're close to it :)

I'd like to thank everyone who took part in new kernel APIs discussions, the feedback was great! Special thanks goes to Linus for letting the kernel parts in early, instead of making them sit out of tree till becoming stable enough.

Tarball with the tool sources is at

http://download.openvz.org/criu/crtools-0.1.tar.bz2

The git repo is at

http://git.criu.org/

And some sort of docs growing at

http://criu.org/

There are still things for which we don't have the kernel support merged (SysVIPC and various anon file descriptors, i.e. inotify, eventpoll, eventfd) yet. We have the kernel branch with the stuff applied available at

Quoted 5 lines

https://github.com/cyrillos/linux-2.6.git

Thanks,
Pavel

Awesome - thanks, Pavel, and congratulations :) I'll have to give it a spin as soon as I get a chance.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw
Quoted 58 lines · Pavel Emelyanov

Fixing the bogus containers@ ml and adding cgroups@ one. Sorry :(

== Original message ==

Hi!

I'm glad to announce the first release of the checkpoint-restore tool.

This project is an attempt to implement the checkpoint-restore functionality for processes and containers without driving too much code into the kernel tree, but putting there "various oddball helper code" instead.

The tool can already be used for checkpointing and restoring various individual applications. And the greatest thing about this so far is that most of the below functionality has the required kernel support in the recently released v3.5!

So, we support now

* x86_64 architecture
* process' linkage
* process groups and sessions (without ttys though :\ )
* memory mappings of any kind (shared, file, etc.)
* threads
* open files (shared between tasks and partially opened-and-unlinked)
* pipes and fifos with data
* unix sockets with packet queues contents
* TCP and UDP sockets (TCP connections support exists, but needs polishing)
* inotifies, eventpoll and eventfd
* tasks' sigactions setup, credentials and itimers
* IPC, mount and PID namespaces

Though namespaces support is in there, we do not yet support an LXC container c/r, but we're close to it :)

I'd like to thank everyone who took part in new kernel APIs discussions, the feedback was great! Special thanks goes to Linus for letting the kernel parts in early, instead of making them sit out of tree till becoming stable enough.

Tarball with the tool sources is at

http://download.openvz.org/criu/crtools-0.1.tar.bz2

The git repo is at

http://git.criu.org/

And some sort of docs growing at

http://criu.org/

There are still things for which we don't have the kernel support merged (SysVIPC and various anon file descriptors, i.e. inotify, eventpoll, eventfd) yet. We have the kernel branch with the stuff applied available at

Quoted 5 lines

https://github.com/cyrillos/linux-2.6.git

Thanks,
Pavel

Awesome - thanks, Pavel, and congratulations :) I'll have to give it a spin as soon as I get a chance.

#5 lore raw
In reply to a message not in the archive
Quoted 3 lines · Pavel Emelyanov

This project is an attempt to implement the checkpoint-restore functionality for processes and containers without driving too much code into the kernel tree, but putting there "various oddball helper code" instead.

Is CRIU able to checkpoint a process tree where one or more processes are traced using ptrace()?
crtools seem to use ptrace() to execute parasite system calls. AFAIK this will only work if the target process is currently not ptraced() by another process.

Signature · 2 lines
Thanks,
//richard
#6 lore raw
In reply to Pavel Emelyanov
Quoted 3 lines · Pavel Emelyanov

This project is an attempt to implement the checkpoint-restore functionality for processes and containers without driving too much code into the kernel tree, but putting there "various oddball helper code" instead.

Is CRIU able to checkpoint a process tree where one or more processes are traced using ptrace()?
crtools seem to use ptrace() to execute parasite system calls. AFAIK this will only work if the target process is currently not ptraced() by another process.

Signature · 2 lines
Thanks,
//richard
author
#7 lore raw
Quoted 7 lines · richard -rw- weinberger
Quoted 3 lines · Pavel Emelyanov

This project is an attempt to implement the checkpoint-restore functionality for processes and containers without driving too much code into the kernel tree, but putting there "various oddball helper code" instead.

Is CRIU able to checkpoint a process tree where one or more processes are traced using ptrace()?

No it's not (currently).

Quoted 4 lines

crtools seem to use ptrace() to execute parasite system calls. AFAIK this will only work if the target process is currently not ptraced() by another process.

Yup, exactly.

Thanks,
Pavel

#8 lore raw
Quoted 9 lines · Pavel Emelyanov
Quoted 7 lines · richard -rw- weinberger
Quoted 3 lines · Pavel Emelyanov

This project is an attempt to implement the checkpoint-restore functionality for processes and containers without driving too much code into the kernel tree, but putting there "various oddball helper code" instead.

Is CRIU able to checkpoint a process tree where one or more processes are traced using ptrace()?

Quoted 8 lines

No it's not (currently).

Quoted 4 lines

crtools seem to use ptrace() to execute parasite system calls. AFAIK this will only work if the target process is currently not ptraced() by another process.

Yup, exactly.

Dammit, I hoped we can get suspend/resume for UserModeLinux for free with CRIU. 8-)

Signature · 2 lines
Thanks,
//richard
#9 lore raw
Quoted 10 lines · richard -rw- weinberger
Quoted 7 lines
Quoted 5 lines

crtools seem to use ptrace() to execute parasite system calls. AFAIK this will only work if the target process is currently not ptraced() by another process.

Yup, exactly.

Dammit, I hoped we can get suspend/resume for UserModeLinux for free with CRIU. 8-)

We have plans to implement seizing on traced tasks but no dates at the moment ;)

Cyrill

A message that is not in the archive (sent off-list or not yet received): CAFLxGvygCxER3gHfj-onwnm_m6xugOVkynGyJU2CDheD3A2HOQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#11 lore raw
Quoted 10 lines · richard -rw- weinberger
Quoted 7 lines
Quoted 5 lines

crtools seem to use ptrace() to execute parasite system calls. AFAIK this will only work if the target process is currently not ptraced() by another process.

Yup, exactly.

Dammit, I hoped we can get suspend/resume for UserModeLinux for free with CRIU. 8-)

We have plans to implement seizing on traced tasks but no dates at the moment ;)

Cyrill

author
#12 lore raw
In reply to richard -rw- weinberger
Quoted 17 lines · richard -rw- weinberger
Quoted 16 lines · Pavel Emelyanov
Quoted 7 lines · richard -rw- weinberger
Quoted 3 lines · Pavel Emelyanov

This project is an attempt to implement the checkpoint-restore functionality for processes and containers without driving too much code into the kernel tree, but putting there "various oddball helper code" instead.

Is CRIU able to checkpoint a process tree where one or more processes are traced using ptrace()?

No it's not (currently).

Quoted 4 lines

crtools seem to use ptrace() to execute parasite system calls. AFAIK this will only work if the target process is currently not ptraced() by another process.

Quoted 4 lines
Quoted 1 line

Yup, exactly.

Dammit, I hoped we can get suspend/resume for UserModeLinux for free with CRIU. 8-)

Sorry :( However, if you can help with patches on that issue that would be highly appreciated :)

Thanks,
Pavel

#13 lore raw
Quoted 3 lines · Pavel Emelyanov

Sorry :( However, if you can help with patches on that issue that would be highly appreciated :)

Yeah, but I fear it's not that easy.
We'd have to change crtools to work without ptrace().

Signature · 2 lines
Thanks,
//richard
author
#14 lore raw
Quoted 7 lines · richard -rw- weinberger
Quoted 3 lines · Pavel Emelyanov

Sorry :( However, if you can help with patches on that issue that would be highly appreciated :)

Yeah, but I fear it's not that easy.
We'd have to change crtools to work without ptrace().

Well, this is hard. Using ptrace saved us from having many special-purpose APIs for dumping various stuff (there will be an article about it). Thus I don't know which way is simpler -- stop using ptrace or teach ptrece to allow several tracers to attach to one task %)

Thanks,
Pavel

#15 lore raw
Quoted 7 lines · Pavel Emelyanov
Quoted 2 lines

Yeah, but I fear it's not that easy.
We'd have to change crtools to work without ptrace().

Well, this is hard. Using ptrace saved us from having many special-purpose APIs for dumping various stuff (there will be an article about it). Thus I don't know which way is simpler -- stop using ptrace or teach ptrece to allow several tracers to attach to one task %)

Allowing multiple tracers in a safe way is IMHO even more harder.

BTW: While reading prctl_set_mm() I noticed two things.
1. Why isn't the return value of find_vma() verified? It looks like one can set an addr which does not belong to any vma.
2. What will happen if addr is not page aligned?

Signature · 2 lines
Thanks,
//richard
#16 lore raw
Quoted 9 lines · richard -rw- weinberger
Quoted 7 lines · Pavel Emelyanov
Quoted 2 lines

Yeah, but I fear it's not that easy.
We'd have to change crtools to work without ptrace().

Well, this is hard. Using ptrace saved us from having many special-purpose APIs for dumping various stuff (there will be an article about it). Thus I don't know which way is simpler -- stop using ptrace or teach ptrece to allow several tracers to attach to one task %)

Quoted 4 lines

Allowing multiple tracers in a safe way is IMHO even more harder.

BTW: While reading prctl_set_mm() I noticed two things.
1. Why isn't the return value of find_vma() verified?

prctl_set_mm

	vma = find_vma(mm, addr);
	...
	if (!vma) {
		error = -EFAULT;
		goto out;
	}

these values are used in procfs statistics only. So I don't get which verify you mean here.

Quoted 2 lines

It looks like one can set an addr which does not belong to any vma.
2. What will happen if addr is not page aligned?

you'll have a bit weird output in procfs stat (probably I should add page alignment check here).

Cyrill

#17 lore raw
Quoted 23 lines · Cyrill Gorcunov
Quoted 9 lines · richard -rw- weinberger
Quoted 7 lines · Pavel Emelyanov
Quoted 2 lines

Yeah, but I fear it's not that easy.
We'd have to change crtools to work without ptrace().

Well, this is hard. Using ptrace saved us from having many special-purpose APIs for dumping various stuff (there will be an article about it). Thus I don't know which way is simpler -- stop using ptrace or teach ptrece to allow several tracers to attach to one task %)

Quoted 4 lines

Allowing multiple tracers in a safe way is IMHO even more harder.

BTW: While reading prctl_set_mm() I noticed two things.
1. Why isn't the return value of find_vma() verified?

prctl_set_mm

        vma = find_vma(mm, addr);
        ...
        if (!vma) {
                error = -EFAULT;
                goto out;
        }
Quoted 2 lines

these values are used in procfs statistics only. So I don't get which verify you mean here.

If I do PR_SET_MM_START_BRK the if(!vma) will never be executed because there a break in case PR_SET_MM_START_BRK.

Signature · 2 lines
Thanks,
//richard
#18 lore raw
Quoted 24 lines · richard -rw- weinberger
Quoted 23 lines · Cyrill Gorcunov
Quoted 9 lines · richard -rw- weinberger
Quoted 7 lines · Pavel Emelyanov
Quoted 2 lines

Yeah, but I fear it's not that easy.
We'd have to change crtools to work without ptrace().

Well, this is hard. Using ptrace saved us from having many special-purpose APIs for dumping various stuff (there will be an article about it). Thus I don't know which way is simpler -- stop using ptrace or teach ptrece to allow several tracers to attach to one task %)

Quoted 4 lines

Allowing multiple tracers in a safe way is IMHO even more harder.

BTW: While reading prctl_set_mm() I noticed two things.
1. Why isn't the return value of find_vma() verified?

prctl_set_mm

        vma = find_vma(mm, addr);
        ...
        if (!vma) {
                error = -EFAULT;
                goto out;
        }
Quoted 5 lines
Quoted 2 lines

these values are used in procfs statistics only. So I don't get which verify you mean here.

If I do PR_SET_MM_START_BRK the if(!vma) will never be executed because there a break in case PR_SET_MM_START_BRK.

Yes, and this is done by purpose, since we need to setup _completely_ new memory map on restore procedure.

There is a minimal check for value being sane

	if (addr >= TASK_SIZE || addr < mmap_min_addr)
		return -EINVAL;

and the address belongs to mm::start_data|end_data area. But sure, better to add checks that at least code/data areas do exist, otherwise the proc output will not reflect the real state of memory maps.

Cyrill

#19 lore raw
In reply to Cyrill Gorcunov
Quoted 23 lines · Cyrill Gorcunov
Quoted 9 lines · richard -rw- weinberger
Quoted 7 lines · Pavel Emelyanov
Quoted 2 lines

Yeah, but I fear it's not that easy.
We'd have to change crtools to work without ptrace().

Well, this is hard. Using ptrace saved us from having many special-purpose APIs for dumping various stuff (there will be an article about it). Thus I don't know which way is simpler -- stop using ptrace or teach ptrece to allow several tracers to attach to one task %)

Quoted 4 lines

Allowing multiple tracers in a safe way is IMHO even more harder.

BTW: While reading prctl_set_mm() I noticed two things.
1. Why isn't the return value of find_vma() verified?

prctl_set_mm

        vma = find_vma(mm, addr);
        ...
        if (!vma) {
                error = -EFAULT;
                goto out;
        }
Quoted 2 lines

these values are used in procfs statistics only. So I don't get which verify you mean here.

If I do PR_SET_MM_START_BRK the if(!vma) will never be executed because there a break in case PR_SET_MM_START_BRK.

Signature · 2 lines
Thanks,
//richard

A message that is not in the archive (sent off-list or not yet received): CAFLxGvwu8-ii7hDbDfco3JrgKgOK+hsdTN_Q--AHbom0Ni8WYA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#21 lore raw
Quoted 9 lines · richard -rw- weinberger
Quoted 7 lines · Pavel Emelyanov
Quoted 2 lines

Yeah, but I fear it's not that easy.
We'd have to change crtools to work without ptrace().

Well, this is hard. Using ptrace saved us from having many special-purpose APIs for dumping various stuff (there will be an article about it). Thus I don't know which way is simpler -- stop using ptrace or teach ptrece to allow several tracers to attach to one task %)

Quoted 4 lines

Allowing multiple tracers in a safe way is IMHO even more harder.

BTW: While reading prctl_set_mm() I noticed two things.
1. Why isn't the return value of find_vma() verified?

prctl_set_mm

	vma = find_vma(mm, addr);
	...
	if (!vma) {
		error = -EFAULT;
		goto out;
	}

these values are used in procfs statistics only. So I don't get which verify you mean here.

Quoted 2 lines

It looks like one can set an addr which does not belong to any vma.
2. What will happen if addr is not page aligned?

you'll have a bit weird output in procfs stat (probably I should add page alignment check here).

Cyrill

A message that is not in the archive (sent off-list or not yet received): [email protected]

#23 lore raw
Quoted 3 lines · Pavel Emelyanov

Sorry :( However, if you can help with patches on that issue that would be highly appreciated :)

Yeah, but I fear it's not that easy.
We'd have to change crtools to work without ptrace().

Signature · 2 lines
Thanks,
//richard
author
#24 lore raw

[Announce] Checkpoint-restore tool v0.2

In reply to Pavel Emelyanov

Hi!

I'm happy to announce that the next big step is done by the CRIU project.

The major achievement so far -- the tool can now dump and restore a simple LXC container. Another notable feature is the initial support for PTYs which is enough to play with c/r of a container with active ssh sessions from outside, or with a screen app with some simple stuff running inside.

A HOWTO describing an LXC container dump and restore is at http://criu.org/LXC

The v0.2 can be downloaded from

	http://download.openvz.org/criu/crtools-0.2.tar.bz2

Other project resources are

  documentation and news:             http://criu.org
  mainling list:                      https://openvz.org/mailman/listinfo/criu
  git repo:                           http://git.criu.org/?p=crtools.git

kernel with not-yet-merged patches: https://github.com/cyrillos/linux-2.6.git

Thanks,
Pavel

#25 lore raw

Re: [Announce] Checkpoint-restore tool v0.2

Congratulations, Pavel and criu team, that's awesome!

Quoted 30 lines · Pavel Emelyanov

Hi!

I'm happy to announce that the next big step is done by the CRIU project.

The major achievement so far -- the tool can now dump and restore a simple LXC container. Another notable feature is the initial support for PTYs which is enough to play with c/r of a container with active ssh sessions from outside, or with a screen app with some simple stuff running inside.

A HOWTO describing an LXC container dump and restore is at http://criu.org/LXC

The v0.2 can be downloaded from

	http://download.openvz.org/criu/crtools-0.2.tar.bz2

Other project resources are

  documentation and news:             http://criu.org
  mainling list:                      https://openvz.org/mailman/listinfo/criu
  git repo:                           http://git.criu.org/?p=crtools.git

kernel with not-yet-merged patches: https://github.com/cyrillos/linux-2.6.git

Thanks,
Pavel
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

A message that is not in the archive (sent off-list or not yet received): [email protected]

#27 lore raw

Re: [Announce] Checkpoint-restore tool v0.2

Congratulations, Pavel and criu team, that's awesome!

Quoted 30 lines · Pavel Emelyanov

Hi!

I'm happy to announce that the next big step is done by the CRIU project.

The major achievement so far -- the tool can now dump and restore a simple LXC container. Another notable feature is the initial support for PTYs which is enough to play with c/r of a container with active ssh sessions from outside, or with a screen app with some simple stuff running inside.

A HOWTO describing an LXC container dump and restore is at http://criu.org/LXC

The v0.2 can be downloaded from

	http://download.openvz.org/criu/crtools-0.2.tar.bz2

Other project resources are

  documentation and news:             http://criu.org
  mainling list:                      https://openvz.org/mailman/listinfo/criu
  git repo:                           http://git.criu.org/?p=crtools.git

kernel with not-yet-merged patches: https://github.com/cyrillos/linux-2.6.git

Thanks,
Pavel
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linuxfoundation.org/mailman/listinfo/containers