Skip to contentSkip to conversation

State of linux checkpointing?

DISCUSSION Neal D. Becker 5 participants 7 messages Apr 28, 2004 – Apr 29, 2004

lore
Share & tools
lore
#0 lore raw

I wonder if there is a checkpointing that will work with 2.6 kernels?

I only need relatively basic checkpointing. No sockets or fancy stuff.

#1 lore raw
Quoted 3 lines · Neal D. Becker

I wonder if there is a checkpointing that will work with 2.6 kernels?

I only need relatively basic checkpointing. No sockets or fancy stuff.

You only need checkpointing when your application programmers are lazy and don't care about data integrity. :)

Jeff

#2 lore raw

Jeff Garzik <[email protected]> said on Wed, 28 Apr 2004 16:23:00 -0400:

Quoted 7 lines
Quoted 3 lines · Neal D. Becker

I wonder if there is a checkpointing that will work with 2.6 kernels?

I only need relatively basic checkpointing. No sockets or fancy stuff.

You only need checkpointing when your application programmers are lazy and don't care about data integrity. :)

Or you are running some kind of cluster where you want the applications to be checkpointed transparently without the application knowing the details of how or when they will be swapped out (but this will need sockets anyway, so won't happen anytime soon).

'Tis a pain that the alpha cluster here can suspend long running jobs for a pile of smaller jobs, and then resume, but the linux cluster can do no such fanciness (yes, we do manual checkpointing, but it's prone to bugs - and finding such a bug after 30 days of compute time really sucks balls).

Signature · 2 lines
TimC -- http://astronomy.swin.edu.au/staff/tconnors/
Beware of Programmers who carry screwdrivers.
#3 lore raw
Quoted 6 lines · Tim Connors

Jeff Garzik <[email protected]> said on Wed, 28 Apr 2004 16:23:00 -0400:

Quoted 5 lines
Quoted 3 lines · Neal D. Becker

I wonder if there is a checkpointing that will work with 2.6 kernels?

I only need relatively basic checkpointing. No sockets or fancy stuff.

Quoted 8 lines
Quoted 2 lines

You only need checkpointing when your application programmers are lazy and don't care about data integrity. :)

Or you are running some kind of cluster where you want the applications to be checkpointed transparently without the application knowing the details of how or when they will be swapped out (but this will need sockets anyway, so won't happen anytime soon).

I want checkpointing for:

1) Protect against job interruption due to system crash, operator error, power loss, whatever

2) Job mygration. Even manual job mygration would be nice.

#5 lore raw

Thomas Davis <[email protected]> said on Thu, 29 Apr 2004 09:31:14 -0700:

Quoted 4 lines
Quoted 3 lines · Neal Becker

I want checkpointing for:

Quoted 8 lines
Quoted 4 lines

1) Protect against job interruption due to system crash, operator error, power loss, whatever

2) Job mygration. Even manual job mygration would be nice.

Two possible solutions:

1) http://ftg.lbl.gov/checkpoint

Oooh. Shiny.

That looks relatively new? I haven't come across it before...

Signature · 5 lines
TimC -- http://astronomy.swin.edu.au/staff/tconnors/
Yesterday, after years of trying, I finally managed to take a photo of a
subway train that said "INSTRUCTION CAR" just so that someday I can caption
it "...but where's the DATA CDR?" when I'm ready to make a joke that's
nerdy even by the standards of jokes about LISP.    -- James "Kibo" Perry
#6 lore raw
Quoted 8 lines · Tim Connors

Thomas Davis <[email protected]> said on Thu, 29 Apr 2004 09:31:14 -0700:

Quoted 7 lines
Quoted 6 lines · Neal Becker

I want checkpointing for:

1) Protect against job interruption due to system crash, operator error, power loss, whatever

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

2) Job mygration. Even manual job mygration would be nice.

Two possible solutions:

1) http://ftg.lbl.gov/checkpoint

Oooh. Shiny.

Looks interesting. Kernel-2.4 only AFAICT.