Automatic snapshot system?

Has anyone worked up a system that automatically snapshots volumes at regular intervals and then expires the snapshots as necessary? If not, I may take a stab at it.

- J<

We have a home-brewed perl script called from cron that does a snap, rsync's out the data, then releases the snapshot.

Is that what you're talking about?

Share and Enjoy. 

p> We have a home-brewed perl script called from cron that does a p> snap, rsync's out the data, then releases the snapshot.

Interesting, but not quite what I'm trying to do. Essentially I'd like to use snapshots as a sort of quick online backup to guard against users nuking their files. Of course we do backups nightly, but restoring from a snapshot is much quicker than messing with tapes.

So, depending on how snapshots actually work and the performance penalty they incur and various other bits, I was thinking of doing a daily or twice-daily snapshot, keeping them around for a few days, then deleting them. But that entails having around 100 snapshots sitting around, and I don't know if there are performance problems with doing that.

- J<

Well, I won't get into the debate on whether it's better to protect the users or educate them (ie easy to access backups make users sloppy kind of argument), but were I tasked with such a thing, I'd do it thus:

I'm assuming a situation were you have "working hours", but if that's not the case, then only the times have to be modified:

Establish a cron-job to run at <work-start-time>, <work-start+4> and <work-end>. This cron job calls a script that expires the old snap and creates a new one. If you want (for whatever reason) to keep multiple old snapshots around (and it can be done I believe) you would just make the script smarter and have it expire <current-<x>> snapshots.

I'd imagine there is, since the snap doesn't "store" all the data, but rather it stores the unchanged version of changed data, meaning that as long as the snap is active, it's getting writes. You may be able to deactivate a snap and keep it around, but I don't know how that works.

Share and Enjoy. 

p> Well, I won't get into the debate on whether it's better to p> protect the users or educate them (ie easy to access backups make p> users sloppy kind of argument),

Uh, well, I've been an administrator for 13 years and a computer user for 24, and I still accidentally overwrite a file or delete a directory on occasion. And when this happens, I don't particularly want to have to mess with the tape drive. The the LVM system already includes a method for making snapshots; why not make use of it? Educating users doesn't really come into it.

p> Establish a cron-job to run at <work-start-time>, <work-start+4> p> and <work-end>. This cron job calls a script that expires the old p> snap and creates a new one.

Well, I understand how to do it. I was simply asking if anyone had already done it so I can save myself some work. (It's actually mildly complicated. You have to parse various /proc files to get the LVM state and figure out which snapshots you have active and whether any of them have filled and thus become unusable. You also have to update /etc/fstab so a reboot doesn't lose your snapshots.)

- J<

It's a nice idea, but with the current implementation there's no way I'd want 100 snapshots of the same origin hanging around.

- Joe

JT> It's a nice idea, but with the current implementation there's no JT> way I'd want 100 snapshots of the same origin hanging around.

They wouldn't all be of the same LV, fortunately; I split my VG up into lots of manageable chunks. (Large users get a volume to themselves, small users get grouped into various user volumes.) But still, I assume that you have some details of performance with the current snapshot implementation. Is it that bad?

- J<

Well, it does

- copy on write (COW) synchronously *before* et allows the write to the original LV to proceed

- do multiple reads in case you`ve got multiple snapshots

- does the COWs for multiple snapshots sequentially

This leads to an unecessary read per additional snapshot on the original LV and therefore to something around n-times delay in case you've got n snapshots.

We will have an asynchrounous implementation which avoids multiple reads with LVM2.

Regards,
Heinz    -- The LVM Guy --

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
[email protected]                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-