Backup Command
Here is my proposed cron:
btrfs subvolume snapshot hex:///home /media/backups/snapshots/hex-{DATE}
rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=/media/backups/exclude-hex hex:///home /media/backups/hex
btrfs subvolume snapshot droog:///home /media/backups/snapshots/droog-{DATE}
rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=/media/backups/exclude-droog droog:///home /media/backups/droog
Comments? Criticisms?
Quoted 3 lines · On Monday 10 of January 2011 14:25:32 Carl Cook wrote:
Here is my proposed cron:
=20
btrfs subvolume snapshot hex:///home /media/backups/snapshots/hex-{DA=
TE}
Quoted 2 lines
=20
rsync --archive --hard-links --delete-during --delete-excluded --inpl=
ace
Quoted 1 line
--numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-hex hex:=
///home
Quoted 6 lines
/media/backups/hex
=20
btrfs subvolume snapshot droog:///home /media/backups/snapshots/droog-{DATE}
=20
rsync --archive --hard-links --delete-during --delete-excluded --inpl=
ace
Quoted 4 lines
--numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-droog droog:///home /media/backups/droog
=20
Comments? Criticisms?
This will make the dates associated with snapshots offset by how often = cron is=20
run.
In other words, if you run above script daily you will have data from=20 2011.01.01 in the hex-2011.01.02 directory.
I do save the current date, do a LVM snapshot on the source, rsync --in= place=20
data over and do a local snapshot naming the folder using the saved dat= e. This=20
way the date in the name of backup directory is exact to about a second= =2E
--=20
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawer=F3w 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 21 lines · Hubert Kario
Quoted 8 lines · On Monday 10 of January 2011 14:25:32 Carl Cook wrote:
Here is my proposed cron:
btrfs subvolume snapshot hex:///home /media/backups/snapshots/hex-{DATE}
rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=/media/backups/exclude-hex hex:///home /media/backups/hex
Quoted 8 lines
btrfs subvolume snapshot droog:///home
/media/backups/snapshots/droog-{DATE}rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=/media/backups/exclude-droog droog:///home /media/backups/droog
Comments? Criticisms?
This will make the dates associated with snapshots offset by how often cron is run.
Quoted 6 lines
In other words, if you run above script daily you will have data from 2011.01.01 in the hex-2011.01.02 directory.
I do save the current date, do a LVM snapshot on the source, rsync --inplace data over and do a local snapshot naming the folder using the saved date. This way the date in the name of backup directory is exact to about a second.
If you are mounting a LVM snapshot of an already mounted filesystem, would you be willing verify that it is really a snapshot that is mounted?
e.g. touch /mnt/live/its_alive && ls /mnt/snapshot/
I am nearly willing to bet it is not a snapshot.
--
ivan
Quoted 5 lines · On Tuesday, January 11, 2011 14:54:38 Ivan Lab=C3=A1th wrote:
Quoted 4 lines · Hubert Kario
Quoted 3 lines · On Monday 10 of January 2011 14:25:32 Carl Cook wrote:
Here is my proposed cron:
=20
btrfs subvolume snapshot hex:///home /media/backups/snapshots/hex-=
{DATE}
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
=20
rsync --archive --hard-links --delete-during --delete-excluded --i=
nplace
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
--numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-hex hex:///home /media/backups/hex
=20
btrfs subvolume snapshot droog:///home /media/backups/snapshots/droog-{DATE}
=20
rsync --archive --hard-links --delete-during --delete-excluded --i=
nplace
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
--numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-droog droog:///home /media/backups/droog
=20
Comments? Criticisms?=20
This will make the dates associated with snapshots offset by how of=
ten
Quoted 3 lines
Quoted 3 lines
cron is run.
=20
In other words, if you run above script daily you will have data fr=
om
Quoted 4 lines
Quoted 4 lines
2011.01.01 in the hex-2011.01.02 directory.
=20
I do save the current date, do a LVM snapshot on the source, rsync --inplace data over and do a local snapshot naming the folder using=
the
Quoted 1 line
Quoted 1 line
saved date. This way the date in the name of backup directory is ex=
act
Quoted 4 lines
Quoted 1 line
to about a second.
=20
If you are mounting a LVM snapshot of an already mounted filesystem, would you be willing verify that it is really a snapshot that is moun=
ted?
Quoted 4 lines
=20
e.g. touch/mnt/live/its_alive&& ls/mnt/snapshot/=20
I am nearly willing to bet it is not a snapshot.
well, by "LVM snapshot on the source" I meant:
1. do lvcreate --snapshot
2. mount newly created volume=20
3. use the new directory as the base for rsync
4. arrange umount and destruction of the snapshot after rsync completes= (no=20
matter if it was successful)
and this will in fact not make the "its_alive" visible in /mnt/snapshot
You have to use this procedure if you use LVM snapshots for backup no m= atter=20
to where do you copy data. That's why I shortened it to a single point = -- it's=20
not the part that is important from btrfs perspective.
Regards.
--=20
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawer=C3=B3w 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 6 lines · Hubert Kario
Quoted 5 lines · On Tuesday, January 11, 2011 14:54:38 Ivan Lab=C3=A1th wrote:
Quoted 4 lines · Hubert Kario
Quoted 3 lines · On Monday 10 of January 2011 14:25:32 Carl Cook wrote:
Here is my proposed cron:
btrfs subvolume snapshot hex:///home /media/backups/snapshots/hex-=
{DATE}
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
rsync --archive --hard-links --delete-during --delete-excluded --i=
nplace
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
--numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-hex hex:///home /media/backups/hex
btrfs subvolume snapshot droog:///home
/media/backups/snapshots/droog-{DATE}rsync --archive --hard-links --delete-during --delete-excluded --i=
nplace
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
--numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-droog droog:///home /media/backups/droog
Comments? Criticisms?
This will make the dates associated with snapshots offset by how of=
ten
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
cron is run.
In other words, if you run above script daily you will have data fr=
om
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
2011.01.01 in the hex-2011.01.02 directory.
I do save the current date, do a LVM snapshot on the source, rsync --inplace data over and do a local snapshot naming the folder using=
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
saved date. This way the date in the name of backup directory is ex=
act
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
to about a second.
If you are mounting a LVM snapshot of an already mounted filesystem, would you be willing verify that it is really a snapshot that is mou=
nted?
Quoted 10 lines
Quoted 4 lines
e.g. touch
/mnt/live/its_alive&& ls/mnt/snapshot/I am nearly willing to bet it is not a snapshot.
=20
well, by "LVM snapshot on the source" I meant:
1. do lvcreate --snapshot
2. mount newly created volume=20
3. use the new directory as the base for rsync
4. arrange umount and destruction of the snapshot after rsync complet=
es (no=20
Quoted 3 lines
matter if it was successful)
=20
and this will in fact not make the "its_alive" visible in /mnt/snapsh=
ot
Quoted 2 lines
=20
You have to use this procedure if you use LVM snapshots for backup no=
matter=20
Quoted 1 line
to where do you copy data. That's why I shortened it to a single poin=
t -- it's=20
Quoted 3 lines
not the part that is important from btrfs perspective. =20
Regards.
The point I was trying to make is: it does not work with btrfs. Try the above with a btrfs and you will be surprised.
If the source volume uses another filesystem, it should work properly.
regards,
ivan
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 23 lines · On Tuesday, January 11, 2011 15:33:38 Ivan Lab=C3=A1th wrote:
Quoted 22 lines · Hubert Kario
Quoted 21 lines · On Tuesday, January 11, 2011 14:54:38 Ivan Lab=C3=A1th wrote:
Quoted 20 lines · Hubert Kario
Quoted 19 lines · On Monday 10 of January 2011 14:25:32 Carl Cook wrote:
Here is my proposed cron:
=20
btrfs subvolume snapshot hex:///home /media/backups/snapshots/hex-{DATE}
=20
rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-hex hex:///home /media/backups/hex
=20
btrfs subvolume snapshot droog:///home /media/backups/snapshots/droog-{DATE}
=20
rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-droog droog:///home /media/backups/droog
=20
Comments? Criticisms?
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
=20
This will make the dates associated with snapshots offset by how =
often
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
cron is run.
=20
In other words, if you run above script daily you will have data =
from
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
2011.01.01 in the hex-2011.01.02 directory.
=20
I do save the current date, do a LVM snapshot on the source, rsyn=
c
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
--inplace data over and do a local snapshot naming the folder usi=
ng the
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
saved date. This way the date in the name of backup directory is =
exact
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
to about a second.
=20
If you are mounting a LVM snapshot of an already mounted filesyste=
m,
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
would you be willing verify that it is really a snapshot that is mounted?
=20
e.g. touch/mnt/live/its_alive&& ls/mnt/snapshot/=20
I am nearly willing to bet it is not a snapshot.
Quoted 6 lines
Quoted 6 lines
=20
well, by "LVM snapshot on the source" I meant:
1. do lvcreate --snapshot
2. mount newly created volume
3. use the new directory as the base for rsync
4. arrange umount and destruction of the snapshot after rsync compl=
etes
Quoted 3 lines
Quoted 3 lines
(no matter if it was successful)
=20
and this will in fact not make the "its_alive" visible in /mnt/snap=
shot
Quoted 2 lines
Quoted 2 lines
=20
You have to use this procedure if you use LVM snapshots for backup =
no
Quoted 1 line
Quoted 1 line
matter to where do you copy data. That's why I shortened it to a si=
ngle
Quoted 1 line
Quoted 1 line
point -- it's not the part that is important from btrfs perspective=
=2E
Quoted 6 lines
Quoted 2 lines
=20
Regards.=20
The point I was trying to make is: it does not work with btrfs. Try the above with a btrfs and you will be surprised.
If the source volume uses another filesystem, it should work properly=
=2E
Quoted 3 lines
=20
regards,
ivan
Yes, you are right, but I don't see a point in using LVM snapshots with=
btrfs,=20
after all the ability to snapshot it on fs level is one of its defining= =20
features...
--=20
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawer=C3=B3w 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 18 lines · Hubert Kario
Quoted 17 lines · On Tuesday, January 11, 2011 15:33:38 Ivan Lab=C3=A1th wrote:
Quoted 16 lines · Hubert Kario
Quoted 15 lines · On Tuesday, January 11, 2011 14:54:38 Ivan Lab=C3=A1th wrote:
Quoted 14 lines · Hubert Kario
Quoted 13 lines · On Monday 10 of January 2011 14:25:32 Carl Cook wrote:
Here is my proposed cron:
btrfs subvolume snapshot hex:///home
/media/backups/snapshots/hex-{DATE}rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-hex hex:///home /media/backups/hex
btrfs subvolume snapshot droog:///home
/media/backups/snapshots/droog-{DATE}
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=3D/media/backups/exclude-droog droog:///home /media/backups/droog
Comments? Criticisms?
This will make the dates associated with snapshots offset by how =
often
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
cron is run.
In other words, if you run above script daily you will have data =
from
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
2011.01.01 in the hex-2011.01.02 directory.
I do save the current date, do a LVM snapshot on the source, rsyn=
c
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
--inplace data over and do a local snapshot naming the folder usi=
ng the
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
saved date. This way the date in the name of backup directory is =
exact
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
to about a second.
If you are mounting a LVM snapshot of an already mounted filesyste=
m,
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
would you be willing verify that it is really a snapshot that is mounted?
e.g. touch
/mnt/live/its_alive&& ls/mnt/snapshot/
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
I am nearly willing to bet it is not a snapshot.
well, by "LVM snapshot on the source" I meant:
1. do lvcreate --snapshot
2. mount newly created volume
3. use the new directory as the base for rsync
4. arrange umount and destruction of the snapshot after rsync compl=
etes
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
(no matter if it was successful)
and this will in fact not make the "
its_alive" visible in /mnt/snap=
shot
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
You have to use this procedure if you use LVM snapshots for backup =
no
Quoted 1 line
Quoted 1 line
Quoted 1 line
matter to where do you copy data. That's why I shortened it to a si=
ngle
Quoted 1 line
Quoted 1 line
Quoted 1 line
point -- it's not the part that is important from btrfs perspective=
=2E
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Regards.
The point I was trying to make is: it does not work with btrfs. Try the above with a btrfs and you will be surprised.
If the source volume uses another filesystem, it should work properl=
y.
Quoted 5 lines
Quoted 3 lines
regards,
ivan=20
Yes, you are right, but I don't see a point in using LVM snapshots wi=
th btrfs,=20
Quoted 1 line
after all the ability to snapshot it on fs level is one of its defini=
ng=20
Quoted 2 lines
features...
=20
Yes, silently mounting another filesystem is the right thing to do. When there already is a mounted filesystem probably containing nearly identical data, the kernel should not bother allocating separate struct= ures
for yet another copy of it. The user probably won't notice anyway.
--
ivan
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Apparently my command is no good. No one ever responded to my question below, so maybe I'm not smart enough to ask the right way.
I can ssh from the backup server to the HTPC, but it crashes with a 'dirty' error:
# rsync --progress --archive --hard-links --links --delete-during --delete-excluded --inplace --numeric-ids -e "ssh -l root" root@hex:///home /media/backups/hex-home root@hex's password: protocol version mismatch -- is your shell clean? (see the rsync man page for an explanation) rsync error: protocol incompatibility (code 2) at compat.c(173) [Receiver=3.0.7]
"see the rsync man page for an explanation"? No idea where, but it's sure not in the first third. I have the same rsync and ssh on each.
Quoted 14 lines · On Mon 10 January 2011 05:25:32 Carl Cook wrote:
Here is my proposed cron:
btrfs subvolume snapshot hex:///home /media/backups/snapshots/hex-{DATE}
rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=/media/backups/exclude-hex hex:///home /media/backups/hex
btrfs subvolume snapshot droog:///home /media/backups/snapshots/droog-{DATE}
rsync --archive --hard-links --delete-during --delete-excluded --inplace --numeric-ids -e ssh --exclude-from=/media/backups/exclude-droog droog:///home /media/backups/droog
Comments? Criticisms?
On Fri, Jan 14, 2011 at 7:17 PM, Carl Cook <[email protected]> wro= te:
Quoted 2 lines
Apparently my command is no good. =A0No one ever responded to my ques=
tion below, so maybe I'm not smart enough to ask the right way.
Quoted 2 lines
I can ssh from the backup server to the HTPC, but it crashes with a '=
dirty' error:
Quoted 1 line
# rsync --progress --archive --hard-links --links --delete-during --d=
elete-excluded --inplace --numeric-ids -e "ssh -l root" root@hex:///hom= e /media/backups/hex-home
Quoted 4 lines
root@hex's password:
protocol version mismatch -- is your shell clean?
(see the rsync man page for an explanation)
rsync error: protocol incompatibility (code 2) atcompat.c(173)[Rece=
iver=3D3.0.7]
Quoted 2 lines
"see the rsync man page for an explanation"? =A0No idea where, but it=
's sure not in the first third. =A0I have the same rsync and ssh on eac= h.
rsync can speak ssh already, you shouldn't need to provide '-e "ssh -l root"' unless you're doing something odd. That said, try: '-e "ssh -l root -e none"' to disable ssh's escape character.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html