Skip to contentSkip to conversation

Receive on same subvolume

DISCUSSION Matthew Lai 3 participants 8 messages Jan 29, 2014 – Feb 4, 2014 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hello,

Is this supposed to work? (/data is the root volume, /data/a is a subvolume)

btrfs subvolume snapshot /data/a /data/b

# make some changes in b
btrfs send -p /data/a /data/b > delta
btrfs receive /data/a < delta

I'm getting "ERROR: could not find parent subvolume" on receive.

What I'm trying to do is to back up using send/receive, but I don't have 50% free space, and (please correct me if I'm wrong) since receive doesn't do deduplication, I want to use snapshot to do the initial bootstrapping, instead of send/receive without a parent.

Thanks!
Matthew

#1 lore raw

Hi Matthew,

I'm not sure what you are trying to achive. Couldn't you simply do another snapshot of the subvolume? I don't understand why you want to use send/receive on the same subvolume to be honest.

Regards,
Felix

Quoted 22 lines · Matthew Lai

Hello,

Is this supposed to work? (/data is the root volume, /data/a is a subvolume)

btrfs subvolume snapshot /data/a /data/b

# make some changes in b
btrfs send -p /data/a /data/b > delta
btrfs receive /data/a < delta

I'm getting "ERROR: could not find parent subvolume" on receive.

What I'm trying to do is to back up using send/receive, but I don't have 50% free space, and (please correct me if I'm wrong) since receive doesn't do deduplication, I want to use snapshot to do the initial bootstrapping, instead of send/receive without a parent.

Thanks!
Matthew
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
In reply to Matthew Lai
Quoted 4 lines · Matthew Lai

Hello,

Is this supposed to work? (/data is the root volume, /data/a is a subvolume)

Quoted 8 lines

btrfs subvolume snapshot /data/a /data/b

# make some changes in b
btrfs send -p /data/a /data/b > delta
btrfs receive /data/a < delta

I'm getting "ERROR: could not find parent subvolume" on receive.

What I'm trying to do is to back up using send/receive, but I don't have 50% free space, and (please correct me if I'm wrong) since receive doesn't do deduplication, I want to use snapshot to do the initial bootstrapping, instead of send/receive without a parent.

I think you've oversimplified your commands, because it looks like you're using send/receive on the same file system. But if it's a backup, necessarily you'd have to be sending the subvolume(s) to another file system on another disk (either on the same system or remotely). So that needs some clarification.

Also, btrfs send requires subvolumes to be read only. Are they?

And btrfs incremental receive expects the identical parent already on the destination. Is it?

Also, while I'm not certain it matters, man page says to use -f to specify files. I haven't tested < and >. But then also the step where you create this intermediate snapshot file isn't necessary, just combine the send receive commands through pipe.

https://btrfs.wiki.kernel.org/index.php/Incremental_Backup

Chris Murphy

#3 lore raw

Thanks. I should clarify what I'm trying to do.

I'm trying to use btrfs send for backup, without having another btrfs volume.

So the initial backup is a complete send, piped to Amazon Glacier (so my machine never has the whole file, and doesn't have space for one).

At the same time I'm keeping a snapshot of the current volume.

On the next incremental backup, I would use the first snapshot as the parent, and send the differences to Glacier again (without having the entire file on the system at any time).

It looks like the problem now is the sent file can't be applied to the original volume (for restore).

Thanks
Matthew

Quoted 31 lines · Chris Murphy
Quoted 11 lines · Matthew Lai

Hello,

Is this supposed to work? (/data is the root volume, /data/a is a subvolume)

btrfs subvolume snapshot /data/a /data/b

# make some changes in b
btrfs send -p /data/a /data/b > delta
btrfs receive /data/a < delta

I'm getting "ERROR: could not find parent subvolume" on receive.
What I'm trying to do is to back up using send/receive, but I don't have 50% free space, and (please correct me if I'm wrong) since receive doesn't do deduplication, I want to use snapshot to do the initial bootstrapping, instead of send/receive without a parent.

I think you've oversimplified your commands, because it looks like you're using send/receive on the same file system. But if it's a backup, necessarily you'd have to be sending the subvolume(s) to another file system on another disk (either on the same system or remotely). So that needs some clarification.

Also, btrfs send requires subvolumes to be read only. Are they?

And btrfs incremental receive expects the identical parent already on the destination. Is it?

Also, while I'm not certain it matters, man page says to use -f to specify files. I haven't tested < and >. But then also the step where you create this intermediate snapshot file isn't necessary, just combine the send receive commands through pipe.

https://btrfs.wiki.kernel.org/index.php/Incremental_Backup

Chris Murphy

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
Quoted 5 lines · Matthew Lai

Thanks. I should clarify what I'm trying to do.

I'm trying to use btrfs send for backup, without having another btrfs volume.

So the initial backup is a complete send, piped to Amazon Glacier (so my machine never has the whole file, and doesn't have space for one).

OK so you've use btrfs send piped to Glacier which creates a *file*, I'll call it "initial", not a navigable directory of files? Right?

Quoted 4 lines

At the same time I'm keeping a snapshot of the current volume.

On the next incremental backup, I would use the first snapshot as the parent, and send the differences to Glacier again (without having the entire file on the system at any time).

That's fine as long as the stdout from btrfs send ends up as a self contained file on Glacier. I'll call this "increment1"

Quoted 2 lines

It looks like the problem now is the sent file can't be applied to the original volume (for restore).

I'm counting two sent files: initial, increment1. I'm not sure which one you're applying. If you have the exact same read-only snapshot that the btrfs send file "initial" is based on, then you'd apply the increment1 to that read-only snapshot which will cause a new read-only snapshot to be created with the incremental data applied to it. The error you're getting sounds like the parent read-only snapshot isn't available?

Have you tried -vv flag to get more verbose error information when using btrfs receive?

Chris Murphy

#5 lore raw
Quoted 8 lines · Chris Murphy
Quoted 5 lines · Matthew Lai

Thanks. I should clarify what I'm trying to do.

I'm trying to use btrfs send for backup, without having another btrfs volume.

So the initial backup is a complete send, piped to Amazon Glacier (so my machine never has the whole file, and doesn't have space for one).

OK so you've use btrfs send piped to Glacier which creates a *file*, I'll call it "initial", not a navigable directory of files? Right?

That is correct.

Quoted 3 lines
Quoted 1 line

It looks like the problem now is the sent file can't be applied to the original volume (for restore).

I'm counting two sent files: initial, increment1. I'm not sure which one you're applying. If you have the exact same read-only snapshot that the btrfs send file "initial" is based on, then you'd apply the increment1 to that read-only snapshot which will cause a new read-only snapshot to be created with the incremental data applied to it. The error you're getting sounds like the parent read-only snapshot isn't available?

That is also correct. There are 2 sent files. I am trying to apply increment1, on a snapshot of the parent (that was used to create increment1).

I added -vv. Here is the test script for reproducing this entire setup.

-----------------------------------------------------------------------
#!/bin/sh

btrfs subvolume create data
btrfs subvolume snapshot -r data first_backup touch data/a
btrfs subvolume snapshot -r data second_backup btrfs send -p first_backup second_backup > increment btrfs subvolume snapshot first_backup first_backup_rw btrfs receive -vv first_backup_rw < increment
-----------------------------------------------------------------------

Output:
-----------------------------------------------------------------------
Create subvolume './data'
Create a readonly snapshot of 'data' in './first_backup'
Create a readonly snapshot of 'data' in './second_backup'
At subvol second_backup
Create a snapshot of 'first_backup' in './first_backup_rw'
At snapshot second_backup
receiving snapshot second_backup uuid=e6159a2a-3430-344a-a23d-b9bb83851a63, ctransid=28 parent_uuid=20c4ff66-a9ec-fc44-93c6-2c12637e95e6, parent_ctransid=26
ERROR: could not find parent subvolume
-----------------------------------------------------------------------

I would think applying the "patch" to first_backup_rw should succeed, because it's exactly the same as first_backup, which is the parent for the send, but it doesn't.

Thanks
Matthew

#6 lore raw
Quoted 7 lines · Matthew Lai
Quoted 6 lines · Chris Murphy
Quoted 4 lines · Matthew Lai

Thanks. I should clarify what I'm trying to do.

I'm trying to use btrfs send for backup, without having another btrfs volume.

Quoted 7 lines
Quoted 2 lines
Quoted 1 line

So the initial backup is a complete send, piped to Amazon Glacier (so my machine never has the whole file, and doesn't have space for one).

OK so you've use btrfs send piped to Glacier which creates a *file*, I'll call it "initial", not a navigable directory of files? Right?

That is correct.

Quoted 3 lines
Quoted 1 line

It looks like the problem now is the sent file can't be applied to the original volume (for restore).

I'm counting two sent files: initial, increment1. I'm not sure which one you're applying. If you have the exact same read-only snapshot that the btrfs send file "initial" is based on, then you'd apply the increment1 to that read-only snapshot which will cause a new read-only snapshot to be created with the incremental data applied to it. The error you're getting sounds like the parent read-only snapshot isn't available?

That is also correct. There are 2 sent files. I am trying to apply increment1, on a snapshot of the parent (that was used to create increment1).

I don't understand how you can apply increment1 to the snapshot of increment1; and also I don't understand how the parent is also increment1.

Quoted 27 lines

I added -vv. Here is the test script for reproducing this entire setup.

-----------------------------------------------------------------------
#!/bin/sh

btrfs subvolume create data
btrfs subvolume snapshot -r data first_backup touch data/a
btrfs subvolume snapshot -r data second_backup btrfs send -p first_backup second_backup > increment btrfs subvolume snapshot first_backup first_backup_rw btrfs receive -vv first_backup_rw < increment
-----------------------------------------------------------------------

Output:
-----------------------------------------------------------------------
Create subvolume './data'
Create a readonly snapshot of 'data' in './first_backup'
Create a readonly snapshot of 'data' in './second_backup'
At subvol second_backup
Create a snapshot of 'first_backup' in './first_backup_rw'
At snapshot second_backup
receiving snapshot second_backup uuid=e6159a2a-3430-344a-a23d-b9bb83851a63, ctransid=28 parent_uuid=20c4ff66-a9ec-fc44-93c6-2c12637e95e6, parent_ctransid=26
ERROR: could not find parent subvolume
-----------------------------------------------------------------------

Quoted 1 line

I would think applying the "patch" to first_backup_rw should succeed, because it's exactly the same as first_backup, which is the parent for the send, but it doesn't.

btrfs sub snap -r subvol.1 subvol
btrfs send subvol.1 -f <toglacier>/subvol.1.btrfs #write some more files to subvol
btrfs sub snap -r subvol.2 subvol
btrfs send -p subvol.1 subvol.2 -f <toglacier>/subvol.2.btrfs

#To make subvol.1 into subvol.2 by applying subvol.2.btrfs to subvol.1, the actual original subvol.1 must be present first or you need to "receive" it from subvol.1.btrfs first. And also, I'm pretty sure you can't have subvol.2 already present because receive must create it.

Again, I haven't tried > and < so I don't know they work. Have you tried -f to point to the files?

Chris Murphy

#7 lore raw
Quoted 54 lines · Chris Murphy
Quoted 7 lines · Matthew Lai
Quoted 6 lines · Chris Murphy
Quoted 4 lines · Matthew Lai

Thanks. I should clarify what I'm trying to do.

I'm trying to use btrfs send for backup, without having another btrfs volume.

Quoted 7 lines
Quoted 2 lines
Quoted 1 line

So the initial backup is a complete send, piped to Amazon Glacier (so my machine never has the whole file, and doesn't have space for one).

OK so you've use btrfs send piped to Glacier which creates a *file*, I'll call it "initial", not a navigable directory of files? Right?

That is correct.

Quoted 3 lines
Quoted 1 line

It looks like the problem now is the sent file can't be applied to the original volume (for restore).

I'm counting two sent files: initial, increment1. I'm not sure which one you're applying. If you have the exact same read-only snapshot that the btrfs send file "initial" is based on, then you'd apply the increment1 to that read-only snapshot which will cause a new read-only snapshot to be created with the incremental data applied to it. The error you're getting sounds like the parent read-only snapshot isn't available?

That is also correct. There are 2 sent files. I am trying to apply increment1, on a snapshot of the parent (that was used to create increment1).

I don't understand how you can apply increment1 to the snapshot of increment1; and also I don't understand how the parent is also increment1.

Quoted 26 lines

I added -vv. Here is the test script for reproducing this entire setup.

-----------------------------------------------------------------------
#!/bin/sh

btrfs subvolume create data
btrfs subvolume snapshot -r data first_backup touch data/a
btrfs subvolume snapshot -r data second_backup btrfs send -p first_backup second_backup > increment btrfs subvolume snapshot first_backup first_backup_rw btrfs receive -vv first_backup_rw < increment
-----------------------------------------------------------------------

Output:
-----------------------------------------------------------------------
Create subvolume './data'
Create a readonly snapshot of 'data' in './first_backup'
Create a readonly snapshot of 'data' in './second_backup'
At subvol second_backup
Create a snapshot of 'first_backup' in './first_backup_rw'
At snapshot second_backup
receiving snapshot second_backup uuid=e6159a2a-3430-344a-a23d-b9bb83851a63, ctransid=28 parent_uuid=20c4ff66-a9ec-fc44-93c6-2c12637e95e6, parent_ctransid=26
ERROR: could not find parent subvolume
-----------------------------------------------------------------------

Quoted 1 line

I would think applying the "patch" to first_backup_rw should succeed, because it's exactly the same as first_backup, which is the parent for the send, but it doesn't.

btrfs sub snap -r subvol.1 subvol
btrfs send subvol.1 -f <toglacier>/subvol.1.btrfs #write some more files to subvol
btrfs sub snap -r subvol.2 subvol
btrfs send -p subvol.1 subvol.2 -f <toglacier>/subvol.2.btrfs

Quoted 3 lines

#To make subvol.1 into subvol.2 by applying subvol.2.btrfs to subvol.1, the actual original subvol.1 must be present first or you need to "receive" it from subvol.1.btrfs first. And also, I'm pretty sure you can't have subvol.2 already present because receive must create it.

Again, I haven't tried > and < so I don't know they work. Have you tried -f to point to the files?

According to the manpage, -f is the same as output redirection.
"Output is normally written to stdout. To write to a file, use this option. An alternative would be to use pipes."

The reason why I can't use something like your sequence of commands (assuming the order of arguments for snap should be reversed) is that I want to be able to verify that the diff is correct, since there are still integrity problems with send/receive.

I was planning to do that by applying the "patch" to a snapshot of the parent right away, and make sure the patched volume is equal to the current snapshot (by trying another send, and making sure the output is 0).

Thanks
Matthew