Skip to contentSkip to conversation

btrfs send / receive via netcat, fails halfway?

DISCUSSION [email protected] 4 participants 6 messages Jan 10, 2021 Btrfs file system

lore
Share & tools
lore
#0 lore raw

­I'm trying to transfer a btrfs snapshot via the network.

First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.

btrfs subvolume delete /mnt/rec/snapshots/*
receive side:

# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0

sending side:

# btrfs send  /mnt/send/snapshots/0 | nc -v 127.0.0.1 6790
At subvol /mnt/send/snapshots/0
localhost [127.0.0.1] 6790 (hnmp) open

Second attempt: both nc programs exit ok at snapshot 0,1,2, but snapshot3 fails halfway, and 4 fails, as 3 is not complete. receive side:

# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 1
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 2
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 3
read(net): Connection reset by peer
ERROR: short read from stream: expected 49183 read 10450
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 4
ERROR: cannot find parent subvolume
write(stdout): Broken pipe

sending side:

# btrfs send  /mnt/send/snapshots/0 | nc -v -c 127.0.0.1 6790
At subvol /mnt/send/snapshots/0
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/0 /mnt/send/snapshots/1 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/1
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/1 /mnt/send/snapshots/2 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/2
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/2 /mnt/send/snapshots/3 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/3
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/3 /mnt/send/snapshots/4 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/4
localhost [127.0.0.1] 6790 (hnmp) open
write(net): Connection reset by peer

---

Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!

#1 lore raw
Quoted 34 lines · On 2021-01-10 11:34, [email protected] wrote:

­I'm trying to transfer a btrfs snapshot via the network.

First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.

btrfs subvolume delete /mnt/rec/snapshots/*
receive side:

# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0

sending side:

# btrfs send  /mnt/send/snapshots/0 | nc -v 127.0.0.1 6790
At subvol /mnt/send/snapshots/0
localhost [127.0.0.1] 6790 (hnmp) open

Second attempt: both nc programs exit ok at snapshot 0,1,2, but snapshot3 fails halfway, and 4 fails, as 3 is not complete. receive side:

# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 1
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 2
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 3
read(net): Connection reset by peer
ERROR: short read from stream: expected 49183 read 10450
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 4
ERROR: cannot find parent subvolume
write(stdout): Broken pipe

sending side:

Quoted 19 lines
# btrfs send  /mnt/send/snapshots/0 | nc -v -c 127.0.0.1 6790
At subvol /mnt/send/snapshots/0
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/0 /mnt/send/snapshots/1 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/1
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/1 /mnt/send/snapshots/2 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/2
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/2 /mnt/send/snapshots/3 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/3
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/3 /mnt/send/snapshots/4 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/4
localhost [127.0.0.1] 6790 (hnmp) open
write(net): Connection reset by peer

Hi,

Haven't used netcat for btrfs send, so I am not sure why you have these issues. Have you tried mbuffer instead? This is what I use when I don't need to go over ssh.

Mbuffer[1] allows you to specify remote host and port. It also has an async buffer so it should be a bit faster too. In fact, it is useful on a localhost too, for example when sending snaps to a slower local backup media.

Receiving side:

# mbuffer -I port | btrfs receive /mnt/rec/snapshots

Sending side:

# btrfs send /mnt/send/snapshots/0 | mbuffer -O remotehost:port

Good Luck!

//Forza

[1]https://www.maier-komor.de/mbuffer.html

#2 lore raw
In reply to [email protected]
Quoted 3 lines · On Sun, 10 Jan 2021 11:34:27 +0100 " " <[email protected]> wrote:

­I'm trying to transfer a btrfs snapshot via the network.

First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.

It is a common annoyance that NC doesn't exit in such scenario and needs to be Ctrl-C'ed after verifying that the transfer is over.

Instead, at host2 try:

ssh host1 "btrfs send ..." | btrfs receive ...

Also much more secure.

Signature · 2 lines
With respect,
Roman
#3 lore raw

It is a common annoyance that NC doesn't exit in such scenario and needs to be
Ctrl-C'ed after verifying that the transfer is over.

Instead, at host2 try:

ssh host1 "btrfs send ..." | btrfs receive ...

Also much more secure.

Signature · 9 lines
With respect,
Roman

I was practicing on a test system, the real transfer was supposed to be between two systems on a LAN. I didn't want encryption, as my system then only is capable of sending 30MB/sec, instead of the 110MB/sec over the 1GB network.


---

Take your mailboxes with you. Free, fast and secure Mail &amp; Cloud: https://www.eclipso.eu - Time to change!
#4 lore raw
In reply to [email protected]
Quoted 6 lines · On Sun, Jan 10, 2021 at 11:34:27AM +0100, wrote:

­I'm trying to transfer a btrfs snapshot via the network.

First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.

btrfs subvolume delete /mnt/rec/snapshots/*
receive side:

Quoted 7 lines
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0

sending side:

# btrfs send  /mnt/send/snapshots/0 | nc -v 127.0.0.1 6790
At subvol /mnt/send/snapshots/0
localhost [127.0.0.1] 6790 (hnmp) open

Use -q 15 on the sending side. That will exit after 15 seconds of
no activity from the send process.

Quoted 12 lines

Second attempt: both nc programs exit ok at snapshot 0,1,2, but snapshot3 fails halfway, and 4 fails, as 3 is not complete. receive side:

# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 1
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 2
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 3
read(net): Connection reset by peer
ERROR: short read from stream: expected 49183 read 10450

This failed because of a network disconnect.

Quoted 4 lines
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 4
ERROR: cannot find parent subvolume
write(stdout): Broken pipe

This is expected because the previous one failed.

There's no btrfs problem here. You just need better error handling
(to retry a failed transfer, for example).

Hugo.

Signature · 4 lines
Hugo Mills             | How do you become King? You stand in the marketplace
hugo@... carfax.org.uk | and announce you're going to tax everyone. If you
http://carfax.org.uk/  | get out alive, you're King.
PGP: E2AB1DE4          |                                        Harry Harrison
#5 lore raw

By the way, Cedric, your SMTP server is rejecting mail from mine
with a "forged HELO" error. I'm not sure why, but I've not knowingly encountered this with anyone else's mail server.

Hugo.

Signature · 4 lines
Hugo Mills             | The last man on Earth sat in a room. Suddenly, there
hugo@... carfax.org.uk | was a knock at the door.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                        Frederic Brown