Unexpected: send/receive much slower than rsync ?

I'm trying to update from an old snapshot of a directory to a new one using send/receive. It seems a great deal slower than I was expecting, perhaps much slower than rsync and has been running for hours. Everything looks ok with how I set up the snapshots, and there are no error messages, but I don't think it should be running this long. The directory structure is rather complex, so that may have something to do with it. It contains reflinked incremental backups of root file systems from a number of machines. It should not actually be very large due to the reflinks.

Sending the old version of the snapshot for the directory did not seem to take this long, and I expected the "send -p <old> <new>" to be much faster than that.

I tried running the "send" and "receive" with "-vv" to get more detail on what was happening.

I had thought that btrfs send/receive purely dealt with block/extent level changes.

I could be mistaken, but it seems that btrfs receive actually does a great deal of manipulation at the level of individual files, and rather less efficiently than rsync at that. I am not sure whether it is using system calls to do this, or actual shell commands themselves. I see quite a bit of what looks like file level manipulation in the verbose output. It is indeed very fast for simple directory trees even with very large files. However, it seems to be far slower than rsync with moderately complex directory trees, even if no large files are present.

I hope I'm overlooking something, and that this is not actually the case. Any ideas on this ?

J. Hart

Hi,

I am not an expert just a btrfs user who uses send/receive quite frequently but I am pretty sure your problem is not on the receive side but on the send end.
Can you check with e.g. iotop if receive is writing anything to the disk or if it's just waiting for send?
How much is send reading from disk and how much memory is it allocating?

I am asking this because I reckon your problem is caused by the way clone detection is done in send. There is a proposed patch
https://patchwork.kernel.org/patch/9245287/
that addresses the problem. This did indeed help me when I had a similar problem when trying to send a previously deduplicated filesystem!

Greetings
Hermann