Btrfs-progs: some receive related patches
cover Btrfs-progs: some receive related patches
Most fixes are trivial.
The one from Alex is fixing a real bug that several users have reported. Alex sent the patch half a year ago and it was not yet integrated.
The patch "Use /proc/mounts instead of /etc/mtab" is a repost.
The patch "btrfs-receive optionally honors the end-cmd" is a preparation step to allow backup tools to multiplex a single communication stream (e.g. a single TCP stream) to carry multiple Btrfs send/receive streams and a request/response handshake. It basically corrects the handling of the "end-cmd" in the send/receive stream. Without, only the closing of the stream (EOF) was terminating the receiver. The change is fully compatible as it is explained in more detail in the commit itself. Unfortunately the change to the sender is only compatible together with a kernel change, it is therefore not part of this patch set.
Everything is also ongit://btrfs.giantdisaster.de/git/btrfs-progs recv1
Alex Lyakas (1):
btrfs-progs: Fix the receive code pathing
Stefan Behrens (16):
Btrfs-progs: Use /proc/mounts instead of /etc/mtab
Btrfs-progs: ignore subvols above BTRFS_LAST_FREE_OBJECTID
Btrfs-progs: close file descriptor in cmds-send.c
Btrfs-progs: fix a small memory leak in btrfs-list.c
Btrfs-progs: add a function to free subvol_uuid_search memory
Btrfs-progs: cleanup subvol_uuid_search memory in btrfs send/receive
Btrfs-progs: free memory and close file descriptor in btrfs receive
Btrfs-progs: Set the root-id for received subvols in btrfs receive
Btrfs-progs: btrfs-receive: different levels (amount) of debug output
Btrfs-progs: small parent_subvol cleanup for cmds-receive.c
Btrfs-progs: fix bug in find_root_gen
Btrfs-progs: btrfs-receive optionally honors the end-cmd
Btrfs-progs: don't allocate one byte too much each time
Btrfs-progs: Fix that BTRFS_FSID_SIZE is used instead of
BTRFS_UUID_SIZE
Btrfs-progs: remove some unused code
Btrfs-progs: allow to receive to relative directories
btrfs-list.c | 14 ++--- cmds-receive.c | 164 ++++++++++++++++++++++++++++++++++++++++++--------------- cmds-send.c | 12 ++++- send-stream.c | 9 ++-- send-stream.h | 3 +- send-utils.c | 49 +++++++++++++++-- send-utils.h | 2 + 7 files changed, 194 insertions(+), 59 deletions(-)
Signature · 1 line
1.8.2.1
PATCH 1/17 Btrfs-progs: Use /proc/mounts instead of /etc/mtab
/etc/mtab is not working correctly in situations where multiple mount namespaces are used. Use /proc/mounts instead like the rest of the code is doing it.
--- cmds-send.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
cmds-send.c +1 -1
@@ -70,7 +70,7 @@ int find_mount_root(const char *path, char **mount_root)return -errno;close (fd); - mnttab =fopen ("/etc/mtab" ,"r" ); + mnttab =fopen ("/proc/mounts" ,"r" );while ((ent =getmntent (mnttab))) { len =strlen (ent->mnt_dir);if (strncmp (ent->mnt_dir, path, len) ==0 ) {
Signature · 1 line
1.8.2.1
PATCH 2/17 Btrfs-progs: ignore subvols above BTRFS_LAST_FREE_OBJECTID
In reply to Stefan Behrens--- send-utils.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
send-utils.c +1 -1
@@ -226,7 +226,7 @@ int subvol_uuid_search_init(int mnt_fd, struct subvol_uuid_search *s)if ((sh->objectid !=5 && sh->objectid <BTRFS_FIRST_FREE_OBJECTID ) || - sh->objectid ==BTRFS_FREE_INO_OBJECTID ) + sh->objectid >BTRFS_LAST_FREE_OBJECTID ) goto skip;if (sh->type ==BTRFS_ROOT_ITEM_KEY ) {
Signature · 1 line
1.8.2.1
PATCH 3/17 Btrfs-progs: close file descriptor in cmds-send.c
In reply to Stefan Behrensvalgrind found this very obvious issue.
--- cmds-send.c | 1 + 1 file changed, 1 insertion(+)
cmds-send.c +1 -0
@@ -82,6 +82,7 @@ int find_mount_root(const char *path, char **mount_root) } } } +fclose (mnttab); *mount_root =realpath (longest_match,NULL );free (longest_match);
Signature · 1 line
1.8.2.1
PATCH 4/17 Btrfs-progs: fix a small memory leak in btrfs-list.c
In reply to Stefan Behrensvalgrind found this very obvious issue.
--- btrfs-list.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
btrfs-list.c +6 -6
@@ -577,13 +577,13 @@ static int resolve_root(struct root_lookup *rl, struct root_info *ri, add_len =strlen (found->path); -/* room for / and for null */ - tmp =malloc (add_len +2 + len); -if (!tmp) { -perror ("malloc failed" ); -exit (1 ); - }if (full_path) { +/* room for / and for null */ + tmp =malloc (add_len +2 + len); +if (!tmp) { +perror ("malloc failed" ); +exit (1 ); + }memcpy (tmp + add_len +1 , full_path, len); tmp[add_len] ='/' ;memcpy (tmp, found->path, add_len);
Signature · 1 line
1.8.2.1
PATCH 5/17 Btrfs-progs: add a function to free subvol_uuid_search memory
In reply to Stefan BehrensThere was no way to free the memory that was used for the subvol_uuid_search functions. Since this is part of the libbtrfs, add such a cleanup function.
--- send-utils.c | 23 +++++++++++++++++++++++ send-utils.h | 1 + 2 files changed, 24 insertions(+)
send-utils.c +23 -0
@@ -304,6 +304,29 @@ out:return ret; } +/* +* It's safe to call this function even without the subvol_uuid_search_init() +* call before as long as the subvol_uuid_search structure is all-zero. +*/ +void subvol_uuid_search_finit (struct subvol_uuid_search *s) +{ +struct rb_root *root = &s->root_id_subvols; +struct rb_node *node; + +while ((node =rb_first (root))) { +struct subvol_info *entry = + rb_entry(node, struct subvol_info, rb_root_id_node); + +free (entry->path); +rb_erase (node, root); +free (entry); + } + + s->root_id_subvols =RB_ROOT ; + s->local_subvols =RB_ROOT ; + s->received_subvols =RB_ROOT ; + s->path_subvols =RB_ROOT ; +}char *path_cat (const char *p1,const char *p2) {
send-utils.h +1 -0
@@ -62,6 +62,7 @@ struct subvol_uuid_search { };int subvol_uuid_search_init (int mnt_fd,struct subvol_uuid_search *s); +void subvol_uuid_search_finit (struct subvol_uuid_search *s); struct subvol_info *subvol_uuid_search(struct subvol_uuid_search *s, u64 root_id, const u8 *uuid, u64 transid, const char *path,
Signature · 1 line
1.8.2.1
PATCH 6/17 Btrfs-progs: cleanup subvol_uuid_search memory in btrfs send/receive
In reply to Stefan BehrensCall the cleanup function that was introduced with the other commit.
--- cmds-receive.c | 1 + cmds-send.c | 2 ++ 2 files changed, 3 insertions(+)
cmds-receive.c +1 -0
@@ -824,6 +824,7 @@ int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd) ret =0 ; out: +subvol_uuid_search_finit (&r->sus);return ret; }
cmds-send.c +2 -0
@@ -458,6 +458,7 @@ int cmd_send_start(int argc, char **argv) goto out; }add_clone_source (&send, root_id); +subvol_uuid_search_finit (&send.sus);free (subvol); full_send =0 ;break ;
@@ -630,6 +631,7 @@ int cmd_send_start(int argc, char **argv) out:if (send.mnt_fd >=0 )close (send.mnt_fd); +subvol_uuid_search_finit (&send.sus);return ret; }
Signature · 1 line
1.8.2.1
PATCH 7/17 Btrfs-progs: free memory and close file descriptor in btrfs receive
In reply to Stefan BehrensNothing really important since this is not part of the library and at the end exit() is called.
--- cmds-receive.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-)
cmds-receive.c +24 -3
@@ -151,6 +151,7 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid, r->parent_subvol =NULL ; r->cur_subvol->path =strdup (path); +free (r->full_subvol_path); r->full_subvol_path =path_cat (r->root_path, path);fprintf (stderr,"At subvol %s\n" , path);
@@ -196,6 +197,7 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, r->parent_subvol =NULL ; r->cur_subvol->path =strdup (path); +free (r->full_subvol_path); r->full_subvol_path =path_cat (r->root_path, path);fprintf (stderr,"At snapshot %s\n" , path);
@@ -803,9 +805,7 @@ int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd) ret =subvol_uuid_search_init (r->mnt_fd, &r->sus);if (ret <0 ) -return ret; - - r->write_fd =-1 ; + goto out;while (!end) { ret =btrfs_read_and_process_send_stream (r_fd, &send_ops, r);
@@ -824,7 +824,26 @@ int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd) ret =0 ; out: +if (r->write_fd !=-1 ) { +close (r->write_fd); + r->write_fd =-1 ; + } +free (r->root_path); + r->root_path =NULL ; +free (r->write_path); + r->write_path =NULL ; +free (r->full_subvol_path); + r->full_subvol_path =NULL ; +if (r->cur_subvol) { +free (r->cur_subvol->path); +free (r->cur_subvol); + r->cur_subvol =NULL ; + }subvol_uuid_search_finit (&r->sus); +if (r->mnt_fd !=-1 ) { +close (r->mnt_fd); + r->mnt_fd =-1 ; + }return ret; }
@@ -839,6 +858,8 @@ static int do_cmd_receive(int argc, char **argv)int ret;memset (&r,0 ,sizeof (r)); + r.mnt_fd =-1 ; + r.write_fd =-1 ;while ((c =getopt (argc, argv,"vf:" )) !=-1 ) {switch (c) {
Signature · 1 line
1.8.2.1
PATCH 8/17 Btrfs-progs: Set the root-id for received subvols in btrfs receive
In reply to Stefan BehrensWhen an entry was added to the subvol search tree, the root_id was always 0 (not set at all) and therefore only the first one was added, all the others had been ignored. This commit adds a function to retrieve the root_id for a given fd, and this function is used to set the root_id before the entry is added.
--- cmds-receive.c | 3 +++ send-utils.c | 20 ++++++++++++++++++++ send-utils.h | 1 + 3 files changed, 24 insertions(+)
cmds-receive.c +3 -0
@@ -125,6 +125,9 @@ static int finish_subvol(struct btrfs_receive *r) goto out; } + ret =btrfs_get_root_id (subvol_fd, &r->cur_subvol->root_id); +if (ret <0 ) + goto out;subvol_uuid_search_add (&r->sus, r->cur_subvol); r->cur_subvol =NULL ; ret =0 ;
send-utils.c +20 -0
@@ -23,6 +23,26 @@#include "ioctl.h" #include "btrfs-list.h" +int btrfs_get_root_id (int fd,u64 *root_id) +{ +struct btrfs_ioctl_ino_lookup_args ino_args; +int ret; + +memset (&ino_args,0 ,sizeof (ino_args)); + ino_args.objectid =BTRFS_FIRST_FREE_OBJECTID ; + +/* this ioctl fills in ino_args->treeid */ + ret =ioctl (fd,BTRFS_IOC_INO_LOOKUP , &ino_args); +if (ret) { +fprintf (stderr,"ERROR: Failed to lookup root_id - %s\n" , +strerror (errno)); +return ret; + } + + *root_id = ino_args.treeid; +return 0 ; +} +static struct rb_node *tree_insert (struct rb_root *root,struct subvol_info *si,enum subvol_search_type type)
send-utils.h +1 -0
@@ -61,6 +61,7 @@ struct subvol_uuid_search {struct rb_root path_subvols; }; +int btrfs_get_root_id (int fd,u64 *root_id);int subvol_uuid_search_init (int mnt_fd,struct subvol_uuid_search *s);void subvol_uuid_search_finit (struct subvol_uuid_search *s); struct subvol_info *subvol_uuid_search(struct subvol_uuid_search *s,
Signature · 1 line
1.8.2.1
Re: [PATCH 08/17] Btrfs-progs: Set the root-id for received subvols in btrfs receive
Hello,
Quoted 18 lines
When an entry was added to the subvol search tree, the
root_idwas always 0 (not set at all) and therefore only the first one was added, all the others had been ignored. This commit adds a function to retrieve theroot_idfor a given fd, and this function is used to set theroot_idbefore the entry is added.Signed-off-by Stefan Behrens <[email protected]>--- cmds-receive.c | 3 +++ send-utils.c | 20 ++++++++++++++++++++ send-utils.h | 1 + 3 files changed, 24 insertions(+)cmds-receive.c +0
-0
cmds-receive.c static int finish_subvol(struct btrfs_receive *r) · line 128
Quoted 4 lines
cmds-receive.c +1
-0@@ -125,6 +125,9 @@ static int finish_subvol(struct btrfs_receive *r) goto out; } + ret =btrfs_get_root_id (subvol_fd, &r->cur_subvol->root_id);
There has been a function in btrfs-list.c named 'btrfs_list_get_path_rootid' which does the same things as 'btrfs_get_root_id'.I think you can reuse it.
Thanks,
Wang
Quoted 48 lines
cmds-receive.c +2
-0+if (ret <0 ) + goto out;subvol_uuid_search_add (&r->sus, r->cur_subvol); r->cur_subvol =NULL ; ret =0 ;send-utils.c +20
-0@@ -23,6 +23,26 @@#include "ioctl.h" #include "btrfs-list.h" +int btrfs_get_root_id (int fd,u64 *root_id) +{ +struct btrfs_ioctl_ino_lookup_args ino_args; +int ret; + +memset (&ino_args,0 ,sizeof (ino_args)); + ino_args.objectid =BTRFS_FIRST_FREE_OBJECTID ; + +/* this ioctl fills in ino_args->treeid */ + ret =ioctl (fd,BTRFS_IOC_INO_LOOKUP , &ino_args); +if (ret) { +fprintf (stderr,"ERROR: Failed to lookup root_id - %s\n" , +strerror (errno)); +return ret; + } + + *root_id = ino_args.treeid; +return 0 ; +} +static struct rb_node *tree_insert (struct rb_root *root,struct subvol_info *si,enum subvol_search_type type)send-utils.h +1
-0@@ -61,6 +61,7 @@ struct subvol_uuid_search {struct rb_root path_subvols; }; +int btrfs_get_root_id (int fd,u64 *root_id);int subvol_uuid_search_init (int mnt_fd,struct subvol_uuid_search *s);void subvol_uuid_search_finit (struct subvol_uuid_search *s); struct subvol_info *subvol_uuid_search(struct subvol_uuid_search *s,
PATCH 9/17 Btrfs-progs: btrfs-receive: different levels (amount) of debug output
In reply to Stefan BehrensThere used to be 2 levels of verbose output, now there are 3:
- None at all (no -v option given).
- Some information about received snapshots / subvolumes (-v option).
- Each received command is printed (-vv option).
--- cmds-receive.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-)
cmds-receive.c +16 -16
@@ -268,7 +268,7 @@ static int process_mkfile(const char *path, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"mkfile %s\n" , path); ret =creat (full_path,0600 );
@@ -292,7 +292,7 @@ static int process_mkdir(const char *path, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"mkdir %s\n" , path); ret =mkdir (full_path,0700 );
@@ -312,7 +312,7 @@ static int process_mknod(const char *path, u64 mode, u64 dev, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"mknod %s mode=%llu, dev=%llu\n" , path, mode, dev);
@@ -333,7 +333,7 @@ static int process_mkfifo(const char *path, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"mkfifo %s\n" , path); ret =mkfifo (full_path,0600 );
@@ -353,7 +353,7 @@ static int process_mksock(const char *path, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"mksock %s\n" , path); ret =mknod (full_path,0600 |S_IFSOCK ,0 );
@@ -373,7 +373,7 @@ static int process_symlink(const char *path, const char *lnk, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"symlink %s -> %s\n" , path, lnk); ret =symlink (lnk, full_path);
@@ -394,7 +394,7 @@ static int process_rename(const char *from, const char *to, void *user)char *full_from =path_cat (r->full_subvol_path, from);char *full_to =path_cat (r->full_subvol_path, to); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"rename %s -> %s\n" , from, to); ret =rename (full_from, full_to);
@@ -416,7 +416,7 @@ static int process_link(const char *path, const char *lnk, void *user)char *full_path =path_cat (r->full_subvol_path, path);char *full_link_path =path_cat (r->full_subvol_path, lnk); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"link %s -> %s\n" , path, lnk); ret =link (full_link_path, full_path);
@@ -438,7 +438,7 @@ static int process_unlink(const char *path, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"unlink %s\n" , path); ret =unlink (full_path);
@@ -458,7 +458,7 @@ static int process_rmdir(const char *path, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"rmdir %s\n" , path); ret =rmdir (full_path);
@@ -631,7 +631,7 @@ static int process_set_xattr(const char *path, const char *name,struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) { +if (g_verbose >=2 ) {fprintf (stderr,"set_xattr %s - name=%s data_len=%d " "data=%.*s\n" , path, name, len, len, (char *)data);
@@ -656,7 +656,7 @@ static int process_remove_xattr(const char *path, const char *name, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) { +if (g_verbose >=2 ) {fprintf (stderr,"remove_xattr %s - name=%s\n" , path, name); }
@@ -680,7 +680,7 @@ static int process_truncate(const char *path, u64 size, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"truncate %s size=%llu\n" , path, size); ret =truncate (full_path, size);
@@ -702,7 +702,7 @@ static int process_chmod(const char *path, u64 mode, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"chmod %s - mode=0%o\n" , path, (int )mode); ret =chmod (full_path, mode);
@@ -724,7 +724,7 @@ static int process_chown(const char *path, u64 uid, u64 gid, void *user)struct btrfs_receive *r = user;char *full_path =path_cat (r->full_subvol_path, path); -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"chown %s - uid=%llu, gid=%llu\n" , path, uid, gid);
@@ -750,7 +750,7 @@ static int process_utimes(const char *path, struct timespec *at,char *full_path =path_cat (r->full_subvol_path, path);struct timespec tv[2 ]; -if (g_verbose >=1 ) +if (g_verbose >=2 )fprintf (stderr,"utimes %s\n" , path); tv[0 ] = *at;
Signature · 1 line
1.8.2.1
PATCH 10/17 Btrfs-progs: small parent_subvol cleanup for cmds-receive.c
In reply to Stefan Behrensparent_subvol is local to process_snapshot() and not needed outside.
--- cmds-receive.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-)
cmds-receive.c +6 -8
@@ -63,7 +63,6 @@ struct btrfs_receivechar *full_subvol_path;struct subvol_info *cur_subvol; -struct subvol_info *parent_subvol;struct subvol_uuid_search sus; };
@@ -151,7 +150,6 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid, goto out; r->cur_subvol =calloc (1 ,sizeof (*r->cur_subvol)); - r->parent_subvol =NULL ; r->cur_subvol->path =strdup (path);free (r->full_subvol_path);
@@ -191,13 +189,13 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,struct btrfs_receive *r = user;char uuid_str[128 ];struct btrfs_ioctl_vol_args_v2 args_v2; +struct subvol_info *parent_subvol; ret =finish_subvol (r);if (ret <0 ) goto out; r->cur_subvol =calloc (1 ,sizeof (*r->cur_subvol)); - r->parent_subvol =NULL ; r->cur_subvol->path =strdup (path);free (r->full_subvol_path);
@@ -221,9 +219,9 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,memset (&args_v2,0 ,sizeof (args_v2));strncpy_null (args_v2.name, path); - r->parent_subvol =subvol_uuid_search (&r->sus,0 , parent_uuid, + parent_subvol =subvol_uuid_search (&r->sus,0 , parent_uuid, parent_ctransid,NULL , subvol_search_by_received_uuid); -if (!r->parent_subvol) { +if (!parent_subvol) { ret = -ENOENT ;fprintf (stderr,"ERROR: could not find parent subvolume\n" ); goto out;
@@ -239,12 +237,12 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,} }*/ - args_v2.fd =openat (r->mnt_fd, r->parent_subvol->path, + args_v2.fd =openat (r->mnt_fd, parent_subvol->path,O_RDONLY |O_NOATIME );if (args_v2.fd <0 ) { ret = -errno;fprintf (stderr,"ERROR: open %s failed. %s\n" , - r->parent_subvol->path,strerror (-ret)); + parent_subvol->path,strerror (-ret)); goto out; }
@@ -253,7 +251,7 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,if (ret <0 ) { ret = -errno;fprintf (stderr,"ERROR: creating snapshot %s -> %s " -"failed. %s\n" , r->parent_subvol->path, +"failed. %s\n" , parent_subvol->path, path,strerror (-ret)); goto out; }
Signature · 1 line
1.8.2.1
PATCH 11/17 Btrfs-progs: fix bug in find_root_gen
In reply to Stefan BehrensA copy & paste error.
--- btrfs-list.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
btrfs-list.c +1 -1
@@ -775,7 +775,7 @@ static u64 find_root_gen(int fd)if (sk->min_type !=BTRFS_ROOT_ITEM_KEY )break ; -if (sk->min_objectid !=BTRFS_ROOT_ITEM_KEY ) +if (sk->min_objectid != ino_args.treeid)break ; }return max_found;
Signature · 1 line
1.8.2.1
PATCH 12/17 Btrfs-progs: btrfs-receive optionally honors the end-cmd
In reply to Stefan BehrensA new option is added to btrfs-receive to change the behavior when an <end cmd> is received in the Btrfs send stream.
The traditional behavior (which still is the default) is to continue to read the stream until an EOF condition is encountered. If an <end cmd> is received, afterwards either an EOF or a new <stream header> is expected.
The new behavior (if the -e option is set on the command line) is to terminate after an <end cmd> is read without the need for an EOF. This allows the stream (e.g. a single TCP stream) to carry additional data or even multiple Btrfs send streams.
Old btrfs-send tools used to encode multiple snapshots like this (with 2 snapshots in this example):
<stream header> + <sequence of commands> + <end cmd> + <stream header> + <sequence of commands> + <end cmd> + EOF
If the new -e option is set, the expected format is like this:
<stream header> + <sequence of commands> +
<sequence of commands> + <end cmd>
The btrfs-send tool is changed in a seperate commit to always use the new format, i.e. to send an <end cmd> only at the end.
Note that the currently existing receivers treat <end cmd> only as an indication that a new <stream header> is following. This means, you can just skip the sequence <end cmd> <stream header> without loosing compatibility. As long as an EOF is following, the currently existing receivers handle the new format (if the two new flags are used) exactly as the old one.
The goal of changing the semantic of <end cmd> is to be able to use a single stream (one TCP connection) to multiplex a request/response handshake plus Btrfs send streams, all in the same stream. In this case you cannot evaluate an EOF condition as an end of the Btrfs send stream. You need something else, and the <end cmd> is just perfect for this purpose.
--- cmds-receive.c | 16 +++++++++++++--- send-stream.c | 6 ++++-- send-stream.h | 3 ++- 3 files changed, 19 insertions(+), 6 deletions(-)
cmds-receive.c +13 -3
@@ -65,6 +65,8 @@ struct btrfs_receivestruct subvol_info *cur_subvol;struct subvol_uuid_search sus; + +int honor_end_cmd; };static int finish_subvol (struct btrfs_receive *r)
@@ -809,7 +811,8 @@ int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd) goto out;while (!end) { - ret =btrfs_read_and_process_send_stream (r_fd, &send_ops, r); + ret =btrfs_read_and_process_send_stream (r_fd, &send_ops, r, + r->honor_end_cmd);if (ret <0 ) goto out;if (ret)
@@ -862,7 +865,7 @@ static int do_cmd_receive(int argc, char **argv) r.mnt_fd =-1 ; r.write_fd =-1 ; -while ((c =getopt (argc, argv,"vf:" )) !=-1 ) { +while ((c =getopt (argc, argv,"evf:" )) !=-1 ) {switch (c) {case 'v' : g_verbose++;
@@ -870,6 +873,9 @@ static int do_cmd_receive(int argc, char **argv)case 'f' : fromfile = optarg;break ; +case 'e' : + r.honor_end_cmd =1 ; +break ;case '?' :default :fprintf (stderr,"ERROR: receive args invalid.\n" );
@@ -903,7 +909,7 @@ static const char * const receive_cmd_group_usage[] = { };const char *const cmd_receive_usage[] = { -"btrfs receive [-v] [-f <infile>] <mount>" , +"btrfs receive [-ve] [-f <infile>] <mount>" ,"Receive subvolumes from stdin." ,"Receives one or more subvolumes that were previously " ,"sent with btrfs send. The received subvolumes are stored" ,
@@ -919,6 +925,10 @@ const char * const cmd_receive_usage[] = {"-f <infile> By default, btrfs receive uses stdin" ," to receive the subvolumes. Use this" ," option to specify a file to use instead." , +"-e Terminate after receiving an <end cmd>" , +" in the data stream. Without this option," , +" the receiver terminates only if an error" , +" is recognized or on EOF." , NULL };
send-stream.c +4 -2
@@ -439,7 +439,8 @@ out: }int btrfs_read_and_process_send_stream (int fd, -struct btrfs_send_ops *ops,void *user) +struct btrfs_send_ops *ops,void *user, +int honor_end_cmd) {int ret;struct btrfs_send_stream s;
@@ -476,7 +477,8 @@ int btrfs_read_and_process_send_stream(int fd,if (ret <0 ) goto out;if (ret) { - ret =0 ; +if (!honor_end_cmd) + ret =0 ; goto out; } }
send-stream.h +2 -1
@@ -57,7 +57,8 @@ struct btrfs_send_ops { };int btrfs_read_and_process_send_stream (int fd, -struct btrfs_send_ops *ops,void *user); +struct btrfs_send_ops *ops,void *user, +int honor_end_cmd); #ifdef __cplusplus }
Signature · 1 line
1.8.2.1
PATCH 13/17 Btrfs-progs: don't allocate one byte too much each time
In reply to Stefan Behrensstr1 + '/' + str2 + '\0' requires a buffer with the size strlen(str1) + strlen(str2) + 2 bytes.
str1 + '/' + str2 + '/' + str3 + '\0' requires a buffer with the size strlen(str1) + strlen(str2) + strlen(str3) + 3 bytes.
--- send-utils.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
send-utils.c +2 -2
@@ -352,7 +352,7 @@ char *path_cat(const char *p1, const char *p2) {int p1_len =strlen (p1);int p2_len =strlen (p2); -char *new =malloc (p1_len + p2_len +3 ); +char *new =malloc (p1_len + p2_len +2 );if (p1_len && p1[p1_len -1 ] =='/' ) p1_len--;
@@ -368,7 +368,7 @@ char *path_cat3(const char *p1, const char *p2, const char *p3)int p1_len =strlen (p1);int p2_len =strlen (p2);int p3_len =strlen (p3); -char *new =malloc (p1_len + p2_len + p3_len +4 ); +char *new =malloc (p1_len + p2_len + p3_len +3 );if (p1_len && p1[p1_len -1 ] =='/' ) p1_len--;
Signature · 1 line
1.8.2.1
PATCH 14/17 Btrfs-progs: Fix that BTRFS_FSID_SIZE is used instead of BTRFS_UUID_SIZE
In reply to Stefan BehrensBoth are 16 but it's wrong anyway to use FSID_SIZE for UUIDs.
--- cmds-receive.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
cmds-receive.c +1 -1
@@ -565,7 +565,7 @@ static int process_clone(const char *path, u64 offset, u64 len, subvol_search_by_received_uuid);if (!si) {if (memcmp (clone_uuid, r->cur_subvol->received_uuid, -BTRFS_FSID_SIZE ) ==0 ) { +BTRFS_UUID_SIZE ) ==0 ) {/* TODO check generation of extent */ subvol_path =strdup (r->cur_subvol->path); }else {
Signature · 1 line
1.8.2.1
PATCH 15/17 Btrfs-progs: remove some unused code
In reply to Stefan Behrens--- send-stream.c | 3 --- 1 file changed, 3 deletions(-)
send-stream.c +0 -3
@@ -410,9 +410,6 @@ static int read_and_process_cmd(struct btrfs_send_stream *s)break ;case BTRFS_SEND_C_UTIMES :TLV_GET_STRING (s,BTRFS_SEND_A_PATH , &path); -if (strstr (path,".bak_1.log" )) { - ret =0 ; - }TLV_GET_TIMESPEC (s,BTRFS_SEND_A_ATIME , &at);TLV_GET_TIMESPEC (s,BTRFS_SEND_A_MTIME , &mt);TLV_GET_TIMESPEC (s,BTRFS_SEND_A_CTIME , &ct);
Signature · 1 line
1.8.2.1
PATCH 16/17 Btrfs-progs: allow to receive to relative directories
In reply to Stefan Behrens--- cmds-receive.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
cmds-receive.c +1 -1
@@ -755,7 +755,7 @@ static int process_utimes(const char *path, struct timespec *at, tv[0 ] = *at; tv[1 ] = *mt; - ret =utimensat (-1 , full_path, tv,AT_SYMLINK_NOFOLLOW ); + ret =utimensat (AT_FDCWD , full_path, tv,AT_SYMLINK_NOFOLLOW );if (ret <0 ) { ret = -errno; fprintf(stderr, "ERROR: utimes %s failed. %s\n",
Signature · 1 line
1.8.2.1
PATCH 17/17 btrfs-progs: Fix the receive code pathing
In reply to Stefan BehrensFrom: Alex Lyakas <[email protected]>
The receive code was not distinguishing properly between the mount root and the directory to create the received subvolume in.
Also make sure the find_mount_root reports an error if it cannot find a match at all.
--- cmds-receive.c | 67 +++++++++++++++++++++++++++++++++++++++++++++++++--------- cmds-send.c | 7 ++++++ 2 files changed, 64 insertions(+), 10 deletions(-)
cmds-receive.c +57 -10
@@ -55,11 +55,13 @@ static int g_verbose = 0;struct btrfs_receive {int mnt_fd; +int dest_dir_fd;int write_fd;char *write_path;char *root_path; +char *dest_dir_path;/* relative to root_path */ char *full_subvol_path;struct subvol_info *cur_subvol;
@@ -153,9 +155,12 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid, r->cur_subvol =calloc (1 ,sizeof (*r->cur_subvol)); - r->cur_subvol->path =strdup (path); +if (strlen (r->dest_dir_path) ==0 ) + r->cur_subvol->path =strdup (path); +else + r->cur_subvol->path =path_cat (r->dest_dir_path, path);free (r->full_subvol_path); - r->full_subvol_path =path_cat (r->root_path, path); + r->full_subvol_path =path_cat3 (r->root_path, r->dest_dir_path, path);fprintf (stderr,"At subvol %s\n" , path);
@@ -171,7 +176,7 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid,memset (&args_v1,0 ,sizeof (args_v1));strncpy_null (args_v1.name, path); - ret =ioctl (r->mnt_fd,BTRFS_IOC_SUBVOL_CREATE , &args_v1); + ret =ioctl (r->dest_dir_fd,BTRFS_IOC_SUBVOL_CREATE , &args_v1);if (ret <0 ) { ret = -errno; fprintf(stderr, "ERROR: creating subvolume %s failed. "
@@ -199,9 +204,12 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, r->cur_subvol =calloc (1 ,sizeof (*r->cur_subvol)); - r->cur_subvol->path =strdup (path); +if (strlen (r->dest_dir_path) ==0 ) + r->cur_subvol->path =strdup (path); +else + r->cur_subvol->path =path_cat (r->dest_dir_path, path);free (r->full_subvol_path); - r->full_subvol_path =path_cat (r->root_path, path); + r->full_subvol_path =path_cat3 (r->root_path, r->dest_dir_path, path);fprintf (stderr,"At snapshot %s\n" , path);
@@ -248,7 +256,7 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid, goto out; } - ret =ioctl (r->mnt_fd,BTRFS_IOC_SNAP_CREATE_V2 , &args_v2); + ret =ioctl (r->dest_dir_fd,BTRFS_IOC_SNAP_CREATE_V2 , &args_v2);close (args_v2.fd);if (ret <0 ) { ret = -errno;
@@ -795,17 +803,49 @@ struct btrfs_send_ops send_ops = {int do_receive (struct btrfs_receive *r,const char *tomnt,int r_fd) {int ret; +char *dest_dir_full_path;int end =0 ; - r->root_path =strdup (tomnt); - r->mnt_fd =open (tomnt,O_RDONLY |O_NOATIME ); + dest_dir_full_path =realpath (tomnt,NULL ); +if (!dest_dir_full_path) { + ret = -errno; +fprintf (stderr,"ERROR: realpath(%s) failed. %s\n" , tomnt, +strerror (-ret)); + goto out; + } + r->dest_dir_fd =open (dest_dir_full_path,O_RDONLY |O_NOATIME ); +if (r->dest_dir_fd <0 ) { + ret = -errno; +fprintf (stderr, +"ERROR: failed to open destination directory %s. %s\n" , + dest_dir_full_path,strerror (-ret)); + goto out; + } + + ret =find_mount_root (dest_dir_full_path, &r->root_path); +if (ret <0 ) { + ret = -EINVAL ; +fprintf (stderr,"ERROR: failed to determine mount point " +"for %s\n" , dest_dir_full_path); + goto out; + } + r->mnt_fd =open (r->root_path,O_RDONLY |O_NOATIME );if (r->mnt_fd <0 ) { ret = -errno; -fprintf (stderr,"ERROR: failed to open %s. %s\n" , tomnt, -strerror (-ret)); +fprintf (stderr,"ERROR: failed to open %s. %s\n" , r->root_path, +strerror (-ret)); goto out; } +/* +* find_mount_root returns a root_path that is a subpath of +* dest_dir_full_path. Now get the other part of root_path, +* which is the destination dir relative to root_path. +*/ + r->dest_dir_path = dest_dir_full_path +strlen (r->root_path); +while (r->dest_dir_path[0 ] =='/' ) + r->dest_dir_path++; + ret =subvol_uuid_search_init (r->mnt_fd, &r->sus);if (ret <0 ) goto out;
@@ -838,6 +878,8 @@ out: r->write_path =NULL ;free (r->full_subvol_path); r->full_subvol_path =NULL ; + r->dest_dir_path =NULL ; +free (dest_dir_full_path);if (r->cur_subvol) {free (r->cur_subvol->path);free (r->cur_subvol);
@@ -848,6 +890,10 @@ out:close (r->mnt_fd); r->mnt_fd =-1 ; } +if (r->dest_dir_fd !=-1 ) { +close (r->dest_dir_fd); + r->dest_dir_fd =-1 ; + }return ret; }
@@ -864,6 +910,7 @@ static int do_cmd_receive(int argc, char **argv)memset (&r,0 ,sizeof (r)); r.mnt_fd =-1 ; r.write_fd =-1 ; + r.dest_dir_fd =-1 ;while ((c =getopt (argc, argv,"evf:" )) !=-1 ) {switch (c) {
cmds-send.c +7 -0
@@ -84,6 +84,13 @@ int find_mount_root(const char *path, char **mount_root) }fclose (mnttab); +if (!longest_match) { +fprintf (stderr, +"ERROR: Failed to find mount root for path %s.\n" , + path); +return -ENOENT ; + } + *mount_root =realpath (longest_match,NULL );free (longest_match);
Signature · 1 line
1.8.2.1
PATCH v2 8/17 Btrfs-progs: Set the root-id for received subvols in btrfs receive
In reply to Stefan BehrensWhen an entry was added to the subvol search tree, the root_id was always 0 (not set at all) and therefore only the first one was added, all the others had been ignored. This commit sets the root_id before the entry is added.
---
v1 -> v2:
- Wang Shilong noticed that a similar subfunction to retrieve the root-id for a subvolume was already there. This subfunction is now used instead of introducing a new one.
cmds-receive.c | 4 ++++ 1 file changed, 4 insertions(+)
cmds-receive.c +4 -0
@@ -45,6 +45,7 @@#include "commands.h" #include "utils.h" #include "list.h" +#include "btrfs-list.h" #include "send.h" #include "send-stream.h"
@@ -125,6 +126,9 @@ static int finish_subvol(struct btrfs_receive *r) goto out; } + ret =btrfs_list_get_path_rootid (subvol_fd, &r->cur_subvol->root_id); +if (ret <0 ) + goto out;subvol_uuid_search_add (&r->sus, r->cur_subvol); r->cur_subvol =NULL ; ret =0 ;
Signature · 1 line
1.8.2.1
Hi,
Quoted 13 lines · Stefan Behrens
Alex Lyakas (1):
btrfs-progs: Fix the receive code pathingStefan Behrens (16):
Btrfs-progs: Use/proc/mountsinstead of/etc/mtab
Btrfs-progs: ignore subvols aboveBTRFS_LAST_FREE_OBJECTID
Btrfs-progs: close file descriptor incmds-send.c
Btrfs-progs: fix a small memory leak inbtrfs-list.c
Btrfs-progs: add a function to freesubvol_uuid_searchmemory
Btrfs-progs: cleanupsubvol_uuid_searchmemory in btrfs send/receive
Btrfs-progs: free memory and close file descriptor in btrfs receive
Btrfs-progs: Set the root-id for received subvols in btrfs receive
Btrfs-progs: btrfs-receive: different levels (amount) of debug output
Quoted 8 lines
Btrfs-progs: small
parent_subvolcleanup forcmds-receive.c
Btrfs-progs: fix bug infind_root_gen
Btrfs-progs: btrfs-receive optionally honors the end-cmd
Btrfs-progs: don't allocate one byte too much each time
Btrfs-progs: Fix thatBTRFS_FSID_SIZEis used instead of
BTRFS_UUID_SIZE
Btrfs-progs: remove some unused code
Btrfs-progs: allow to receive to relative directories
All merged, thanks.
david