Utility btrfs-diff-go confused by btrfs-send output - a file isn't changed but is associated with an UPDATE_EXTENT command
Hi all,
First, thank you for the btrfs FS and all your work.
This is my first interaction with this community and I'll try to communicate to my best.
I am developing 'btrfs-diff-go' (https://github.com/mbideau/btrfs-diff-go), an utility that analyse the output stream of a 'btrfs-send' command, and produce an output like the 'diff' utility to visualise which files/directories have changed (and how) between two btrfs subvolumes/snapshots.
While I was testing it I encountered an issue: some files were detected as changed, when they where actually not (no difference with 'diff' nor 'sha256sum').
With my program's debug log I found out that it was related with the UPDATE_EXTENT command, always interpreted as a file change.
Digging into it I ended up in the 'btrfs-send' source code and found out that this command was used when data actually has changed I guess (in function 'send_extent_data'
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git/tree/fs/btrfs/send.c#n5682
), but also for "holes" (in function 'send_hole'
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git/tree/fs/btrfs/send.c#n5456
).
I couldn't debug further nor check myself (according to my current free time and knowledge), but as I read the code I understand that the function 'send_update_extent' always send a path, and maybe that path is wrong or should be empty when sending a "hole" ?
Could you please help me:
- to reproduce a simple test case that send the UPDATE_EXTENT command for a real file change and for a hole ? This way I could acknowledge properly the issue in my program 'btrfs-diff-go', and later on the fix.
- to understand what's going on ... Is a "hole" an actual file change even no data has changed in that file, and if so, how I am supposed to filter them out to only report real data changes to match 'diff' output ?
- to report this as a bug if you think it is one ?
See also:
- btrfs-progs sending UPDATE_EXTENT command:
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/tree/libbtrfs/send-stream.c#n455
- btrsf-progs receiving UPDATE_EXTENT command:
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/tree/cmds/receive.c#n986
Thanks in advance for your time and answers.
Best regards,
Michael Bideau [France]
Quoted 29 lines · Michael Bideau
Hi all,
First, thank you for the btrfs FS and all your work.
This is my first interaction with this community and I'll try to communicate to my best.I am developing 'btrfs-diff-go' (https://github.com/mbideau/btrfs-diff-go), an utility that analyse the output stream of a 'btrfs-send' command, and produce an output like the 'diff' utility to visualise which files/directories have changed (and how) between two btrfs subvolumes/snapshots.
While I was testing it I encountered an issue: some files were detected as changed, when they where actually not (no difference with 'diff' nor 'sha256sum').
With my program's debug log I found out that it was related with theUPDATE_EXTENTcommand, always interpreted as a file change.
Digging into it I ended up in the 'btrfs-send' source code and found out that this command was used when data actually has changed I guess (in function 'send_extent_data'
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git/tree/fs/btrfs/send.c#n5682
), but also for "holes" (in function 'send_hole'
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git/tree/fs/btrfs/send.c#n5456
).
I couldn't debug further nor check myself (according to my current free time and knowledge), but as I read the code I understand that the function 'send_update_extent' always send a path, and maybe that path is wrong or should be empty when sending a "hole" ?
Quoted 8 lines
Could you please help me:
- to reproduce a simple test case that send theUPDATE_EXTENTcommand for a real file change and for a hole ? This way I could acknowledge properly the issue in my program 'btrfs-diff-go', and later on the fix.
- to understand what's going on ... Is a "hole" an actual file change even no data has changed in that file, and if so, how I am supposed to filter them out to only report real data changes to match 'diff' output ?
- to report this as a bug if you think it is one ?
There at least a couple reasons why an UPDATE_EXTENT command may be sent yet the data in the file did not change:
1) The file is fully or partially rewritten with the same data - for e.g. running defrag - this only changes the extent layout - but we have no way to quickly and efficiently check it's the same data, so we send the UPDATE_EXTENT command - it may also be interesting for some use cases to know the extent layout changed - so it's not necessarily a bad thing;
2) A hole is punched into a file range with all data bytes having a value of 0. Or an application writes zeros to a range with a hole, or calls fallocate against a range that has a hole.
If you find a case where the extent layout is exactly the same and no data changed, then it's a bug (although a minor one as it didn't incorrectly change the file).
Quoted 12 lines
See also:
- btrfs-progs sendingUPDATE_EXTENTcommand:
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/tree/libbtrfs/send-stream.c#n455
- btrsf-progs receivingUPDATE_EXTENTcommand:
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/tree/cmds/receive.c#n986Thanks in advance for your time and answers.
Best regards,
Michael Bideau [France]
Hi all,
Thanks Filipe for your quick and precise answer.
My follow-up questions below...
Le vendredi 10 mars 2023 à 13:41 +0000, Filipe Manana a écrit :
Quoted 55 lines · Michael Bideau
Quoted 38 lines
Hi all,
First, thank you for the btrfs FS and all your work.
This is my first interaction with this community and I'll try to communicate to
my best.I am developing 'btrfs-diff-go' (https://github.com/mbideau/btrfs-diff-go),
an
utility that analyse the output stream of a 'btrfs-send' command, and produce an
output like the 'diff' utility to visualise which files/directories have changed
(and how) between two btrfs subvolumes/snapshots.While I was testing it I encountered an issue: some files were detected as changed, when they where actually not (no difference with 'diff' nor 'sha256sum').
With my program's debug log I found out that it was related with theUPDATE_EXTENTcommand, always interpreted as a file change. Digging into it I ended up in the 'btrfs-send' source code and found out that
this command was used when data actually has changed I guess (in function 'send_extent_data'
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git/tree/fs/btrf
s/send.c#n5682
), but also for "holes" (in function 'send_hole'
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git/tree/fs/btrf
s/send.c#n5456
).
I couldn't debug further nor check myself (according to my current free time and
knowledge), but as I read the code I understand that the function 'send_update_extent' always send a path, and maybe that path is wrong or should
be empty when sending a "hole" ?Quoted 12 lines
Could you please help me:
- to reproduce a simple test case that send theUPDATE_EXTENTcommand for a real
file change and for a hole ? This way I could acknowledge properly the issue in
my program 'btrfs-diff-go', and later on the fix.
- to understand what's going on ... Is a "hole" an actual file change even no
data has changed in that file, and if so, how I am supposed to filter them out
to only report real data changes to match 'diff' output ?
- to report this as a bug if you think it is one ?There at least a couple reasons why an
UPDATE_EXTENTcommand may be sent yet the data in the file did not change:
Quoted 6 lines
1) The file is fully or partially rewritten with the same data - for e.g. running defrag - this only changes the extent layout - but we have no way to quickly and efficiently check it's the same data, so we send the
UPDATE_EXTENTcommand - it may also be interesting for some use cases to know the extent layout changed - so it's not necessarily a bad thing;
Agreed that is it not a bad thing, but I would have liked to have the information if only the layout changed and not the data (from a user perspective).
Do you think it is possible to code an evolution, either of the btrfs kernel or of the btrfs-progs, to get that information (data change/layout change), may be with an option flag ? I am sure we could find a solution that would cost less than a full check-summing or a diff of the content and still be usable ... May be there is different strategies to get that feature ...
I am not able to design and code it myself (lack of time and knowledge) but I could contribute to issue description, use case, and may be testing... with some help/guidance.
Quoted 8 lines
2) A hole is punched into a file range with all data bytes having a value of 0. Or an application writes zeros to a range with a hole, or calls fallocate against a range that has a hole.
If you find a case where the extent layout is exactly the same and no data changed, then it's a bug (although a minor one as it didn't incorrectly change the file).
I struggled a lot to check that and failed. Could you provide me guidance on how to check that please ?
The following were my different attempts to find a way to compare the extents layout :
``` $ sudo sha256sum /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state \ /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state 03dbfae7fc7cc0e59ef20df6e3fde60caa086da0af0c6ef6db2c543e49870a58 /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state 03dbfae7fc7cc0e59ef20df6e3fde60caa086da0af0c6ef6db2c543e49870a58 /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state $ sudo stat /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state Fichier : /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state Taille : 154 Blocs : 8 Blocs d'E/S : 4096 fichier Périphérique : 0/49 Inœud : 292461 Liens : 1 Accès : (0644/-rw-r--r--) UID : ( 0/ root) GID : ( 0/ root) Accès : 2023-03-12 12:12:11.601909283 +0100 Modif. : 2023-02-22 19:56:50.915720066 +0100 Changt : 2023-02-22 19:56:50.915720066 +0100 Créé : 2022-11-12 22:21:37.822349260 +0100 $ sudo stat /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state Fichier : /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state Taille : 154 Blocs : 8 Blocs d'E/S : 4096 fichier Périphérique : 0/51 Inœud : 292461 Liens : 1 Accès : (0644/-rw-r--r--) UID : ( 0/ root) GID : ( 0/ root) Accès : 2023-03-07 16:58:03.027010803 +0100 Modif. : 2023-03-02 13:23:43.909033727 +0100 Changt : 2023-03-02 13:23:43.909033727 +0100 Créé : 2022-11-12 22:21:37.822349260 +0100 $ sudo filefrag -e /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state Filesystem type is: 9123683e File size of /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state is 154 (1 block of 4096 bytes) ext: logical_offset: physical_offset: length: expected: flags: 0: 0.. 4095: 0.. 4095: 4096: last,not_aligned,inline,eof /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state: 1 extent found $ sudo filefrag -e /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state Filesystem type is: 9123683e File size of /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state is 154 (1 block of 4096 bytes) ext: logical_offset: physical_offset: length: expected: flags: 0: 0.. 4095: 0.. 4095: 4096: last,not_aligned,inline,eof /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state: 1 extent found $ sudo python3 btrfs-debugfs -f /mnt/toplevel/timeshift-btrfs/snapshots/2023-02- 22_23-00-06/@/var/lib/blueman/network.state (292461 0): ram 154 disk 0 disk_size 154 -- inline file: /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state extents 1 disk size 154 logical size 154 ratio 1.00 $ sudo python3 btrfs-debugfs -f /mnt/toplevel/timeshift-btrfs/snapshots/2023-03- 02_13-31-55/@/var/lib/blueman/network.state (292461 0): ram 154 disk 0 disk_size 154 -- inline file: /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state extents 1 disk size 154 logical size 154 ratio 1.00 $ sudo btrfs fi du /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state \ /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state Total Exclusive Set shared Filename 0.00B 0.00B 0.00B /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@/var/lib/blueman/network.state 0.00B 0.00B 0.00B /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@/var/lib/blueman/network.state $ sudo btrfs sub show /mnt/toplevel/timeshift-btrfs/snapshots/2023-02-22_23-00- 06/@ timeshift-btrfs/snapshots/2023-02-22_23-00-06/@ Name: @ UUID: 2209acc6-1f29-1042-95ac-5d9a3b59b9e0 Parent UUID: 24d1cc44-64a9-7547-a325-b99f871c83c0 Received UUID: - Creation time: 2023-02-22 23:00:06 +0100 Subvolume ID: 468 Generation: 79512 Gen at creation: 73193 Parent ID: 5 Top level ID: 5 Flags: - Send transid: 0 Send time: 2023-02-22 23:00:06 +0100 Receive transid: 0 Receive time: - Snapshot(s): .2023-02-22_23-00-06.ro/@ $ sudo btrfs sub show /mnt/toplevel/timeshift-btrfs/snapshots/2023-03-02_13-31- 55/@ timeshift-btrfs/snapshots/2023-03-02_13-31-55/@ Name: @ UUID: cd6b2945-cce2-e843-8b0a-ce48711303e2 Parent UUID: 24d1cc44-64a9-7547-a325-b99f871c83c0 Received UUID: - Creation time: 2023-03-02 13:31:55 +0100 Subvolume ID: 482 Generation: 79512 Gen at creation: 74292 Parent ID: 5 Top level ID: 5 Flags: - Send transid: 0 Send time: 2023-03-02 13:31:55 +0100 Receive transid: 0 Receive time: - Snapshot(s): .2023-03-02_13-31-55.ro/@ $ sudo btrfs inspect-internal dump-tree -e /dev/mapper/luks-d62a42e4-b94e-4c99- a869-23c6cdc3f55f > /tmp/dump_tree_extents.txt # not found any inode matching the one of the files ```
Thanks again (to all).
Quoted 17 lines
Quoted 17 lines
See also:
- btrfs-progs sendingUPDATE_EXTENTcommand:
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-
progs.git/tree/libbtrfs/send-stream.c#n455
- btrsf-progs receivingUPDATE_EXTENTcommand:
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-
progs.git/tree/cmds/receive.c#n986Thanks in advance for your time and answers.
Best regards,
Michael Bideau [France]