Bug in 'btrfs filesystem du' ?
Hello,
after a balance ‘btrfs filesystem du’ probably shows false data about shared data.
To reproduce, create a (smal) btrfs-filesystem, copy some data in a directory, then ‘cp -a --reflink’ the data. Now all data is shared and ‘btrfs fi du’ shows it correct. In my case:
Total Exclusive Set shared Filename
59.38MiB 29.69MiB 29.69MiB .
after a balance ‘btrfs fi du’ shows no shared data any more, but all data as exclusive. In my case:
Total Exclusive Set shared Filename
59.38MiB 59.38MiB 0.00B .
As ‘btrfs fi df’ still shows used=29.69MiB, the problem probabaly is in btrfs-tools.
Test-session log:
# dd if=/dev/urandom of=dev-btrfs bs=4K count=100000 100000+0 Datensätze ein 100000+0 Datensätze aus 409600000 bytes (410 MB, 391 MiB) copied, 24.7574 s, 16.5 MB/s
# mkfs.btrfs dev-btrfs btrfs-progs v4.5.2 See http://btrfs.wiki.kernel.org for more information.
Label: (null)
UUID: 698a2755-8ecb-468d-9577-9a48947361ea
Node size: 16384
Sector size: 4096
Filesystem size: 390.62MiB
Block group profiles:
Data: single 8.00MiB
Metadata: DUP 40.00MiB
System: DUP 12.00MiB
SSD detected: no
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 390.62MiB dev-btrfs# mount /tmp/dev-btrfs /mnt/
# cd /mnt/
# btrfs fi du -s .
Total Exclusive Set shared Filename
0.00B 0.00B 0.00B .# cp -a /scratch/kernel/linux-4.6/drivers/usb .
# btrfs fi du -s .
Total Exclusive Set shared Filename
28.96MiB 28.96MiB 0.00B .# btrfs fi df . Data, single: total=56.00MiB, used=3.61MiB System, DUP: total=8.00MiB, used=16.00KiB Metadata, DUP: total=32.00MiB, used=192.00KiB GlobalReserve, single: total=16.00MiB, used=0.00B
# btrfs fi usage .
Overall:
Device size: 390.62MiB
Device allocated: 136.00MiB
Device unallocated: 254.62MiB
Device missing: 0.00B
Used: 32.06MiB
Free (estimated): 280.94MiB (min: 153.62MiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 16.00MiB (used: 0.00B)Data,single: Size:56.00MiB, Used:29.69MiB
/dev/loop0 56.00MiB
Metadata,DUP: Size:32.00MiB, Used:1.17MiB
/dev/loop0 64.00MiB
System,DUP: Size:8.00MiB, Used:16.00KiB
/dev/loop0 16.00MiB
Unallocated:
/dev/loop0 254.62MiB
# cp -a --reflink usb usb2
# btrfs fi du -s .
Total Exclusive Set shared Filename
59.38MiB 29.69MiB 29.69MiB .# btrfs fi df . Data, single: total=56.00MiB, used=29.69MiB System, DUP: total=8.00MiB, used=16.00KiB Metadata, DUP: total=32.00MiB, used=1.17MiB GlobalReserve, single: total=16.00MiB, used=0.00B
# btrfs fi usage .
Overall:
Device size: 390.62MiB
Device allocated: 136.00MiB
Device unallocated: 254.62MiB
Device missing: 0.00B
Used: 32.06MiB
Free (estimated): 280.94MiB (min: 153.62MiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 16.00MiB (used: 0.00B)Data,single: Size:56.00MiB, Used:29.69MiB
/dev/loop0 56.00MiB
Metadata,DUP: Size:32.00MiB, Used:1.17MiB
/dev/loop0 64.00MiB
System,DUP: Size:8.00MiB, Used:16.00KiB
/dev/loop0 16.00MiB
Unallocated:
/dev/loop0 254.62MiB
# btrfs balance start . WARNING:
Full balance without filters requested. This operation is very intense and takes potentially very long. It is recommended to use the balance filters to narrow down the balanced data. Use 'btrfs balance start --full-balance' option to skip this warning. The operation will start in 10 seconds. Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting balance without any filters.
Done, had to relocate 4 out of 4 chunks
# btrfs fi du -s .
Total Exclusive Set shared Filename
59.38MiB 59.38MiB 0.00B .# btrfs fi df . Data, single: total=48.00MiB, used=29.69MiB System, DUP: total=24.00MiB, used=16.00KiB Metadata, DUP: total=24.00MiB, used=2.08MiB GlobalReserve, single: total=16.00MiB, used=0.00B
# btrfs fi usage .
Overall:
Device size: 390.62MiB
Device allocated: 144.00MiB
Device unallocated: 246.62MiB
Device missing: 0.00B
Used: 33.88MiB
Free (estimated): 264.94MiB (min: 141.62MiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 16.00MiB (used: 0.00B)Data,single: Size:48.00MiB, Used:29.69MiB
/dev/loop0 48.00MiB
Metadata,DUP: Size:24.00MiB, Used:2.08MiB
/dev/loop0 48.00MiB
System,DUP: Size:24.00MiB, Used:16.00KiB
/dev/loop0 48.00MiB
Unallocated:
/dev/loop0 246.62MiB
MfG
bmg
Signature · 3 lines
„Des is völlig wurscht, was heut beschlos- | M G Berberich sen wird: I bin sowieso dagegn!“ | [email protected] (SPD-Stadtrat Kurt Schindler; Regensburg) |
Am Montag, den 27. Juni schrieb M G Berberich:
Quoted 2 lines
after a balance ‘btrfs filesystem du’ probably shows false data about shared data.
Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.
MfG
bmg
Signature · 3 lines
„Des is völlig wurscht, was heut beschlos- | M G Berberich sen wird: I bin sowieso dagegn!“ | [email protected] (SPD-Stadtrat Kurt Schindler; Regensburg) |
Quoted 5 lines · M G Berberich
Am Montag, den 27. Juni schrieb M G Berberich:
Quoted 2 lines
after a balance ‘btrfs filesystem du’ probably shows false data about shared data.
Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.
With btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct about shared data.
Hello,
Am Montag, den 27. Juni schrieb Henk Slager:
Quoted 9 lines · M G Berberich
Quoted 5 lines
Am Montag, den 27. Juni schrieb M G Berberich:
Quoted 2 lines
after a balance ‘btrfs filesystem du’ probably shows false data about shared data.
Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.
With btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct about shared data.
I tested with kernels 4.6.3 and 4.7-rc5 and with btrfs-progs 4.5.2 and 4.61. The later kernel with two patches to make the kernel work:
https://lkml.org/lkml/2016/6/1/310 https://lkml.org/lkml/2016/6/1/311 .
You can see the script¹ I used (do-btrfs-du-test) and the logs at
http://m-berberich.de/btrfs/
In all four cases, ‘btrfs fi du -s .’ reports
Total Exclusive Set shared Filename
59.38MiB 0.00B 29.69MiB .
befor balance and
Total Exclusive Set shared Filename
59.38MiB 59.38MiB 0.00B .
after balance.
Disclaimer: The script works for me, no guaranty at all.
MfG
bmg
__________
¹ Disclaimer: The script works for me, no guaranty at all.
Signature · 3 lines
„Des is völlig wurscht, was heut beschlos- | M G Berberich sen wird: I bin sowieso dagegn!“ | [email protected] (SPD-Stadtrat Kurt Schindler; Regensburg) |
Quoted 8 lines · M G Berberich
Hello,
Am Montag, den 27. Juni schrieb Henk Slager:
Quoted 5 lines · M G Berberich
Quoted 4 lines
Am Montag, den 27. Juni schrieb M G Berberich:
Quoted 2 lines
after a balance ‘btrfs filesystem du’ probably shows false data about shared data.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.
With btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct about shared data.
I tested with kernels 4.6.3 and 4.7-rc5 and with btrfs-progs 4.5.2 and 4.61.
Also with kernel 4.6.2-1-default and btrfs-progs v4.5.3+20160516 (current stock opensuse tumbleweed) I cannot reproduce the problem.
The later kernel with two patches to make the kernel work:
... so these seem to cause the problem
Quoted 29 lines
You can see the script¹ I used (do-btrfs-du-test) and the logs at
http://m-berberich.de/btrfs/In all four cases, ‘btrfs fi du -s .’ reports
Total Exclusive Set shared Filename
59.38MiB 0.00B 29.69MiB .befor balance and
Total Exclusive Set shared Filename
59.38MiB 59.38MiB 0.00B .after balance.
Disclaimer: The script works for me, no guaranty at all.
MfG
bmg
__________
¹ Disclaimer: The script works for me, no guaranty at all.
--
„Des is völlig wurscht, was heut beschlos- | M G Berberich
sen wird: I bin sowieso dagegn!“ | [email protected]
(SPD-Stadtrat Kurt Schindler; Regensburg) |
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
28.06.2016 19:55, Henk Slager пишет:
Quoted 11 lines · M G Berberich
Quoted 10 lines
Hello,
Am Montag, den 27. Juni schrieb Henk Slager:
Quoted 7 lines · M G Berberich
Quoted 5 lines
Am Montag, den 27. Juni schrieb M G Berberich:
Quoted 2 lines
after a balance ‘btrfs filesystem du’ probably shows false data about shared data.
Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
With btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct about shared data.
I tested with kernels 4.6.3 and 4.7-rc5 and with btrfs-progs 4.5.2 and 4.61.
Also with kernel 4.6.2-1-default and btrfs-progs v4.5.3+20160516 (current stock opensuse tumbleweed) I cannot reproduce the problem.
I confirm the same behavior on openSUSE Tumbleweed with kernel 4.6.2-1.2 and btrfsprogs 4.5.3-1.2 using provided script.
Quoted 39 lines
The later kernel with two patches to make the kernel work:
... so these seem to cause the problem
Quoted 17 lines
You can see the script¹ I used (do-btrfs-du-test) and the logs at
http://m-berberich.de/btrfs/In all four cases, ‘btrfs fi du -s .’ reports
Total Exclusive Set shared Filename
59.38MiB 0.00B 29.69MiB .befor balance and
Total Exclusive Set shared Filename
59.38MiB 59.38MiB 0.00B .after balance.
Disclaimer: The script works for me, no guaranty at all.
Quoted 12 lines
MfG
bmg
__________
¹ Disclaimer: The script works for me, no guaranty at all.
--
„Des is völlig wurscht, was heut beschlos- | M G Berberich
sen wird: I bin sowieso dagegn!“ | [email protected]
(SPD-Stadtrat Kurt Schindler; Regensburg) |
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
28.06.2016 20:20, Andrei Borzenkov пишет:
Quoted 15 lines
28.06.2016 19:55, Henk Slager пишет:
Quoted 14 lines · M G Berberich
Quoted 13 lines
Hello,
Am Montag, den 27. Juni schrieb Henk Slager:
Quoted 9 lines · M G Berberich
Quoted 5 lines
Am Montag, den 27. Juni schrieb M G Berberich:
Quoted 2 lines
after a balance ‘btrfs filesystem du’ probably shows false data about shared data.
Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.
With btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct about shared data.
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines
I tested with kernels 4.6.3 and 4.7-rc5 and with btrfs-progs 4.5.2 and 4.61.
Also with kernel 4.6.2-1-default and btrfs-progs v4.5.3+20160516 (current stock opensuse tumbleweed) I cannot reproduce the problem.
I confirm the same behavior on openSUSE Tumbleweed with kernel 4.6.2-1.2 and btrfsprogs 4.5.3-1.2 using provided script.
I realized that it sounded ambiguous. I do see the reported bug on openSUSE Tumbleweed using the same versions as you. After rebalance shared data disappears completely in du output.