Effect of punching holes
Hi,
I noticed that if you punch a hole in the middle of a file the available filesystem space seems not to increase.
Kernel is 5.2.11
To reproduce:
->mkfs.btrfs /dev/loop1 -f
btrfs-progs v4.15.1
See http://btrfs.wiki.kernel.org for more information.
Detected a SSD, turning off metadata duplication. Mkfs with -m dup if you want to force metadata duplication. Label: (null)
UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587
Node size: 16384
Sector size: 4096
Filesystem size: 1.00GiB
Block group profiles:
Data: single 8.00MiB
Metadata: single 8.00MiB
System: single 4.00MiB
SSD detected: yes
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 1.00GiB /dev/loop1
->mount /dev/loop1 /srv/btrtest2
->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ; sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done
this failed from the 16th file on because of no space left
->df -T .
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 btrfs 1048576 935856 2272 100% /srv/btrtest2
->btrfs fi du .
Total Exclusive Set shared Filename
8.00KiB 8.00KiB - ./test1
8.00KiB 8.00KiB - ./test2
8.00KiB 8.00KiB - ./test3
8.00KiB 8.00KiB - ./test4
8.00KiB 8.00KiB - ./test5
8.00KiB 8.00KiB - ./test6
8.00KiB 8.00KiB - ./test7
8.00KiB 8.00KiB - ./test8
8.00KiB 8.00KiB - ./test9
8.00KiB 8.00KiB - ./test10
8.00KiB 8.00KiB - ./test11
8.00KiB 8.00KiB - ./test12
8.00KiB 8.00KiB - ./test13
8.00KiB 8.00KiB - ./test14
8.00KiB 8.00KiB - ./test15
4.00KiB 4.00KiB - ./test16
4.00KiB 4.00KiB - ./test17
4.00KiB 4.00KiB - ./test18
4.00KiB 4.00KiB - ./test19
4.00KiB 4.00KiB - ./test20
140.00KiB 140.00KiB 0.00B .
When doing this on XFS or EXT4 it works as expected:
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 ext4 999320 2764 927744 1% /srv/btrtest /dev/loop2 xfs 1038336 40456 997880 4% /srv/xfstest
How to i reclaim the space on BTRFS? Defrag does not seem to help.
Best regards
Tobias
Quoted 33 lines · Tobias Reinhard
Hi,
I noticed that if you punch a hole in the middle of a file the available filesystem space seems not to increase.
Kernel is 5.2.11
To reproduce:
->mkfs.btrfs
/dev/loop1-fbtrfs-progs v4.15.1
See http://btrfs.wiki.kernel.org for more information.Detected a SSD, turning off metadata duplication. Mkfs with -m dup if you want to force metadata duplication.
Label: (null)
UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587 Node size: 16384
Sector size: 4096
Filesystem size: 1.00GiB
Block group profiles:
Data: single 8.00MiB
Metadata: single 8.00MiB
System: single 4.00MiB
SSD detected: yes
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 1.00GiB/dev/loop1
Quoted 6 lines
->mount
/dev/loop1/srv/btrtest2->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ; sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done
this failed from the 16th file on because of no space left
Btrfs doesn't free the space until all space of a data extent get freed.
In your case, your hole punch is [4k, 64M-4K), thus the 64M extent still has 4K being used.
So the data extent won't be freed until you free the last 4K.
Quoted 29 lines
->df -T .
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 btrfs 1048576 935856 2272 100%/srv/btrtest2->btrfs fi du .
Total Exclusive Set shared Filename
8.00KiB 8.00KiB - ./test1
8.00KiB 8.00KiB - ./test2
8.00KiB 8.00KiB - ./test3
8.00KiB 8.00KiB - ./test4
8.00KiB 8.00KiB - ./test5
8.00KiB 8.00KiB - ./test6
8.00KiB 8.00KiB - ./test7
8.00KiB 8.00KiB - ./test8
8.00KiB 8.00KiB - ./test9
8.00KiB 8.00KiB - ./test10
8.00KiB 8.00KiB - ./test11
8.00KiB 8.00KiB - ./test12
8.00KiB 8.00KiB - ./test13
8.00KiB 8.00KiB - ./test14
8.00KiB 8.00KiB - ./test15
4.00KiB 4.00KiB - ./test16
4.00KiB 4.00KiB - ./test17
4.00KiB 4.00KiB - ./test18
4.00KiB 4.00KiB - ./test19
4.00KiB 4.00KiB - ./test20
140.00KiB 140.00KiB 0.00B .
Quoted 7 lines
When doing this on XFS or EXT4 it works as expected:
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 ext4 999320 2764 927744 1%
/srv/btrtest/dev/loop2 xfs 1038336 40456 997880 4%/srv/xfstestHow to i reclaim the space on BTRFS? Defrag does not seem to help.
Rewrite the remaining 4K.
Then the new write 4K will be cowed into a new 4K extent, the old large 64M extent gets fully freed and free space.
Thanks,
Qu
Quoted 6 lines
Best regards
Tobias
Quoted 84 lines · Qu Wenruo
Quoted 33 lines · Tobias Reinhard
Hi,
I noticed that if you punch a hole in the middle of a file the available filesystem space seems not to increase.
Kernel is 5.2.11
To reproduce:
->mkfs.btrfs
/dev/loop1-fbtrfs-progs v4.15.1
See http://btrfs.wiki.kernel.org for more information.Detected a SSD, turning off metadata duplication. Mkfs with -m dup if you want to force metadata duplication.
Label: (null)
UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587 Node size: 16384
Sector size: 4096
Filesystem size: 1.00GiB
Block group profiles:
Data: single 8.00MiB
Metadata: single 8.00MiB
System: single 4.00MiB
SSD detected: yes
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 1.00GiB/dev/loop1Quoted 6 lines
->mount
/dev/loop1/srv/btrtest2->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ; sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done
this failed from the 16th file on because of no space left
Btrfs doesn't free the space until all space of a data extent get freed.
In your case, your hole punch is [4k, 64M-4K), thus the 64M extent still has 4K being used.
So the data extent won't be freed until you free the last 4K.Quoted 35 lines
->df -T .
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 btrfs 1048576 935856 2272 100%/srv/btrtest2->btrfs fi du .
Total Exclusive Set shared Filename
8.00KiB 8.00KiB - ./test1
8.00KiB 8.00KiB - ./test2
8.00KiB 8.00KiB - ./test3
8.00KiB 8.00KiB - ./test4
8.00KiB 8.00KiB - ./test5
8.00KiB 8.00KiB - ./test6
8.00KiB 8.00KiB - ./test7
8.00KiB 8.00KiB - ./test8
8.00KiB 8.00KiB - ./test9
8.00KiB 8.00KiB - ./test10
8.00KiB 8.00KiB - ./test11
8.00KiB 8.00KiB - ./test12
8.00KiB 8.00KiB - ./test13
8.00KiB 8.00KiB - ./test14
8.00KiB 8.00KiB - ./test15
4.00KiB 4.00KiB - ./test16
4.00KiB 4.00KiB - ./test17
4.00KiB 4.00KiB - ./test18
4.00KiB 4.00KiB - ./test19
4.00KiB 4.00KiB - ./test20
140.00KiB 140.00KiB 0.00B .When doing this on XFS or EXT4 it works as expected:
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 ext4 999320 2764 927744 1%
/srv/btrtest/dev/loop2 xfs 1038336 40456 997880 4%/srv/xfstest
Quoted 6 lines
Quoted 1 line
How to i reclaim the space on BTRFS? Defrag does not seem to help.
Rewrite the remaining 4K.
Then the new write 4K will be cowed into a new 4K extent, the old large 64M extent gets fully freed and free space.
Expanding on this a bit, defrag isn't working here because it doesn't, by default, touch extents larger than 32M in size. You should be able to make it work by using the -t option with a size larger than 64M.
Alternatively, use cp --reflink=never --sparse=always to copy the file and then rename the copy over the original. This will use more space, but is likely to be significantly faster than a defrag.
Am Di., 22. Okt. 2019 um 15:04 Uhr schrieb Austin S. Hemmelgarn <[email protected] <mailto:[email protected]>>:
Quoted 7 lines · Qu Wenruo
Quoted 4 lines · Tobias Reinhard
Hi,
I noticed that if you punch a hole in the middle of a file the
available
Quoted 4 lines
Quoted 4 lines
filesystem space seems not to increase.
Kernel is 5.2.11
Quoted 8 lines
Quoted 8 lines
To reproduce:
->mkfs.btrfs
/dev/loop1-fbtrfs-progs v4.15.1
See http://btrfs.wiki.kernel.org for more information.Detected a SSD, turning off metadata duplication. Mkfs with -m
dup if
Quoted 17 lines
Quoted 17 lines
you want to force metadata duplication.
Label: (null)
UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587
Node size: 16384
Sector size: 4096
Filesystem size: 1.00GiB
Block group profiles:
Data: single 8.00MiB
Metadata: single 8.00MiB
System: single 4.00MiB
SSD detected: yes
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 1.00GiB/dev/loop1
Quoted 3 lines
Quoted 3 lines
->mount
/dev/loop1/srv/btrtest2->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M
count=4 ;
>> sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done
>>
>> this failed from the 16th file on because of no space left
>
> Btrfs doesn't free the space until all space of a data extent getfreed.
Quoted 2 lines
In your case, your hole punch is [4k, 64M-4K), thus the 64M
extent still
Quoted 38 lines
has 4K being used.
So the data extent won't be freed until you free the last 4K.Quoted 35 lines
->df -T .
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 btrfs 1048576 935856 2272 100%/srv/btrtest2->btrfs fi du .
Total Exclusive Set shared Filename
8.00KiB 8.00KiB - ./test1
8.00KiB 8.00KiB - ./test2
8.00KiB 8.00KiB - ./test3
8.00KiB 8.00KiB - ./test4
8.00KiB 8.00KiB - ./test5
8.00KiB 8.00KiB - ./test6
8.00KiB 8.00KiB - ./test7
8.00KiB 8.00KiB - ./test8
8.00KiB 8.00KiB - ./test9
8.00KiB 8.00KiB - ./test10
8.00KiB 8.00KiB - ./test11
8.00KiB 8.00KiB - ./test12
8.00KiB 8.00KiB - ./test13
8.00KiB 8.00KiB - ./test14
8.00KiB 8.00KiB - ./test15
4.00KiB 4.00KiB - ./test16
4.00KiB 4.00KiB - ./test17
4.00KiB 4.00KiB - ./test18
4.00KiB 4.00KiB - ./test19
4.00KiB 4.00KiB - ./test20
140.00KiB 140.00KiB 0.00B .When doing this on XFS or EXT4 it works as expected:
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 ext4 999320 2764 927744 1%
/srv/btrtest/dev/loop2 xfs 1038336 40456 997880 4%/srv/xfstest
Quoted 5 lines
Quoted 1 line
How to i reclaim the space on BTRFS? Defrag does not seem to help.
Rewrite the remaining 4K.
Then the new write 4K will be cowed into a new 4K extent, the old
large
Quoted 1 line
64M extent gets fully freed and free space.
Expanding on this a bit, defrag isn't working here because it doesn't, by default, touch extents larger than 32M in size. You should be able to make it work by using the -t option with a size larger than 64M.
Alternatively, use cp --reflink=never --sparse=always to copy the file
and then rename the copy over the original. This will use more space, but is likely to be significantly faster than a defrag.
(sorry - for first bad formated post)
Hi,
I can't get the defrag way to work.
What is the right command to do it?
->df -hT .
Filesystem Type Size Used Avail Use% Mounted on /dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2
->btrfs fi du .
Total Exclusive Set shared Filename 0.00B 0.00B - ./runtest.sh 8.00KiB 8.00KiB - ./test1 8.00KiB 8.00KiB - ./test2 8.00KiB 8.00KiB - ./test3 8.00KiB 8.00KiB - ./test4 8.00KiB 8.00KiB - ./test5 8.00KiB 8.00KiB - ./test6 8.00KiB 8.00KiB - ./test7 8.00KiB 8.00KiB - ./test8 8.00KiB 8.00KiB - ./test9 8.00KiB 8.00KiB - ./test10 8.00KiB 8.00KiB - ./test11 8.00KiB 8.00KiB - ./test12 8.00KiB 8.00KiB - ./test13 8.00KiB 8.00KiB - ./test14 8.00KiB 8.00KiB - ./test15 120.00KiB 120.00KiB 0.00B .
-> btrfs fi de -t 128M *
-> sync
-> df -hT .
Filesystem Type Size Used Avail Use% Mounted on /dev/loop1 btrfs 1,0G 868M 49M 95% /srv/btrtest2
Tobias
Quoted 137 lines · Tobias Reinhard
Am Di., 22. Okt. 2019 um 15:04 Uhr schrieb Austin S. Hemmelgarn <[email protected] <mailto:[email protected]>>:
On 2019-10-22 06:01, Qu Wenruo wrote:
>
>
> On 2019/10/22 下午5:47, Tobias Reinhard wrote:
>> Hi,
>>
>>
>> I noticed that if you punch a hole in the middle of a file the available
>> filesystem space seems not to increase.
>>
>> Kernel is 5.2.11
>>
>> To reproduce:
>>
>> ->mkfs.btrfs/dev/loop1-f
>>
>> btrfs-progs v4.15.1
>> See http://btrfs.wiki.kernel.org for more information. >>
>> Detected a SSD, turning off metadata duplication. Mkfs with -m dup if
>> you want to force metadata duplication.
>> Label: (null)
>> UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587
>> Node size: 16384
>> Sector size: 4096
>> Filesystem size: 1.00GiB
>> Block group profiles:
>> Data: single 8.00MiB
>> Metadata: single 8.00MiB
>> System: single 4.00MiB
>> SSD detected: yes
>> Incompat features: extref, skinny-metadata
>> Number of devices: 1
>> Devices:
>> ID SIZE PATH
>> 1 1.00GiB/dev/loop1
>>
>> ->mount/dev/loop1/srv/btrtest2
>>
>> ->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M count=4 ;
>> sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done >>
>> this failed from the 16th file on because of no space left >
> Btrfs doesn't free the space until all space of a data extent get freed.
>
> In your case, your hole punch is [4k, 64M-4K), thus the 64M extent still
> has 4K being used.
> So the data extent won't be freed until you free the last 4K. >
>>
>> ->df -T .
>> Filesystem Type 1K-blocks Used Available Use% Mounted on >> /dev/loop1 btrfs 1048576 935856 2272 100%/srv/btrtest2>>
>> ->btrfs fi du .
>> Total Exclusive Set shared Filename
>> 8.00KiB 8.00KiB - ./test1
>> 8.00KiB 8.00KiB - ./test2
>> 8.00KiB 8.00KiB - ./test3
>> 8.00KiB 8.00KiB - ./test4
>> 8.00KiB 8.00KiB - ./test5
>> 8.00KiB 8.00KiB - ./test6
>> 8.00KiB 8.00KiB - ./test7
>> 8.00KiB 8.00KiB - ./test8
>> 8.00KiB 8.00KiB - ./test9
>> 8.00KiB 8.00KiB - ./test10
>> 8.00KiB 8.00KiB - ./test11
>> 8.00KiB 8.00KiB - ./test12
>> 8.00KiB 8.00KiB - ./test13
>> 8.00KiB 8.00KiB - ./test14
>> 8.00KiB 8.00KiB - ./test15
>> 4.00KiB 4.00KiB - ./test16
>> 4.00KiB 4.00KiB - ./test17
>> 4.00KiB 4.00KiB - ./test18
>> 4.00KiB 4.00KiB - ./test19
>> 4.00KiB 4.00KiB - ./test20
>> 140.00KiB 140.00KiB 0.00B .
>>
>> When doing this on XFS or EXT4 it works as expected:
>>
>> Filesystem Type 1K-blocks Used Available Use% Mounted on >> /dev/loop1 ext4 999320 2764 927744 1%/srv/btrtest>> /dev/loop2 xfs 1038336 40456 997880 4%/srv/xfstest>>
>> How to i reclaim the space on BTRFS? Defrag does not seem to help. >
> Rewrite the remaining 4K.
>
> Then the new write 4K will be cowed into a new 4K extent, the old large
> 64M extent gets fully freed and free space.Expanding on this a bit, defrag isn't working here because it doesn't, by default, touch extents larger than 32M in size. You should be able to make it work by using the
-toption with a size larger than 64M.Alternatively, use
cp --reflink=never --sparse=alwaysto copy the file
and then rename the copy over the original. This will use more space, but is likely to be significantly faster than a defrag.(sorry - for first bad formated post)
Hi,
I can't get the defrag way to work.
What is the right command to do it?
->df -hT .
Filesystem Type Size Used Avail Use% Mounted on/dev/loop1btrfs 1,0G 868M 49M 95%/srv/btrtest2->btrfs fi du .
Total Exclusive Set shared Filename
0.00B 0.00B - ./runtest.sh
8.00KiB 8.00KiB - ./test1
8.00KiB 8.00KiB - ./test2
8.00KiB 8.00KiB - ./test3
8.00KiB 8.00KiB - ./test4
8.00KiB 8.00KiB - ./test5
8.00KiB 8.00KiB - ./test6
8.00KiB 8.00KiB - ./test7
8.00KiB 8.00KiB - ./test8
8.00KiB 8.00KiB - ./test9
8.00KiB 8.00KiB - ./test10
8.00KiB 8.00KiB - ./test11
8.00KiB 8.00KiB - ./test12
8.00KiB 8.00KiB - ./test13
Quoted 8 lines
8.00KiB 8.00KiB - ./test14 8.00KiB 8.00KiB - ./test15 120.00KiB 120.00KiB 0.00B .
-> btrfs fi de -t 128M *
-> sync
-> df -hT .
Filesystem Type Size Used Avail Use% Mounted on/dev/loop1btrfs 1,0G 868M 49M 95%/srv/btrtest2
That's odd, what you used _should_ do it. For some reason, it's not trying to rewrite things at all. In cases like this, you can force it to rewrite the data by telling it to compress the file using a different algorithm than whatever you have specified via mount options (or just compress it at all if you don't have compression enabled for the mount) and then re-defraging it with the original compression type.
Am Di., 22. Okt. 2019 um 12:01 Uhr schrieb Qu Wenruo <[email protected] <mailto:[email protected]>>:
Quoted 4 lines · Tobias Reinhard
Hi,
I noticed that if you punch a hole in the middle of a file the
available
Quoted 4 lines
filesystem space seems not to increase.
Kernel is 5.2.11
Quoted 8 lines
To reproduce:
->mkfs.btrfs
/dev/loop1-fbtrfs-progs v4.15.1
See http://btrfs.wiki.kernel.org for more information.Detected a SSD, turning off metadata duplication. Mkfs with -m
dup if
Quoted 17 lines
you want to force metadata duplication.
Label: (null)
UUID: 415e925a-588a-4b8f-bdc7-c30a4a0f5587 Node size: 16384
Sector size: 4096
Filesystem size: 1.00GiB
Block group profiles:
Data: single 8.00MiB Metadata: single 8.00MiB System: single 4.00MiB SSD detected: yes
Incompat features: extref, skinny-metadata Number of devices: 1
Devices:
ID SIZE PATH
1 1.00GiB/dev/loop1
Quoted 3 lines
->mount
/dev/loop1/srv/btrtest2->for i in $(seq 1 20); do dd if=/dev/urandom of=test$i bs=16M
count=4 ;
> sync ; fallocate -p -o 4096 -l 67100672 test$i && sync ; done
>
> this failed from the 16th file on because of no space leftBtrfs doesn't free the space until all space of a data extent get freed.
In your case, your hole punch is [4k, 64M-4K), thus the 64M extent still has 4K being used.
So the data extent won't be freed until you free the last 4K.
Quoted 29 lines
->df -T .
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 btrfs 1048576 935856 2272 100%/srv/btrtest2->btrfs fi du .
Total Exclusive Set shared Filename
8.00KiB 8.00KiB - ./test1
8.00KiB 8.00KiB - ./test2
8.00KiB 8.00KiB - ./test3
8.00KiB 8.00KiB - ./test4
8.00KiB 8.00KiB - ./test5
8.00KiB 8.00KiB - ./test6
8.00KiB 8.00KiB - ./test7
8.00KiB 8.00KiB - ./test8
8.00KiB 8.00KiB - ./test9
8.00KiB 8.00KiB - ./test10
8.00KiB 8.00KiB - ./test11
8.00KiB 8.00KiB - ./test12
8.00KiB 8.00KiB - ./test13
8.00KiB 8.00KiB - ./test14
8.00KiB 8.00KiB - ./test15
4.00KiB 4.00KiB - ./test16
4.00KiB 4.00KiB - ./test17
4.00KiB 4.00KiB - ./test18
4.00KiB 4.00KiB - ./test19
4.00KiB 4.00KiB - ./test20
140.00KiB 140.00KiB 0.00B .
Quoted 7 lines
When doing this on XFS or EXT4 it works as expected:
Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/loop1 ext4 999320 2764 927744 1%
/srv/btrtest/dev/loop2 xfs 1038336 40456 997880 4%/srv/xfstestHow to i reclaim the space on BTRFS? Defrag does not seem to help.
Rewrite the remaining 4K.
Then the new write 4K will be cowed into a new 4K extent, the old large 64M extent gets fully freed and free space.
Thanks,
Qu
(sorry - for previous bad formated post)
Hi,
okay - thanks for the explanation.
How can I find out that rewriting parts of an extent would free up space?
Tobias