Skip to contentSkip to conversation

Accepting discard to free space from disk images

DISCUSSION Devon B. 3 participants 9 messages Feb 9, 2015 2 messages not in the archive Btrfs file system

lore
Share & tools
lore

Devon B.

author
#0 lore raw

Looking to use btrfs with disk images that contain ext4, xfs, and other possible filesystems. One of my main concerns is reclaiming disk space when files are deleted on the disk image's filesystem. Using fstrim on the mount point of the disk image or mounting the disk image (loop) with discard doesn't appear to pass the freed blocks to btrfs.

The image file on the btrfs filesystem just keeps growing in size. When hosting images on ext4, using fstrim or discard frees up the space on the host filesystem (ext4).

I see discard is pretty well supported for btrfs to the underlying block devices but is there any way to properly free space on the btrfs host filesystem from overlying filesystems while they are online?

Thanks.

#1 lore raw
Quoted 6 lines · On Mon, 09 Feb 2015 00:17:49 -0500 "Devon B." <[email protected]>…

Looking to use btrfs with disk images that contain ext4, xfs, and other possible filesystems. One of my main concerns is reclaiming disk space when files are deleted on the disk image's filesystem. Using fstrim on the mount point of the disk image or mounting the disk image (loop) with discard doesn't appear to pass the freed blocks to btrfs.

Quoted 7 lines

The image file on the btrfs filesystem just keeps growing in size. When hosting images on ext4, using fstrim or discard frees up the space on the host filesystem (ext4).

I see discard is pretty well supported for btrfs to the underlying block devices but is there any way to properly free space on the btrfs host filesystem from overlying filesystems while they are online?

I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?

Signature · 2 lines
With respect,
Roman
#2 lore raw
Quoted 17 lines · Roman Mamedov
Quoted 6 lines · On Mon, 09 Feb 2015 00:17:49 -0500 "Devon B." <[email protected]>…

Looking to use btrfs with disk images that contain ext4, xfs, and other possible filesystems. One of my main concerns is reclaiming disk space when files are deleted on the disk image's filesystem. Using fstrim on the mount point of the disk image or mounting the disk image (loop) with discard doesn't appear to pass the freed blocks to btrfs.

Quoted 7 lines

The image file on the btrfs filesystem just keeps growing in size. When hosting images on ext4, using fstrim or discard frees up the space on the host filesystem (ext4).

I see discard is pretty well supported for btrfs to the underlying block devices but is there any way to properly free space on the btrfs host filesystem from overlying filesystems while they are online?

Quoted 5 lines

I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?

This is interesting, because I've tried exactly that setup (KVM,
discard=unmap) and it's not worked for me.

Hugo.

Signature · 4 lines
Hugo Mills             | If you see something, say nothing and drink to
hugo@... carfax.org.uk | forget
http://carfax.org.uk/  |
PGP: 65E74AC0          |                                 Welcome to Night Vale
#3 lore raw
Quoted 8 lines · On Mon, 9 Feb 2015 08:39:41 +0000 Hugo Mills <[email protected]> wrote:
Quoted 5 lines

I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?

This is interesting, because I've tried exactly that setup (KVM,
discard=unmap) and it's not worked for me.

Did you use the emulated IDE interface in KVM? Others such as virtio, ahci and maybe SCSI may not support discard. Did you get a successful result issuing TRIM within the guest (e.g. fstrim -v / succeeds and doesn't -ENOSUPP)?

Signature · 2 lines
With respect,
Roman
#4 lore raw
Quoted 9 lines · Roman Mamedov
Quoted 6 lines · On Mon, 9 Feb 2015 08:39:41 +0000 Hugo Mills <[email protected]> wrote:
Quoted 5 lines

I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?

Quoted 6 lines
Quoted 2 lines

This is interesting, because I've tried exactly that setup (KVM,
discard=unmap) and it's not worked for me.

Did you use the emulated IDE interface in KVM? Others such as virtio, ahci and maybe SCSI may not support discard. Did you get a successful result issuing TRIM within the guest (e.g. fstrim -v / succeeds and doesn't -ENOSUPP)?

I used SCSI, so maybe that's the problem. I'll try with IDE.

Hugo.

Signature · 4 lines
Hugo Mills             | I was cursed with poetry very young. It creates
hugo@... carfax.org.uk | unrealistic expectations.
http://carfax.org.uk/  |                                   Victor Frankenstein
PGP: 65E74AC0          |                                        Penny Dreadful

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw
Quoted 2 lines · On Mon, 09 Feb 2015 10:26:33 -0500 "Devon B." <[email protected]>…

If you don't mind me asking, what version kernel are you running and are you using any special mount options?

Well actually I did not claim I have working discard through 'loop', but your post made me curious.

$ sudo dd if=/dev/zero of=100g bs=1M seek=100000 count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00221052 s, 474 MB/s
$ sudo mkfs.ext4 100g
[...]
$ du -hsc 100g 
133M	100g
133M	total
$ sudo mount -o loop 100g /mnt/tmp1/

(then in a new terminal window):

$ cd /mnt/tmp1/
$ df -h .
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop0       96G   60M   92G   1% /mnt/tmp1
$ sudo dd if=/dev/zero of=zerofile bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.944377 s, 1.1 GB/s
$ sync

(back to the original one):

$ du -hsc 100g 
1.2G	100g
1.2G	total

(2nd window):

$ sudo fstrim .

(back to the original one):

$ du -hsc 100g 
133M	100g
133M	total

So it does work for me just fine even with 'loop'. Kernel version 3.14.32, mount options rw,noatime,nodiratime,compress=zlib,space_cache,inode_cache.

Signature · 2 lines
With respect,
Roman
#7 lore raw
Quoted 34 lines · On Mon, 9 Feb 2015 20:42:56 +0500 Roman Mamedov <[email protected]> wrote:
Quoted 2 lines · On Mon, 09 Feb 2015 10:26:33 -0500 "Devon B." <[email protected]>…

If you don't mind me asking, what version kernel are you running and are you using any special mount options?

Well actually I did not claim I have working discard through 'loop', but your post made me curious.

$ sudo dd if=/dev/zero of=100g bs=1M seek=100000 count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00221052 s, 474 MB/s
$ sudo mkfs.ext4 100g
[...]
$ du -hsc 100g 
133M	100g
133M	total
$ sudo mount -o loop 100g /mnt/tmp1/

(then in a new terminal window):

$ cd /mnt/tmp1/
$ df -h .
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop0       96G   60M   92G   1% /mnt/tmp1
$ sudo dd if=/dev/zero of=zerofile bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.944377 s, 1.1 GB/s
$ sync
Quoted 6 lines

(back to the original one):

$ du -hsc 100g 
1.2G	100g
1.2G	total

(2nd window):

Forgot to add I also did 'rm zerofile' here, of course.

Quoted 11 lines
$ sudo fstrim .

(back to the original one):

$ du -hsc 100g 
133M	100g
133M	total

So it does work for me just fine even with 'loop'. Kernel version 3.14.32, mount options rw,noatime,nodiratime,compress=zlib,space_cache,inode_cache.

Signature · 2 lines
With respect,
Roman

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw
Quoted 7 lines · On Mon, 09 Feb 2015 12:07:18 -0500 "Devon B." <[email protected]>…

Thanks for your testing. I haven't tried 3.14. I tried on CentOS 6 box (2.6.32 - which is experimental) and Ubuntu 14.04 (3.13) and neither worked. So the question remains, what is the difference? Possibly a small difference between the 3.13 and 3.14 kernels, I don't think it is any of the mount options. I guess if anyone else has insight on this, that would be great. Otherwise, I'll see if I can get a newer kernel loaded up and do some more testing.

Use "Reply to all" and not just "Reply", or "anyone else" won't see your question.

BTW it is not a good idea to use Btrfs on 2.6.32.

Signature · 2 lines
With respect,
Roman
author
#10 lore raw
In reply to Roman Mamedov

Thanks for your testing. I haven't tried 3.14. I tried on CentOS 6 box (2.6.32 - which is experimental) and Ubuntu 14.04 (3.13) and neither worked. So the question remains, what is the difference? Possibly a small difference between the 3.13 and 3.14 kernels, I don't think it is any of the mount options. I guess if anyone else has insight on this, that would be great. Otherwise, I'll see if I can get a newer kernel loaded up and do some more testing.

Quoted 197 lines

Roman Mamedov <mailto:[email protected]>
Monday, February 9, 2015 10:45 AM

Quoted 33 lines · On Mon, 9 Feb 2015 20:42:56 +0500 Roman Mamedov<[email protected]> wrote:
Quoted 2 lines · On Mon, 09 Feb 2015 10:26:33 -0500 "Devon B."<[email protected]>…

If you don't mind me asking, what version kernel are you running and are you using any special mount options?

Well actually I did not claim I have working discard through 'loop', but your post made me curious.

$ sudo dd if=/dev/zero of=100g bs=1M seek=100000 count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00221052 s, 474 MB/s
$ sudo mkfs.ext4 100g
[...]
$ du -hsc 100g
133M	100g
133M	total
$ sudo mount -o loop 100g /mnt/tmp1/

(then in a new terminal window):

$ cd /mnt/tmp1/
$ df -h .
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop0       96G   60M   92G   1% /mnt/tmp1
$ sudo dd if=/dev/zero of=zerofile bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.944377 s, 1.1 GB/s
$ sync
Quoted 6 lines

(back to the original one):

$ du -hsc 100g
1.2G	100g
1.2G	total

(2nd window):

Forgot to add I also did 'rm zerofile' here, of course.

Quoted 11 lines
$ sudo fstrim .

(back to the original one):

$ du -hsc 100g
133M	100g
133M	total

So it does work for me just fine even with 'loop'. Kernel version 3.14.32, mount options rw,noatime,nodiratime,compress=zlib,space_cache,inode_cache.

Roman Mamedov <mailto:[email protected]>
Monday, February 9, 2015 10:42 AM
On Mon, 09 Feb 2015 10:26:33 -0500

Well actually I did not claim I have working discard through 'loop', but your post made me curious.

$ sudo dd if=/dev/zero of=100g bs=1M seek=100000 count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00221052 s, 474 MB/s
$ sudo mkfs.ext4 100g
[...]
$ du -hsc 100g
133M 100g
133M total
$ sudo mount -o loop 100g /mnt/tmp1/

(then in a new terminal window):

$ cd /mnt/tmp1/
$ df -h .
Filesystem Size Used Avail Use% Mounted on
/dev/loop0 96G 60M 92G 1% /mnt/tmp1
$ sudo dd if=/dev/zero of=zerofile bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.944377 s, 1.1 GB/s
$ sync

(back to the original one):

$ du -hsc 100g
1.2G 100g
1.2G total

(2nd window):

$ sudo fstrim .

(back to the original one):

$ du -hsc 100g
133M 100g
133M total

So it does work for me just fine even with 'loop'. Kernel version 3.14.32, mount options rw,noatime,nodiratime,compress=zlib,space_cache,inode_cache.

Devon B. <mailto:[email protected]>
Monday, February 9, 2015 10:26 AM
If you don't mind me asking, what version kernel are you running and are you using any special mount options?

Here is a quick example:

# qemu-img create -f raw /btrfs/sub/raw.img 100G
Formatting '/btrfs/sub/raw.img', fmt=raw size=107374182400
# mkfs.ext4 /btrfs/sub/raw.img
...
# mount -o loop /btrfs/sub/raw.img /mnt/test
# du -hs /btrfs/sub/raw.img
1.7G    /btrfs/sub/raw.img
# fstrim -v /mnt/test
/mnt/test: 105492688896 bytes were trimmed
# du -hs /btrfs/sub/raw.img
1.7G    /btrfs/sub/raw.img
# dd if=/dev/zero of=/mnt/test/1GB count=1k bs=1M
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.493057 s, 2.2 GB/s
# du -hs /btrfs/sub/raw.img
2.7G    /btrfs/sub/raw.img
# rm -f /mnt/test/1GB
# fstrim -v /mnt/test
/mnt/test: 1186967552 bytes were trimmed
# du -hs /btrfs/sub/raw.img
2.7G    /btrfs/sub/raw.img
# dd if=/dev/zero of=/mnt/test/1GB count=1k bs=1M
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 0.467089 s, 2.3 GB/s
# du -hs /btrfs/sub/raw.img
3.7G    /btrfs/sub/raw.img
# rm -f /mnt/test/1GB
# fstrim -v /mnt/test
/mnt/test: 1203761152 bytes were trimmed
# du -hs /btrfs/sub/raw.img
3.7G    /btrfs/sub/raw.img
# du -hs /mnt/test
20K     /mnt/test

So even though there is nothing in /mnt/test, the disk image is consuming 3.7GB of space. Maybe you could test similarly with your server if you have time on your hands.

Thanks!
Roman Mamedov <mailto:[email protected]>
Monday, February 9, 2015 1:40 AM
On Mon, 09 Feb 2015 00:17:49 -0500

I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?

Devon B. <mailto:[email protected]>
Monday, February 9, 2015 12:17 AM
Looking to use btrfs with disk images that contain ext4, xfs, and other possible filesystems. One of my main concerns is reclaiming disk space when files are deleted on the disk image's filesystem. Using fstrim on the mount point of the disk image or mounting the disk image (loop) with discard doesn't appear to pass the freed blocks to btrfs.

The image file on the btrfs filesystem just keeps growing in size. When hosting images on ext4, using fstrim or discard frees up the space on the host filesystem (ext4).

I see discard is pretty well supported for btrfs to the underlying block devices but is there any way to properly free space on the btrfs host filesystem from overlying filesystems while they are online?

Thanks.