Accepting discard to free space from disk images
Looking to use btrfs with disk images that contain ext4, xfs, and other possible filesystems. One of my main concerns is reclaiming disk space when files are deleted on the disk image's filesystem. Using fstrim on the mount point of the disk image or mounting the disk image (loop) with discard doesn't appear to pass the freed blocks to btrfs.
The image file on the btrfs filesystem just keeps growing in size. When hosting images on ext4, using fstrim or discard frees up the space on the host filesystem (ext4).
I see discard is pretty well supported for btrfs to the underlying block devices but is there any way to properly free space on the btrfs host filesystem from overlying filesystems while they are online?
Thanks.
Quoted 6 lines · On Mon, 09 Feb 2015 00:17:49 -0500 "Devon B." <[email protected]>…
Looking to use btrfs with disk images that contain ext4, xfs, and other possible filesystems. One of my main concerns is reclaiming disk space when files are deleted on the disk image's filesystem. Using fstrim on the mount point of the disk image or mounting the disk image (loop) with discard doesn't appear to pass the freed blocks to btrfs.
Quoted 7 lines
The image file on the btrfs filesystem just keeps growing in size. When hosting images on ext4, using fstrim or discard frees up the space on the host filesystem (ext4).
I see discard is pretty well supported for btrfs to the underlying block devices but is there any way to properly free space on the btrfs host filesystem from overlying filesystems while they are online?
I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?
Signature · 2 lines
With respect, Roman
Quoted 17 lines · Roman Mamedov
Quoted 6 lines · On Mon, 09 Feb 2015 00:17:49 -0500 "Devon B." <[email protected]>…
Looking to use btrfs with disk images that contain ext4, xfs, and other possible filesystems. One of my main concerns is reclaiming disk space when files are deleted on the disk image's filesystem. Using fstrim on the mount point of the disk image or mounting the disk image (loop) with discard doesn't appear to pass the freed blocks to btrfs.
Quoted 7 lines
The image file on the btrfs filesystem just keeps growing in size. When hosting images on ext4, using fstrim or discard frees up the space on the host filesystem (ext4).
I see discard is pretty well supported for btrfs to the underlying block devices but is there any way to properly free space on the btrfs host filesystem from overlying filesystems while they are online?
Quoted 5 lines
I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?
This is interesting, because I've tried exactly that setup (KVM,
discard=unmap) and it's not worked for me.
Hugo.
Signature · 4 lines
Hugo Mills | If you see something, say nothing and drink to hugo@... carfax.org.uk | forget http://carfax.org.uk/ | PGP: 65E74AC0 | Welcome to Night Vale
Quoted 8 lines · On Mon, 9 Feb 2015 08:39:41 +0000 Hugo Mills <[email protected]> wrote:
Quoted 5 lines
I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?
This is interesting, because I've tried exactly that setup (KVM,
discard=unmap) and it's not worked for me.
Did you use the emulated IDE interface in KVM? Others such as virtio, ahci and maybe SCSI may not support discard. Did you get a successful result issuing TRIM within the guest (e.g. fstrim -v / succeeds and doesn't -ENOSUPP)?
Signature · 2 lines
With respect, Roman
Quoted 9 lines · Roman Mamedov
Quoted 6 lines · On Mon, 9 Feb 2015 08:39:41 +0000 Hugo Mills <[email protected]> wrote:
Quoted 5 lines
I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?
Quoted 6 lines
Quoted 2 lines
This is interesting, because I've tried exactly that setup (KVM,
discard=unmap) and it's not worked for me.Did you use the emulated IDE interface in KVM? Others such as virtio, ahci and maybe SCSI may not support discard. Did you get a successful result issuing TRIM within the guest (e.g. fstrim -v / succeeds and doesn't
-ENOSUPP)?
I used SCSI, so maybe that's the problem. I'll try with IDE.
Hugo.
Signature · 4 lines
Hugo Mills | I was cursed with poetry very young. It creates hugo@... carfax.org.uk | unrealistic expectations. http://carfax.org.uk/ | Victor Frankenstein PGP: 65E74AC0 | Penny Dreadful
Quoted 2 lines · On Mon, 09 Feb 2015 10:26:33 -0500 "Devon B." <[email protected]>…
If you don't mind me asking, what version kernel are you running and are you using any special mount options?
Well actually I did not claim I have working discard through 'loop', but your post made me curious.
$ sudo dd if=/dev/zero of=100g bs=1M seek=100000 count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00221052 s, 474 MB/s
$ sudo mkfs.ext4 100g [...]
$ du -hsc 100g 133M 100g 133M total
$ sudo mount -o loop 100g /mnt/tmp1/
(then in a new terminal window):
$ cd /mnt/tmp1/ $ df -h . Filesystem Size Used Avail Use% Mounted on /dev/loop0 96G 60M 92G 1% /mnt/tmp1 $ sudo dd if=/dev/zero of=zerofile bs=1M count=1024 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 0.944377 s, 1.1 GB/s $ sync
(back to the original one):
$ du -hsc 100g 1.2G 100g 1.2G total
(2nd window):
$ sudo fstrim .
(back to the original one):
$ du -hsc 100g 133M 100g 133M total
So it does work for me just fine even with 'loop'. Kernel version 3.14.32, mount options rw,noatime,nodiratime,compress=zlib,space_cache,inode_cache.
Signature · 2 lines
With respect, Roman
Quoted 34 lines · On Mon, 9 Feb 2015 20:42:56 +0500 Roman Mamedov <[email protected]> wrote:
Quoted 2 lines · On Mon, 09 Feb 2015 10:26:33 -0500 "Devon B." <[email protected]>…
If you don't mind me asking, what version kernel are you running and are you using any special mount options?
Well actually I did not claim I have working discard through 'loop', but your post made me curious.
$ sudo dd if=/dev/zero of=100g bs=1M seek=100000 count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00221052 s, 474 MB/s$ sudo mkfs.ext4 100g [...]$ du -hsc 100g 133M 100g 133M total$ sudo mount -o loop 100g /mnt/tmp1/(then in a new terminal window):
$ cd /mnt/tmp1/ $ df -h . Filesystem Size Used Avail Use% Mounted on /dev/loop0 96G 60M 92G 1% /mnt/tmp1 $ sudo dd if=/dev/zero of=zerofile bs=1M count=1024 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 0.944377 s, 1.1 GB/s $ sync
Quoted 6 lines
(back to the original one):
$ du -hsc 100g 1.2G 100g 1.2G total(2nd window):
Forgot to add I also did 'rm zerofile' here, of course.
Quoted 11 lines
$ sudo fstrim .(back to the original one):
$ du -hsc 100g 133M 100g 133M totalSo it does work for me just fine even with 'loop'. Kernel version 3.14.32, mount options rw,noatime,nodiratime,compress=zlib,space_cache,inode_cache.
Signature · 2 lines
With respect, Roman
Quoted 7 lines · On Mon, 09 Feb 2015 12:07:18 -0500 "Devon B." <[email protected]>…
Thanks for your testing. I haven't tried 3.14. I tried on CentOS 6 box (2.6.32 - which is experimental) and Ubuntu 14.04 (3.13) and neither worked. So the question remains, what is the difference? Possibly a small difference between the 3.13 and 3.14 kernels, I don't think it is any of the mount options. I guess if anyone else has insight on this, that would be great. Otherwise, I'll see if I can get a newer kernel loaded up and do some more testing.
Use "Reply to all" and not just "Reply", or "anyone else" won't see your question.
BTW it is not a good idea to use Btrfs on 2.6.32.
Signature · 2 lines
With respect, Roman
Thanks for your testing. I haven't tried 3.14. I tried on CentOS 6 box (2.6.32 - which is experimental) and Ubuntu 14.04 (3.13) and neither worked. So the question remains, what is the difference? Possibly a small difference between the 3.13 and 3.14 kernels, I don't think it is any of the mount options. I guess if anyone else has insight on this, that would be great. Otherwise, I'll see if I can get a newer kernel loaded up and do some more testing.
Quoted 197 lines
Roman Mamedov <mailto:[email protected]>
Monday, February 9, 2015 10:45 AMQuoted 33 lines · On Mon, 9 Feb 2015 20:42:56 +0500 Roman Mamedov<[email protected]> wrote:
Quoted 2 lines · On Mon, 09 Feb 2015 10:26:33 -0500 "Devon B."<[email protected]>…
If you don't mind me asking, what version kernel are you running and are you using any special mount options?
Well actually I did not claim I have working discard through 'loop', but your post made me curious.
$ sudo dd if=/dev/zero of=100g bs=1M seek=100000 count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00221052 s, 474 MB/s$ sudo mkfs.ext4 100g [...]$ du -hsc 100g 133M 100g 133M total$ sudo mount -o loop 100g /mnt/tmp1/(then in a new terminal window):
$ cd /mnt/tmp1/ $ df -h . Filesystem Size Used Avail Use% Mounted on /dev/loop0 96G 60M 92G 1% /mnt/tmp1 $ sudo dd if=/dev/zero of=zerofile bs=1M count=1024 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 0.944377 s, 1.1 GB/s $ syncQuoted 6 lines
(back to the original one):
$ du -hsc 100g 1.2G 100g 1.2G total(2nd window):
Forgot to add I also did 'rm zerofile' here, of course.
Quoted 11 lines
$ sudo fstrim .(back to the original one):
$ du -hsc 100g 133M 100g 133M totalSo it does work for me just fine even with 'loop'. Kernel version 3.14.32, mount options rw,noatime,nodiratime,compress=zlib,space_cache,inode_cache.
Roman Mamedov <mailto:[email protected]>
Monday, February 9, 2015 10:42 AM
On Mon, 09 Feb 2015 10:26:33 -0500Well actually I did not claim I have working discard through 'loop', but your post made me curious.
$ sudo dd if=/dev/zero of=100g bs=1M seek=100000 count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00221052 s, 474 MB/s$ sudo mkfs.ext4 100g [...]$ du -hsc 100g 133M 100g 133M total$ sudo mount -o loop 100g /mnt/tmp1/(then in a new terminal window):
$ cd /mnt/tmp1/ $ df -h . Filesystem Size Used Avail Use% Mounted on /dev/loop0 96G 60M 92G 1% /mnt/tmp1 $ sudo dd if=/dev/zero of=zerofile bs=1M count=1024 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 0.944377 s, 1.1 GB/s $ sync(back to the original one):
$ du -hsc 100g 1.2G 100g 1.2G total(2nd window):
$ sudo fstrim .(back to the original one):
$ du -hsc 100g 133M 100g 133M totalSo it does work for me just fine even with 'loop'. Kernel version 3.14.32, mount options rw,noatime,nodiratime,compress=zlib,space_cache,inode_cache.
Devon B. <mailto:[email protected]>
Monday, February 9, 2015 10:26 AM
If you don't mind me asking, what version kernel are you running and are you using any special mount options?Here is a quick example:
# qemu-img create -f raw /btrfs/sub/raw.img 100G Formatting '/btrfs/sub/raw.img', fmt=raw size=107374182400# mkfs.ext4 /btrfs/sub/raw.img ...# mount -o loop /btrfs/sub/raw.img /mnt/test# du -hs /btrfs/sub/raw.img 1.7G /btrfs/sub/raw.img# fstrim -v /mnt/test /mnt/test: 105492688896 bytes were trimmed# du -hs /btrfs/sub/raw.img 1.7G /btrfs/sub/raw.img# dd if=/dev/zero of=/mnt/test/1GB count=1k bs=1M 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 0.493057 s, 2.2 GB/s# du -hs /btrfs/sub/raw.img 2.7G /btrfs/sub/raw.img# rm -f /mnt/test/1GB# fstrim -v /mnt/test /mnt/test: 1186967552 bytes were trimmed# du -hs /btrfs/sub/raw.img 2.7G /btrfs/sub/raw.img# dd if=/dev/zero of=/mnt/test/1GB count=1k bs=1M 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 0.467089 s, 2.3 GB/s# du -hs /btrfs/sub/raw.img 3.7G /btrfs/sub/raw.img# rm -f /mnt/test/1GB# fstrim -v /mnt/test /mnt/test: 1203761152 bytes were trimmed# du -hs /btrfs/sub/raw.img 3.7G /btrfs/sub/raw.img# du -hs /mnt/test 20K /mnt/testSo even though there is nothing in
/mnt/test, the disk image is consuming 3.7GB of space. Maybe you could test similarly with your server if you have time on your hands.Thanks!
Roman Mamedov <mailto:[email protected]>
Monday, February 9, 2015 1:40 AM
On Mon, 09 Feb 2015 00:17:49 -0500I use KVM (QEMU) with discard pass-through from the VM guest ("discard=unmap" option), with the VM images stored on Btrfs. It works just fine, the disk space used for the image file does shrink when the guest OS issues discards on its FS. Maybe there is a difference in how KVM and the 'loop' module submit discards to Btrfs?
Devon B. <mailto:[email protected]>
Monday, February 9, 2015 12:17 AM
Looking to use btrfs with disk images that contain ext4, xfs, and other possible filesystems. One of my main concerns is reclaiming disk space when files are deleted on the disk image's filesystem. Using fstrim on the mount point of the disk image or mounting the disk image (loop) with discard doesn't appear to pass the freed blocks to btrfs.The image file on the btrfs filesystem just keeps growing in size. When hosting images on ext4, using fstrim or discard frees up the space on the host filesystem (ext4).
I see discard is pretty well supported for btrfs to the underlying block devices but is there any way to properly free space on the btrfs host filesystem from overlying filesystems while they are online?
Thanks.