when does btrfs create sparse extents?
Hello,
there was a bug fixed recently in U-Boot's btrfs driver - the driver failed to read files with sparse extents. This causes that sometimes device failes to boot Linux, since the kernel fails to load from storage.
So when does kernel's btrfs driver write sparse extents? Is it always when it finds a PAGE_SIZEd and aligned all-zeros block? Or is it more complicated?
Thanks.
Marek
Quoted 11 lines · Marek Behun
Hello,
there was a bug fixed recently in U-Boot's btrfs driver - the driver failed to read files with sparse extents. This causes that sometimes device failes to boot Linux, since the kernel fails to load from storage.
So when does kernel's btrfs driver write sparse extents? Is it always when it finds a
PAGE_SIZEdand aligned all-zeros block? Or is it more complicated?
I'm not a btrfs developer, so with respect to kernel code behavior I can't answer directly.
But I wonder if other sources of this sparseness has been considered? Maybe the build system is creating or preserving sparseness? e.g. tar --hole-detection or --sparse is used.
Another possibility is Btrfs supports two kinds of holes in the on-disk format for sparse files. Maybe uboot only supported the original (current default) type, and the bug really fixed the newer 'no-holes' feature version?
Signature · 1 line
Chris Murphy
Quoted 3 lines · On Wed, 22 Apr 2020 14:26:31 -0600 Chris Murphy <[email protected]> wrote:
But I wonder if other sources of this sparseness has been considered? Maybe the build system is creating or preserving sparseness? e.g.
tar --hole-detectionor--sparseis used.
I don't actually know how the kernel is copied into the /boot directory in this case.
Quoted 4 lines
Another possibility is Btrfs supports two kinds of holes in the on-disk format for sparse files. Maybe uboot only supported the original (current default) type, and the bug really fixed the newer 'no-holes' feature version?
No, U-Boot did not support holes at all.
e.g. from a 10m file created with truncate on two Btrfs file systems
original holes format (default)
item 6 key (257 EXTENT_DATA 0) itemoff 15768 itemsize 53
generation 7412 type 1 (regular)
extent data disk byte 0 nr 0
extent data offset 0 nr 10485760 ram 10485760 extent compression 0 (none)
On a file system with no-holes feature set, this item simply doesn't exist. I think basically it works by inference. Both kinds of files have size in the INODE_ITEM, e.g.
item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160
generation 889509 transid 889509 size 10485760 nbytes 0Sparse extents are explicitly stated in the original format with disk byte 0 in an EXTENT_DATA item; whereas in the newer format, sparse extents exist whenever EXTENT_DATA items don't completely describe the file's size.
--
Chris Murphy
Quoted 14 lines · On Wed, 22 Apr 2020 14:44:46 -0600 Chris Murphy <[email protected]> wrote:
e.g. from a 10m file created with truncate on two Btrfs file systems
original holes format (default)
item 6 key (257
EXTENT_DATA0) itemoff 15768 itemsize 53
generation 7412 type 1 (regular)
extent data disk byte 0 nr 0
extent data offset 0 nr 10485760 ram 10485760 extent compression 0 (none)On a file system with no-holes feature set, this item simply doesn't exist. I think basically it works by inference. Both kinds of files have size in the
INODE_ITEM, e.g.
Quoted 7 lines
item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160 generation 889509 transid 889509 size 10485760 nbytes 0Sparse extents are explicitly stated in the original format with disk byte 0 in an
EXTENT_DATAitem; whereas in the newer format, sparse extents exist wheneverEXTENT_DATAitems don't completely describe the file's size.
Ok this means that U-Boot currently gained support for the original sparse extents.
I fear that current u-boot does not handle the new no-holes feature.
Quoted 26 lines · Marek Behun
Quoted 14 lines · On Wed, 22 Apr 2020 14:44:46 -0600 Chris Murphy <[email protected]> wrote:
e.g. from a 10m file created with truncate on two Btrfs file systems
original holes format (default)
item 6 key (257
EXTENT_DATA0) itemoff 15768 itemsize 53
generation 7412 type 1 (regular)
extent data disk byte 0 nr 0
extent data offset 0 nr 10485760 ram 10485760 extent compression 0 (none)On a file system with no-holes feature set, this item simply doesn't exist. I think basically it works by inference. Both kinds of files have size in the
INODE_ITEM, e.g.Quoted 7 lines
item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160 generation 889509 transid 889509 size 10485760 nbytes 0Sparse extents are explicitly stated in the original format with disk byte 0 in an
EXTENT_DATAitem; whereas in the newer format, sparse extents exist wheneverEXTENT_DATAitems don't completely describe the file's size.
Quoted 4 lines
Ok this means that U-Boot currently gained support for the original sparse extents.
I fear that current u-boot does not handle the new no-holes feature.
I'd advise it should support it, I think it's expected to become the default. But I don't know the time frame for that.
Also, when the no-holes feature flag is not set, you can be certain only the original hole type is used. But when no-holes feature is set, it's possible for both types of holes to exist, because no-holes can be set after mkfs time by btrfstune.
Signature · 1 line
Chris Murphy
Quoted 21 lines · Marek Behun
Quoted 17 lines · On Wed, 22 Apr 2020 14:44:46 -0600 Chris Murphy <[email protected]> wrote:
e.g. from a 10m file created with truncate on two Btrfs file systems
original holes format (default)
item 6 key (257
EXTENT_DATA0) itemoff 15768 itemsize 53
generation 7412 type 1 (regular)
extent data disk byte 0 nr 0
extent data offset 0 nr 10485760 ram 10485760 extent compression 0 (none)On a file system with no-holes feature set, this item simply doesn't exist. I think basically it works by inference. Both kinds of files have size in the
INODE_ITEM, e.g.item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160 generation 889509 transid 889509 size 10485760 nbytes 0
Quoted 7 lines
Quoted 4 lines
Sparse extents are explicitly stated in the original format with disk byte 0 in an
EXTENT_DATAitem; whereas in the newer format, sparse extents exist wheneverEXTENT_DATAitems don't completely describe the file's size.Ok this means that U-Boot currently gained support for the original sparse extents.
To clear any confusion, what you mean by sparse extents is actually holes. The concept of sparse files exists (files with holes, regions of a file for which there is no allocated extent), but not sparse extents.
Quoted 2 lines
I fear that current u-boot does not handle the new no-holes feature.
The no-holes feature has been around since 2013, not exactly new, but it's not the default yet when creating a new filesystem.
As it has been mentioned earlier by Chris, it just removes the need for explicitly having metadata representing holes.
When not using the no-holes feature, there is an explicit file extent item pointing to a disk location of 0 (disk_bytenr field has a value of 0) for each file hole.
When using no-holes, there's no such file extent item - btrfs knows about the hole by checking that there is a gap between two consecutive file extent items (both having a disk_bytenr > 0).
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
Quoted 40 lines · On Thu, 23 Apr 2020 11:49:16 +0100 Filipe Manana <[email protected]> wrote:
Quoted 21 lines · Marek Behun
Quoted 17 lines · On Wed, 22 Apr 2020 14:44:46 -0600 Chris Murphy <[email protected]> wrote:
e.g. from a 10m file created with truncate on two Btrfs file systems
original holes format (default)
item 6 key (257
EXTENT_DATA0) itemoff 15768 itemsize 53
generation 7412 type 1 (regular)
extent data disk byte 0 nr 0
extent data offset 0 nr 10485760 ram 10485760 extent compression 0 (none)On a file system with no-holes feature set, this item simply doesn't exist. I think basically it works by inference. Both kinds of files have size in the
INODE_ITEM, e.g.item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160 generation 889509 transid 889509 size 10485760 nbytes 0Quoted 7 lines
Quoted 4 lines
Sparse extents are explicitly stated in the original format with disk byte 0 in an
EXTENT_DATAitem; whereas in the newer format, sparse extents exist wheneverEXTENT_DATAitems don't completely describe the file's size.Ok this means that U-Boot currently gained support for the original sparse extents.
To clear any confusion, what you mean by sparse extents is actually holes. The concept of sparse files exists (files with holes, regions of a file for which there is no allocated extent), but not sparse extents.
Quoted 2 lines
I fear that current u-boot does not handle the new no-holes feature.
The no-holes feature has been around since 2013, not exactly new, but it's not the default yet when creating a new filesystem.
Quoted 8 lines
As it has been mentioned earlier by Chris, it just removes the need for explicitly having metadata representing holes.
When not using the no-holes feature, there is an explicit file extent item pointing to a disk location of 0 (disk_bytenrfield has a value of 0) for each file hole.
When using no-holes, there's no such file extent item - btrfs knows about the hole by checking that there is a gap between two consecutive file extent items (both having adisk_bytenr> 0).
This I already understand. My main question though is: does kernel or btrfs do checking (at least sometimes) when writing a block of data onto disk if this block is all zero, and if yes, then this block is written as a hole (either by writing hole item or not writing anything)?
Or does this happen ONLY when requested by userspace?
Because for the love of god I cannot find why our kernel is being written this way onto disk - the installer doesn't explicitly request for PUNCH_HOLES nor anything, as far as I looked.
Marek
Marek
Quoted 53 lines · Marek Behun
Quoted 40 lines · On Thu, 23 Apr 2020 11:49:16 +0100 Filipe Manana <[email protected]> wrote:
Quoted 21 lines · Marek Behun
Quoted 17 lines · On Wed, 22 Apr 2020 14:44:46 -0600 Chris Murphy <[email protected]> wrote:
e.g. from a 10m file created with truncate on two Btrfs file systems
original holes format (default)
item 6 key (257
EXTENT_DATA0) itemoff 15768 itemsize 53
generation 7412 type 1 (regular)
extent data disk byte 0 nr 0
extent data offset 0 nr 10485760 ram 10485760 extent compression 0 (none)On a file system with no-holes feature set, this item simply doesn't exist. I think basically it works by inference. Both kinds of files have size in the
INODE_ITEM, e.g.item 4 key (257 INODE_ITEM 0) itemoff 32245 itemsize 160 generation 889509 transid 889509 size 10485760 nbytes 0Quoted 7 lines
Quoted 4 lines
Sparse extents are explicitly stated in the original format with disk byte 0 in an
EXTENT_DATAitem; whereas in the newer format, sparse extents exist wheneverEXTENT_DATAitems don't completely describe the file's size.Ok this means that U-Boot currently gained support for the original sparse extents.
To clear any confusion, what you mean by sparse extents is actually holes. The concept of sparse files exists (files with holes, regions of a file for which there is no allocated extent), but not sparse extents.
Quoted 2 lines
I fear that current u-boot does not handle the new no-holes feature.
The no-holes feature has been around since 2013, not exactly new, but it's not the default yet when creating a new filesystem.
Quoted 8 lines
As it has been mentioned earlier by Chris, it just removes the need for explicitly having metadata representing holes.
When not using the no-holes feature, there is an explicit file extent item pointing to a disk location of 0 (disk_bytenrfield has a value of 0) for each file hole.
When using no-holes, there's no such file extent item - btrfs knows about the hole by checking that there is a gap between two consecutive file extent items (both having adisk_bytenr> 0).
Quoted 6 lines
This I already understand. My main question though is: does kernel or btrfs do checking (at least sometimes) when writing a block of data onto disk if this block is all zero, and if yes, then this block is written as a hole (either by writing hole item or not writing anything)?
Or does this happen ONLY when requested by userspace?
There's nothing in btrfs that converts a sequence of zeroes automatically to a hole.
It always has to be done by user space, either by writes that leave holes intentionally (e.g. create file, write 64K to offset 0, write 4K to offset 128, leaves a hole from range 64K to 128K) or by hole punching through fallocate().
Quoted 8 lines
Because for the love of god I cannot find why our kernel is being written this way onto disk - the installer doesn't explicitly request for
PUNCH_HOLESnor anything, as far as I looked.Marek
Marek
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
Quoted 7 lines · On Thu, 23 Apr 2020 12:51:32 +0100 Filipe Manana <[email protected]> wrote:
There's nothing in btrfs that converts a sequence of zeroes automatically to a hole.
It always has to be done by user space, either by writes that leave holes intentionally (e.g. create file, write 64K to offset 0, write 4K to offset 128, leaves a hole from range 64K to 128K) or by hole punching through
fallocate().
Thanks for this information. If you ever come to Prague, let me know, we can have a beer :D
Quoted 7 lines · Marek Behun
Quoted 3 lines · On Thu, 23 Apr 2020 12:51:32 +0100 Filipe Manana <[email protected]> wrote:
There's nothing in btrfs that converts a sequence of zeroes automatically to a hole.
Quoted 7 lines
Quoted 4 lines
It always has to be done by user space, either by writes that leave holes intentionally (e.g. create file, write 64K to offset 0, write 4K to offset 128, leaves a hole from range 64K to 128K) or by hole punching through
fallocate().Thanks for this information. If you ever come to Prague, let me know, we can have a beer :D
Noted! (Through it will be a long while until travel is allowed and safe)
Another case I forgot is a hole created by truncation - truncate a file to a larger size, then you get a hole for the range that goes from the old file size to the new file size.
Signature · 3 lines
Filipe David Manana, “Whether you think you can, or you think you can't — you're right.”
Quoted 14 lines · Filipe Manana
Quoted 12 lines · Marek Behun
Quoted 7 lines · On Thu, 23 Apr 2020 12:51:32 +0100 Filipe Manana <[email protected]> wrote:
There's nothing in btrfs that converts a sequence of zeroes automatically to a hole.
It always has to be done by user space, either by writes that leave holes intentionally (e.g. create file, write 64K to offset 0, write 4K to offset 128, leaves a hole from range 64K to 128K) or by hole punching through
fallocate().
Quoted 8 lines
Quoted 2 lines
Thanks for this information. If you ever come to Prague, let me know, we can have a beer :D
Noted! (Through it will be a long while until travel is allowed and safe)
Another case I forgot is a hole created by truncation - truncate a file to a larger size, then you get a hole for the range that goes from the old file size to the new file size.
I recently discovered the benefit of using both truncate and partially fallocating a VM backing file.
I start with truncate so I can set a typical device size for the guest's view, which I don't have the local space if I were to fallocate. e.g. 100G local, but do truncate -s 1T. Then follow that with fallocate -l 12G so that I don't end up with a massively fragmented backing file.
To be clear, this backing file is set with chattr +C and never snapshot or reflinked. Btrfs initially doesn't seem to perform any different with sparse or fallocated backing files. But with aging, the sparse file can get really fragmented depending on the guest file system. Tens of thousands of extents are possible, and that does impact performance it seems.
Signature · 1 line
Chris Murphy
22.04.2020 21:52, Marek Behun пишет:
Quoted 7 lines
Hello,
there was a bug fixed recently in U-Boot's btrfs driver - the driver failed to read files with sparse extents. This causes that sometimes device failes to boot Linux, since the kernel fails to load from storage.
Do you mean that kernel image (vmlinuz?) had holes? Or there were some other files that caused U-Boot to fail?
Quoted 8 lines
So when does kernel's btrfs driver write sparse extents? Is it always when it finds a
PAGE_SIZEdand aligned all-zeros block? Or is it more complicated?Thanks.
Marek
Quoted 11 lines · On Thu, 23 Apr 2020 08:57:59 +0300 Andrei Borzenkov <[email protected]> wrote:
22.04.2020 21:52, Marek Behun пишет:
Quoted 7 lines
Hello,
there was a bug fixed recently in U-Boot's btrfs driver - the driver failed to read files with sparse extents. This causes that sometimes device failes to boot Linux, since the kernel fails to load from storage.
Do you mean that kernel image (vmlinuz?) had holes? Or there were some other files that caused U-Boot to fail?
Kernel image (non-compressed Image for arm64) had ~200 PAGE_SIZEd all-zero blocks.