Inline data with 128-byte inodes?
Is there a fundamental reason that ext4 *can't* or *shouldn't* support inline data with 128-byte inodes?
As far as I can tell, the kernel ext4 implementation only allows inline data with 256-byte or larger inodes, because it requires the system.data xattr to exist, even if the actual data requires 60 bytes or less. (The implementation in debugfs, on the other hand, handles inline data in 128-byte inodes just fine. And it seems like it'd be fairly straightforward to change the kernel implementation to support it as well.)
For filesystems that don't need to store xattrs in general, and can live with the other limitations of 128-byte inodes, using a 128-byte inode can save substantial space compared to a 256-byte inode (many megabytes worth of inode tables, versus 4k for each file between 61-160 bytes), and many small files or small directories would still fit in 60 bytes.
Jan Kara
reviewerQuoted 2 lines · Josh Triplett
Is there a fundamental reason that ext4 *can't* or *shouldn't* support inline data with 128-byte inodes?
Well, where would we put it on disk? ext4 on-disk inode fills 128-bytes with 'osd2' union...
Or do you mean we should put inline data in an external xattr block?
Honza
Quoted 13 lines
As far as I can tell, the kernel ext4 implementation only allows inline data with 256-byte or larger inodes, because it requires the system.data xattr to exist, even if the actual data requires 60 bytes or less. (The implementation in debugfs, on the other hand, handles inline data in 128-byte inodes just fine. And it seems like it'd be fairly straightforward to change the kernel implementation to support it as well.)
For filesystems that don't need to store xattrs in general, and can live with the other limitations of 128-byte inodes, using a 128-byte inode can save substantial space compared to a 256-byte inode (many megabytes worth of inode tables, versus 4k for each file between 61-160 bytes), and many small files or small directories would still fit in 60 bytes.
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 7 lines · Jan Kara
Quoted 2 lines · Josh Triplett
Is there a fundamental reason that ext4 *can't* or *shouldn't* support inline data with 128-byte inodes?
Well, where would we put it on disk? ext4 on-disk inode fills 128-bytes with 'osd2' union...
There are 60 bytes in the "i_block" field that can be used by inline_data.
Quoted 1 line
Or do you mean we should put inline data in an external xattr block?
Using an 4KB xattr block would IMHO be worse than just using a regular file block for such files, if they don't fit into the 60 i_block bytes. That makes the data handling more complex (data copies each time in/out of the xattr) and has performance impact (all writes essentially data journal because they go via the setxattr code path.
The only time it _might_ be useful is if there are other xattrs that are shared in the external block with inline_data. However, at that point I think you are better off to just create larger inodes to hold the xattrs to avoid the seeking needed to load the external block...
Given the prevalence of xattrs today (SELinux springs to mind), I'd be surprised whether this combination shows any improvement in real life, but I don't have an _objection_ to allowing this combination (e.g. for ultra-compact /etc or boot filesystem images.
Maybe there is a bigger win for small directories avoiding 4KB leaf blocks?
That said, I'd be happy to see some numbers to show this is a win, and I'm definitely not _against_ allowing this to work if there is a use for it.
Cheers, Andreas
Quoted 16 lines
Quoted 8 lines
As far as I can tell, the kernel ext4 implementation only allows inline data with 256-byte or larger inodes, because it requires the system.data xattr to exist, even if the actual data requires 60 bytes or less. (The implementation in debugfs, on the other hand, handles inline data in 128-byte inodes just fine. And it seems like it'd be fairly straightforward to change the kernel implementation to support it as well.)
Quoted 5 lines
For filesystems that don't need to store xattrs in general, and can live with the other limitations of 128-byte inodes, using a 128-byte inode can save substantial space compared to a 256-byte inode (many megabytes worth of inode tables, versus 4k for each file between 61-160 bytes), and many small files or small directories would still fit in 60 bytes.
--
Jan Kara <[email protected]>
SUSE Labs, CR
Quoted 9 lines · Andreas Dilger
Quoted 6 lines · Jan Kara
Quoted 2 lines · Josh Triplett
Is there a fundamental reason that ext4 *can't* or *shouldn't* support inline data with 128-byte inodes?
Well, where would we put it on disk? ext4 on-disk inode fills 128-bytes with 'osd2' union...
There are 60 bytes in the "
i_block" field that can be used byinline_data.
Exactly. But the Linux ext4 implementation doesn't accept inline data unless the system.data xattr exists, even if the file's data fits in 60 bytes (in which case system.data must exist and have 0 length).
Quoted 1 line
Quoted 1 line
Or do you mean we should put inline data in an external xattr block?
Definitely not, no. That seems much more complex to deal with.
I'm only talking about the case of files or directories <= 60 bytes fitting in the inode with 128-byte inodes. Effectively, this would mean accepting inodes with the inline data flag set, whether they have the system.data xattr or not.
Quoted 4 lines
Maybe there is a bigger win for small directories avoiding 4KB leaf blocks?
That said, I'd be happy to see some numbers to show this is a win, and I'm definitely not _against_ allowing this to work if there is a use for it.
Some statistics, for ext4 with 4k blocks and 128-byte inodes, if 60-byte inline data worked with 128-byte inodes:
A filesystem containing the source code of the Linux kernel would save about 1508 disk blocks, or around 6032k.
A filesystem containing only my /etc directory would save about 650 blocks, or 2600k, a substantial fraction of the entire directory (which takes up 9004k total without inline data).
- Josh Triplett
Jan Kara
reviewerQuoted 6 lines · Josh Triplett
Quoted 5 lines · Andreas Dilger
Quoted 4 lines · Jan Kara
Quoted 2 lines · Josh Triplett
Is there a fundamental reason that ext4 *can't* or *shouldn't* support inline data with 128-byte inodes?
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Well, where would we put it on disk? ext4 on-disk inode fills 128-bytes with 'osd2' union...
There are 60 bytes in the "
i_block" field that can be used byinline_data.Exactly. But the Linux ext4 implementation doesn't accept inline data unless the system.data xattr exists, even if the file's data fits in 60 bytes (in which case system.data must exist and have 0 length).
I see now I understand what you meant. Thanks for explanation.
Quoted 8 lines
Quoted 4 lines
Maybe there is a bigger win for small directories avoiding 4KB leaf blocks?
That said, I'd be happy to see some numbers to show this is a win, and I'm definitely not _against_ allowing this to work if there is a use for it.
Some statistics, for ext4 with 4k blocks and 128-byte inodes, if 60-byte inline data worked with 128-byte inodes:
Quoted 6 lines
A filesystem containing the source code of the Linux kernel would save about 1508 disk blocks, or around 6032k.
A filesystem containing only my /etc directory would save about 650 blocks, or 2600k, a substantial fraction of the entire directory (which takes up 9004k total without inline data).
I guess few people care about a few megabytes these days... For really space sensitive applications, people don't pick ext4 as a base filesystem I'd guess (I'd expect squashfs, erofs, or ubifs if you need write access). So the benefit is relatively small, the question about the cost is - how complicated it gets to support inline data without xattrs?
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 33 lines · Josh Triplett
Quoted 9 lines · Andreas Dilger
Quoted 6 lines · Jan Kara
Quoted 2 lines · Josh Triplett
Is there a fundamental reason that ext4 *can't* or *shouldn't* support inline data with 128-byte inodes?
Well, where would we put it on disk? ext4 on-disk inode fills 128-bytes with 'osd2' union...
There are 60 bytes in the "
i_block" field that can be used byinline_data.Exactly. But the Linux ext4 implementation doesn't accept inline data unless the system.data xattr exists, even if the file's data fits in 60 bytes (in which case system.data must exist and have 0 length).
Quoted 1 line
Quoted 1 line
Or do you mean we should put inline data in an external xattr block?
Definitely not, no. That seems much more complex to deal with.
I'm only talking about the case of files or directories <= 60 bytes fitting in the inode with 128-byte inodes. Effectively, this would mean accepting inodes with the inline data flag set, whether they have the system.data xattr or not.
Quoted 4 lines
Maybe there is a bigger win for small directories avoiding 4KB leaf blocks?
That said, I'd be happy to see some numbers to show this is a win, and I'm definitely not _against_ allowing this to work if there is a use for it.
Some statistics, for ext4 with 4k blocks and 128-byte inodes, if 60-byte inline data worked with 128-byte inodes:
Quoted 6 lines
A filesystem containing the source code of the Linux kernel would save about 1508 disk blocks, or around 6032k.
A filesystem containing only my /etc directory would save about 650 blocks, or 2600k, a substantial fraction of the entire directory (which takes up 9004k total without inline data).
Hi Josh,
I started looking into this, and got half-way through a patch before getting pulled into something else. If you are interested to finish it off, I've included it here, but it definitely isn't ready yet.
Looking at the details, it isn't just an issue of changing a single threshold to decide whether 128-byte inodes can handle inline data or not. The code is intermingled between "have system.data xattr" as the defining factor and using "inline_off" (which points to the byte offset of the system.data contents) to determine whether there is inline data or not.
It _should_ be possible to change to using EXT4_STATE_MAY_INLINE_DATA and EXT4_INLINE_DATA_FL to determine if there is any inline data in the inode, and use "inline_off" and i_extra_isize only to detect if any data is in the xattr or not. It doesn't need to check the xattrs if the inode size is EXT4_GOOD_OLD_INODE_SIZE, since we *don't* want "inline" data in an external xattr block for sanity/efficiency reasons.
Hopefully this patch is useful for you as a starting point - I suspect it is about 80% finished, but I wouldn't have time to work on it for a few weeks at least, since 128-byte inodes is not a useful case for me (we are using 1024-byte inodes on our metadata filesystems these days). On the flip side, I think inline_data is very interesting, and am happy for it to gain more widespread usage. I don't think the 128-byte inode support for inline_data increases complexity, so I wouldn't be against landing it.
Cheers, Andreas
--
fs/ext4/ext4.h +1 -2
@@ -3260,8 +3260,7 @@ extern int ext4_inline_data_fiemap(struct inode *inode,static inline int ext4_has_inline_data (struct inode *inode) { -return ext4_test_inode_flag (inode,EXT4_INODE_INLINE_DATA ) && -EXT4_I (inode)->i_inline_off; +return ext4_test_inode_flag (inode,EXT4_INODE_INLINE_DATA ); }/* namei.c */
fs/ext4/inline.c +36 -21
143 lines not on this page · raw message