generic block based fiemap implementation
Subsystems (3)
PATCH 5/5 generic block based fiemap implementation
From: Josef Bacik <[email protected]>
Here is my updated generic block based fiemap implementation with the fixes that Andreas suggested. Basically any block based fs (this patch includes ext3) just has to declare its own fiemap() function and then call this generic function with its own get_block_t. This works well for block based filesystems that will map multiple contiguous blocks at one time, but will work for filesystems that only map one block at a time, you will just end up with an "extent" for each block. One gotcha is this will not play nicely where there is hole+data after the EOF. This function will assume its hit the end of the data as soon as it hits a hole after the EOF, so if there is any data past that it will not pick that up. AFAIK no block based fs does this anyway, but its in the comments of the function anyway just in case. Thanks much,
--- fs/ext3/file.c | 1 + fs/ext3/inode.c | 8 +++ fs/ioctl.c | 128 +++++++++++++++++++++++++++++++++++++++++++++++ include/linux/ext3_fs.h | 2 + include/linux/fs.h | 3 + 5 files changed, 142 insertions(+), 0 deletions(-)
fs/ext3/file.c +1 -0
@@ -134,5 +134,6 @@ const struct inode_operations ext3_file_inode_operations = { .removexattr = generic_removexattr, #endif .permission = ext3_permission, + .fiemap = ext3_fiemap, };
fs/ext3/inode.c +8 -0
@@ -36,6 +36,7 @@#include <linux/mpage.h> #include <linux/uio.h> #include <linux/bio.h> +#include <linux/fiemap.h> #include "xattr.h" #include "acl.h"
@@ -981,6 +982,13 @@ out:return ret; } +int ext3_fiemap (struct inode *inode,struct fiemap_extent_info *fieinfo, +u64 start,u64 len) +{ +return generic_block_fiemap (inode, fieinfo, start, len, + ext3_get_block); +} +/* * `handle' can be NULL if create is zero */
fs/ioctl.c +128 -0
140 lines not on this page · raw message
include/linux/ext3_fs.h +2 -0
@@ -836,6 +836,8 @@ extern void ext3_truncate (struct inode *);extern void ext3_set_inode_flags (struct inode *);extern void ext3_get_inode_flags (struct ext3_inode_info *);extern void ext3_set_aops (struct inode *inode); +extern int ext3_fiemap (struct inode *inode,struct fiemap_extent_info *fieinfo, +u64 start,u64 len);/* ioctl.c */ extern int ext3_ioctl (struct inode *, struct file *, unsigned int,
include/linux/fs.h +3 -0
@@ -1976,6 +1976,9 @@ extern int vfs_fstat(unsigned int, struct kstat *);extern int do_vfs_ioctl (struct file *filp,unsigned int fd,unsigned int cmd,unsigned long arg); +extern int generic_block_fiemap (struct inode *inode, +struct fiemap_extent_info *fieinfo,u64 start, +u64 len,get_block_t *get_block);extern void get_filesystem (struct file_system_type *fs);extern void put_filesystem (struct file_system_type *fs);
Signature · 1 line
1.5.4.1
fs/ext3/inode.c out: · line 990
Quoted 6 lines · Mark Fasheh
fs/ext3/inode.c +6
-0+int ext3_fiemap (struct inode *inode,struct fiemap_extent_info *fieinfo, +u64 start,u64 len) +{ +return generic_block_fiemap (inode, fieinfo, start, len, + ext3_get_block); +}
We can wire up ext2 in the same way, and likely most of the other filesystems.
#define GENERIC_FIEMAP_FLAG_SUPP (FIEMAP_FLAG_NUM_EXTENTS | FIEMAP_FLAG_SYNC |
FIEMAP_FLAG_HSM_READ | FIEMAP_FLAG_LUN_ORDER)
fs/ioctl.c out_bad_flags: · line 239
Quoted 5 lines
fs/ioctl.c +5
-0+int generic_block_fiemap (struct inode *inode, +struct fiemap_extent_info *fieinfo,u64 start, +u64 len,get_block_t *get_block) +{ +int just_count = fieinfo->fi_flags &FIEMAP_FLAG_NUM_EXTENTS ;
Don't need this anymore.
if (fieinfo->fi_flags & ~GENERIC_FIEMAP_FLAG_SUPP) {
fieinfo->fi_flags &= ~GENERIC_FIEMAP_FLAG_SUPP;
return -EBADF;
}fs/ioctl.c out_bad_flags: · line 277
Quoted 12 lines
fs/ioctl.c +12
-0+/* HOLE */ +if (!tmp.b_blocknr) { +if (length <=0 ) { +if (just_count) { + fieinfo->fi_extents_mapped++; +break ; + } + flags =FIEMAP_EXTENT_LAST ; + ret =fiemap_fill_next_extent (fieinfo, logical, + phys, size, + flags,0 ); +break ;
Can remove the just_count check, fiemap_fill_next_extent() will handle it.
Quoted 14 lines
fs/ioctl.c +14
-0+ }else { +if (length <=0 && size && !just_count) { + ret =fiemap_fill_next_extent (fieinfo, logical, + phys, size, + flags,0 ); +if (ret) +break ; + } + + logical =blk_to_logical (inode, start_blk); + phys =blk_to_logical (inode, tmp.b_blocknr); + size = tmp.b_size; + flags =0 ; +
fs/ioctl.c out_bad_flags: · line 310
Quoted 10 lines
fs/ioctl.c +10
-0+ length -= tmp.b_size; + start_blk +=logical_to_blk (inode, size); + +if (just_count) { + fieinfo->fi_extents_mapped++; +if (fieinfo->fi_extents_mapped == + fieinfo->fi_extents_max) +break ; +continue ; + }
Can also remove this just_count check, which is incorrect in any case. The fi_extents_max is only when storing extents in the array, it isn't meaningful when only counting extents.
More important is the extent merging I proposed for fiemap_fill_next_extent() for filesystems that only return single blocks, so we want to always call it from the generic_block_fiemap() code.
Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.
Quoted 13 lines · Mark Fasheh
From: Josef Bacik <[email protected]>
Here is my updated generic block based fiemap implementation with the fixes that Andreas suggested. Basically any block based fs (this patch includes ext3) just has to declare its own
fiemap()function and then call this generic function with its ownget_block_t.fs/ioctl.c +5
-0+int generic_block_fiemap (struct inode *inode, +struct fiemap_extent_info *fieinfo,u64 start, +u64 len,get_block_t *get_block) +{ +struct buffer_head tmp;
[ ... ]
Quoted 4 lines
fs/ioctl.c +4
-0+ do { + /* + * we set b_size to the total size we want so it will map as + * many contiguous blocks as possible at once
fs/ioctl.c out_bad_flags: · line 262
Quoted 10 lines
fs/ioctl.c +10
-0+*/ +memset (&tmp,0 ,sizeof (struct buffer_head )); + tmp.b_size = map_len; + + ret =get_block (inode, start_blk, &tmp,0 ); +if (ret) +break ; + +/* HOLE */ +if (!tmp.b_blocknr) {
This should be if (!buffer_mapped(&tmp)).
You can't be sure that filesystems returning a hole will use a b_blocknr of zero, and you can't be sure that a block number of zero is a hole.
-chris
-chris
Quoted 14 lines · Mark Fasheh
From: Josef Bacik <[email protected]>
Here is my updated generic block based fiemap implementation with the fixes that Andreas suggested. Basically any block based fs (this patch includes ext3) just has to declare its own
fiemap()function and then call this generic function with its ownget_block_t. This works well for block based filesystems that will map multiple contiguous blocks at one time, but will work for filesystems that only map one block at a time, you will just end up with an "extent" for each block. One gotcha is this will not play nicely where there is hole+data after the EOF. This function will assume its hit the end of the data as soon as it hits a hole after the EOF, so if there is any data past that it will not pick that up. AFAIK no block based fs does this anyway, but its in the comments of the function anyway just in case. Thanks much,
Here is my updated patch that should address Andreas's and Chris's concerns. I added ext2 support as well. Thank you,
fs/ext3/file.c +1 -0
@@ -134,5 +134,6 @@ const struct inode_operations ext3_file_ .removexattr = generic_removexattr, #endif .permission = ext3_permission, + .fiemap = ext3_fiemap, };
fs/ext3/inode.c +8 -0
@@ -36,6 +36,7 @@#include <linux/mpage.h> #include <linux/uio.h> #include <linux/bio.h> +#include <linux/fiemap.h> #include "xattr.h" #include "acl.h"
@@ -981,6 +982,13 @@ out:return ret; } +int ext3_fiemap (struct inode *inode,struct fiemap_extent_info *fieinfo, +u64 start,u64 len) +{ +return generic_block_fiemap (inode, fieinfo, start, len, + ext3_get_block); +} +/* * `handle' can be NULL if create is zero */
include/linux/ext3_fs.h +2 -0
@@ -836,6 +836,8 @@ extern void ext3_truncate (struct inodeextern void ext3_set_inode_flags (struct inode *);extern void ext3_get_inode_flags (struct ext3_inode_info *);extern void ext3_set_aops (struct inode *inode); +extern int ext3_fiemap (struct inode *inode,struct fiemap_extent_info *fieinfo, +u64 start,u64 len);/* ioctl.c */ extern int ext3_ioctl (struct inode *, struct file *, unsigned int,
fs/ioctl.c +114 -0
126 lines not on this page · raw message
include/linux/fs.h +3 -0
@@ -1976,6 +1976,9 @@ extern int vfs_fstat(unsigned int, strucextern int do_vfs_ioctl (struct file *filp,unsigned int fd,unsigned int cmd,unsigned long arg); +extern int generic_block_fiemap (struct inode *inode, +struct fiemap_extent_info *fieinfo,u64 start, +u64 len,get_block_t *get_block);extern void get_filesystem (struct file_system_type *fs);extern void put_filesystem (struct file_system_type *fs);
fs/ext2/ext2.h +2 -0
@@ -133,6 +133,8 @@ extern void ext2_truncate (struct inodeextern int ext2_setattr (struct dentry *,struct iattr *);extern void ext2_set_inode_flags (struct inode *inode);extern void ext2_get_inode_flags (struct ext2_inode_info *); +extern int ext2_fiemap (struct inode *inode,struct fiemap_extent_info *fieinfo, +u64 start,u64 len);int __ext2_write_begin (struct file *file,struct address_space *mapping,loff_t pos,unsigned len,unsigned flags,struct page **pagep,void **fsdata);
fs/ext2/file.c +1 -0
@@ -86,4 +86,5 @@ const struct inode_operations ext2_file_#endif .setattr = ext2_setattr, .permission = ext2_permission, + .fiemap = ext2_fiemap, };
fs/ext2/inode.c +8 -0
@@ -31,6 +31,7 @@#include <linux/writeback.h> #include <linux/buffer_head.h> #include <linux/mpage.h> +#include <linux/fiemap.h> #include "ext2.h" #include "acl.h" #include "xip.h"
@@ -704,6 +705,13 @@ int ext2_get_block(struct inode *inode, } +int ext2_fiemap (struct inode *inode,struct fiemap_extent_info *fieinfo, +u64 start,u64 len) +{ +return generic_block_fiemap (inode, fieinfo, start, len, + ext2_get_block); +} +static int ext2_writepage (struct page *page,struct writeback_control *wbc) {return block_write_full_page (page, ext2_get_block, wbc);
Quoted 13 lines · Josef Bacik
Signed-off-by Josef Bacik <[email protected]>+/*
+ * @inode - the inode to map
+ * @arg - the pointer to userspace where we copy everything to
+ * @get_block - the fs'sget_blockfunction
+ *
+ * This does FIEMAP for block based inodes. Basically it will just loop
+ * throughget_blockuntil we hit the number of extents we want to map, or we
+ * go past the end of the file and hit a hole.
+ *
+ * If it is possible to have holes and data past EOF then please don't use
+ * this function, it will do wrong things.
+ */
I think it would be clearer to write at the end "If it is possible to have data blocks beyond a hole past @inode->i_size, then please do not use this function, it will stop at the first unmapped block beyond i_size."
Quoted 20 lines
fs/ioctl.c +20
-0+int generic_block_fiemap (struct inode *inode, +struct fiemap_extent_info *fieinfo,u64 start, +u64 len,get_block_t *get_block) +{ +struct buffer_head tmp; +unsigned int start_blk; +long long length =0 , map_len =0 ; +u64 logical =0 , phys =0 , size =0 ; +u32 flags =0 ; +int ret =0 ; + + start_blk =logical_to_blk (inode, start); + +/* guard against change */ +mutex_lock (&inode->i_mutex); + + length = (long long )min_t (u64, len,i_size_read (inode)); + map_len = length; + + do {
fs/ioctl.c out_bad_flags: · line 258
Quoted 10 lines
fs/ioctl.c +10
-0+/* +* we set b_size to the total size we want so it will map as +* many contiguous blocks as possible at once +*/ +memset (&tmp,0 ,sizeof (struct buffer_head )); + tmp.b_size = map_len; + + ret =get_block (inode, start_blk, &tmp,0 ); +if (ret) +break ;
Shouldn't map_len be reduced as the file is traversed? Otherwise the filesystem may be doing a lot more work than it needs to for block mapping if the requested length is not the whole file. If, for example, the first 1GB of the file is requested (map_len = 2 << 30) and 1022MB are mapped, the last 1MB get_block() request will still request 1GB mapping.
It isn't a major problem (only a corner case really), and the map_len can't be exactly the same as "length", because it still needs to be positive after EOF so that blocks will continue to be mapped.
Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.
fs/ioctl.c out_bad_flags: · line 232
Quoted 4 lines · Josef Bacik
fs/ioctl.c +4
-0+int generic_block_fiemap (struct inode *inode, +struct fiemap_extent_info *fieinfo,u64 start, +u64 len,get_block_t *get_block) +{
This still doesn't do any validity checking on fieinfo->fi_flags. You can't assume that the check in ioctl_fiemap() is correct for this routine. As soon as some new flag is added (e.g. FIEMAP_FLAG_XATTR) this code would do the wrong thing.
Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.