fiemap: introduce DATA_COMPRESSED and PHYS_LENGTH flags

PATCH In review David Sterba 3 participants 12 messages Jul 30, 2014 1 message not in the archive

Subsystems (9)

Documentation VFS Btrfs file system XFS filesystem +5 more subsystems

lore

Share & tools

lore

David Sterba

author

2014-07-30 17:18 UTC #0 lore raw

v5 cover fiemap: introduce DATA_COMPRESSED and PHYS_LENGTH flags

The original FIEMAP patch did not define the bits, btrfs would like to use a flag for compressed extents. The PHYS_LENGTH flag emerged during patchset revisions to keep backward compatibility and flexible fiemap API.

Currently, the 'filefrag' utility has no way to recognize and denote a compressed extent. As implemented in btrfs right now, the compression step splits a big extent into smaller chunks and this is reported as a heavily fragmented file. Adding the flag to filefrag will at least give some explanation why, this has been confusing users for some time already.

fiemap_fill_next_extent is extended and takes argument to fill the physical length.

V5:
Physical length is by default undefined. Btrfs use of compressed flag was enhanced to reflect the fiemap changes. Patches reordered so the generic fiemap go first.

V4:
The physical length is always set and equal to logical, or different and then sets the COMPRESSED flag.
fiemap_extent::fe_length renamed to fe_logi_length

V3:
Based on feedback from Andreas, implement #1 from V2, current users of fiemap_fill_next_extent (fs/, ext4, gfs2, ocfs2, nilfs2, xfs) updated accordingly, no functional change.

V2:
Based on feedback from Andreas, the fiemap_extent is now able to hold the physical extent length, to be filled by the filesystem callback.

1) extend fiemap_fill_next_extent to take phys_length and update all users (ext4, gfs2, ocfs2, nilfs2, xfs)

David Sterba (6):
fiemap: fix comment at EXTENT_DATA_ENCRYPTED
fiemap: add fe_phys_length and EXTENT_PHYS_LENGTH flag
fiemap: add FIEMAP_EXTENT_DATA_COMPRESSED flag Documentation/fiemap: Document DATA_COMPRESSED and PHYS_LENGTH flags
fiemap: rename fe_length to fe_logi_length
btrfs: set FIEMAP_EXTENT_DATA_COMPRESSED for compressed extents

 Documentation/filesystems/fiemap.txt | 24 ++++++++++++++++++++----
 fs/btrfs/extent_io.c                 | 11 ++++++++---
 fs/ext4/extents.c                    |  4 ++--
 fs/ext4/inline.c                     |  2 +-
 fs/gfs2/inode.c                      |  2 +-
 fs/ioctl.c                           | 21 ++++++++++++++-------
 fs/nilfs2/inode.c                    |  8 +++++---
 fs/ocfs2/extent_map.c                |  4 ++--
 fs/xfs/xfs_iops.c                    |  2 +-
 include/linux/fs.h                   |  2 +-
 include/uapi/linux/fiemap.h          | 16 +++++++++++++---
 11 files changed, 68 insertions(+), 28 deletions(-)

Signature · 1 line

1.8.4.5

David Sterba

author

2014-07-30 17:18 UTC #1 lore raw

PATCH v5 1/6 fiemap: fix comment at EXTENT_DATA_ENCRYPTED

The flag was named EXTENT_NO_BYPASS in the original fiemap proposal[1], but renamed to EXTENT_ENCODED afterwards.

[1] http://article.gmane.org/gmane.comp.file-systems.ext4/8871

Signed-off-by David Sterba <[email protected]>

---
 include/uapi/linux/fiemap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

include/uapi/linux/fiemap.h +1 -1

index 0c51d617dae9..93abfcd9ac47 100644
@@ -51,7 +51,7 @@ struct fiemap {
 #define FIEMAP_EXTENT_ENCODED		0x00000008 /* Data can not be read
 						    * while fs is unmounted */
 #define FIEMAP_EXTENT_DATA_ENCRYPTED	0x00000080 /* Data is encrypted by fs.
-						    * Sets EXTENT_NO_BYPASS. */
+						    * Sets EXTENT_ENCODED */
 #define FIEMAP_EXTENT_NOT_ALIGNED	0x00000100 /* Extent offsets may not be
 						    * block aligned. */
 #define FIEMAP_EXTENT_DATA_INLINE	0x00000200 /* Data mixed with metadata.

Signature · 1 line

1.8.4.5

Andreas Dilger

Reviewed-by on v5 1/6

2014-07-30 19:58 UTC #2 lore raw

Re: [PATCH 1/6] fiemap: fix comment at EXTENT_DATA_ENCRYPTED

Quoted 4 lines · David Sterba

The flag was named EXTENT_NO_BYPASS in the original fiemap proposal[1], but renamed to EXTENT_ENCODED afterwards.
[1] http://article.gmane.org/gmane.comp.file-systems.ext4/8871

This patch should be landed regardless of the rest of the series, since it is fixing a bug in the current comments.

Reviewed-by Andreas Dilger <[email protected]>

Quoted 21 lines

Signed-off-by David Sterba <[email protected]>

---
include/uapi/linux/fiemap.h | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

include/uapi/linux/fiemap.h +1 -1

index 0c51d617dae9..93abfcd9ac47 100644
@@ -51,7 +51,7 @@ struct fiemap {
 #define FIEMAP_EXTENT_ENCODED		0x00000008 /* Data can not be read
 						    * while fs is unmounted */
 #define FIEMAP_EXTENT_DATA_ENCRYPTED	0x00000080 /* Data is encrypted by fs.
-						    * Sets EXTENT_NO_BYPASS. */
+						    * Sets EXTENT_ENCODED */
 #define FIEMAP_EXTENT_NOT_ALIGNED	0x00000100 /* Extent offsets may not be
 						    * block aligned. */
 #define FIEMAP_EXTENT_DATA_INLINE	0x00000200 /* Data mixed with metadata.

Signature · 1 line

1.8.4.5

Cheers, Andreas

David Sterba

author

2014-07-30 17:18 UTC #3 lore raw

PATCH v5 2/6 fiemap: add fe_phys_length and EXTENT_PHYS_LENGTH flag

In reply to David Sterba

Add a new member to fiemap_extent that represents the physical extent length. This value is undefined if the flag EXTENT_PHYS_LENGTH is not set.

No functional change to existing fiemap users.

Signed-off-by David Sterba <[email protected]>

---
 fs/btrfs/extent_io.c        |  2 +-
 fs/ext4/extents.c           |  4 ++--
 fs/ext4/inline.c            |  2 +-
 fs/gfs2/inode.c             |  2 +-
 fs/ioctl.c                  | 13 ++++++++-----
 fs/nilfs2/inode.c           |  8 +++++---
 fs/ocfs2/extent_map.c       |  4 ++--
 fs/xfs/xfs_iops.c           |  2 +-
 include/linux/fs.h          |  2 +-
 include/uapi/linux/fiemap.h |  6 +++++-
 10 files changed, 27 insertions(+), 18 deletions(-)

fs/btrfs/extent_io.c +1 -1

index a389820d158b..c85da2e54ce7 100644
@@ -4357,7 +4357,7 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			end = 1;
 		}
 		ret = fiemap_fill_next_extent(fieinfo, em_start, disko,
-					      em_len, flags);
+					      em_len, 0, flags);
 		if (ret)
 			goto out_free;
 	}

fs/ext4/extents.c +2 -2

index 4da228a0e6d0..a194024e96a2 100644
@@ -2253,7 +2253,7 @@ static int ext4_fill_fiemap_extents(struct inode *inode,
 				(__u64)es.es_lblk << blksize_bits,
 				(__u64)es.es_pblk << blksize_bits,
 				(__u64)es.es_len << blksize_bits,
-				flags);
+				0, flags);
 			if (err < 0)
 				break;
 			if (err == 1) {

@@ -5125,7 +5125,7 @@ static int ext4_xattr_fiemap(struct inode *inode,
 
 	if (physical)
 		error = fiemap_fill_next_extent(fieinfo, 0, physical,
-						length, flags);
+						length, 0, flags);
 	return (error < 0 ? error : 0);
 }

fs/ext4/inline.c +1 -1

index 645205d8ada6..16f93d8100db 100644
@@ -1825,7 +1825,7 @@ int ext4_inline_data_fiemap(struct inode *inode,
 
 	if (physical)
 		error = fiemap_fill_next_extent(fieinfo, 0, physical,
-						length, flags);
+						length, 0, flags);
 	brelse(iloc.bh);
 out:
 	up_read(&EXT4_I(inode)->xattr_sem);

fs/gfs2/inode.c +1 -1

index e62e59477884..d0d35557ab5f 100644
@@ -1931,7 +1931,7 @@ static int gfs2_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			len = size - start;
 		if (start < size)
 			ret = fiemap_fill_next_extent(fieinfo, start, phys,
-						      len, flags);
+						      len, 0, flags);
 		if (ret == 1)
 			ret = 0;
 	} else {

fs/ioctl.c +8 -5

index 8ac3fad36192..127551c70b0a 100644
@@ -70,6 +70,7 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
  * @logical:	Extent logical start offset, in bytes
  * @phys:	Extent physical start offset, in bytes
  * @len:	Extent length, in bytes
+ * @phys_len:   Physical extent length in bytes
  * @flags:	FIEMAP_EXTENT flags that describe this extent
  *
  * Called from file system ->fiemap callback. Will populate extent

@@ -83,7 +84,7 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
 #define SET_NO_UNMOUNTED_IO_FLAGS	(FIEMAP_EXTENT_DATA_ENCRYPTED)
 #define SET_NOT_ALIGNED_FLAGS	(FIEMAP_EXTENT_DATA_TAIL|FIEMAP_EXTENT_DATA_INLINE)
 int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
-			    u64 phys, u64 len, u32 flags)
+			    u64 phys, u64 len, u64 phys_len, u32 flags)
 {
 	struct fiemap_extent extent;
 	struct fiemap_extent __user *dest = fieinfo->fi_extents_start;

@@ -109,6 +110,7 @@ int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 	extent.fe_physical = phys;
 	extent.fe_length = len;
 	extent.fe_flags = flags;
+	extent.fe_phys_length = phys_len;
 
 	dest += fieinfo->fi_extents_mapped;
 	if (copy_to_user(dest, &extent, sizeof(extent)))

@@ -318,10 +320,11 @@ int __generic_block_fiemap(struct inode *inode,
 				flags = FIEMAP_EXTENT_MERGED|FIEMAP_EXTENT_LAST;
 				ret = fiemap_fill_next_extent(fieinfo, logical,
 							      phys, size,
-							      flags);
+							      0, flags);
 			} else if (size) {
 				ret = fiemap_fill_next_extent(fieinfo, logical,
-							      phys, size, flags);
+							      phys, size,
+							      0, flags);
 				size = 0;
 			}

@@ -347,7 +350,7 @@ int __generic_block_fiemap(struct inode *inode,
 			if (start_blk > last_blk && !whole_file) {
 				ret = fiemap_fill_next_extent(fieinfo, logical,
 							      phys, size,
-							      flags);
+							      0, flags);
 				break;
 			}

@@ -358,7 +361,7 @@ int __generic_block_fiemap(struct inode *inode,
 			if (size) {
 				ret = fiemap_fill_next_extent(fieinfo, logical,
 							      phys, size,
-							      flags);
+							      0, flags);
 				if (ret)
 					break;
 			}

fs/nilfs2/inode.c +5 -3

index 6252b173a465..a99e6d093f11 100644
@@ -1017,7 +1017,8 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			if (size) {
 				/* End of the current extent */
 				ret = fiemap_fill_next_extent(
-					fieinfo, logical, phys, size, flags);
+					fieinfo, logical, phys, size, 0,
+					flags);
 				if (ret)
 					break;
 			}

@@ -1067,7 +1068,8 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 					flags |= FIEMAP_EXTENT_LAST;
 
 				ret = fiemap_fill_next_extent(
-					fieinfo, logical, phys, size, flags);
+					fieinfo, logical, phys, size,
+					0, flags);
 				if (ret)
 					break;
 				size = 0;

@@ -1083,7 +1085,7 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 					/* Terminate the current extent */
 					ret = fiemap_fill_next_extent(
 						fieinfo, logical, phys, size,
-						flags);
+						0, flags);
 					if (ret || blkoff > end_blkoff)
 						break;

fs/ocfs2/extent_map.c +2 -2

index 767370b656ca..cb0276908155 100644
@@ -736,7 +736,7 @@ static int ocfs2_fiemap_inline(struct inode *inode, struct buffer_head *di_bh,
 					 id2.i_data.id_data);
 
 		ret = fiemap_fill_next_extent(fieinfo, 0, phys, id_count,
-					      flags);
+					      0, flags);
 		if (ret < 0)
 			return ret;
 	}

@@ -809,7 +809,7 @@ int ocfs2_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 		virt_bytes = (u64)le32_to_cpu(rec.e_cpos) << osb->s_clustersize_bits;
 
 		ret = fiemap_fill_next_extent(fieinfo, virt_bytes, phys_bytes,
-					      len_bytes, fe_flags);
+					      len_bytes, 0, fe_flags);
 		if (ret)
 			break;

fs/xfs/xfs_iops.c +1 -1

index 205613a06068..5a7c1f6d1189 100644
@@ -1030,7 +1030,7 @@ xfs_fiemap_format(
 		fiemap_flags |= FIEMAP_EXTENT_LAST;
 
 	error = fiemap_fill_next_extent(fieinfo, logical, physical,
-					length, fiemap_flags);
+					length, 0, fiemap_flags);
 	if (error > 0) {
 		error = 0;
 		*full = 1;	/* user array now full */

include/linux/fs.h +1 -1

index e11d60cc867b..609e1d72c3e0 100644
@@ -1412,7 +1412,7 @@ struct fiemap_extent_info {
 							fiemap_extent array */
 };
 int fiemap_fill_next_extent(struct fiemap_extent_info *info, u64 logical,
-			    u64 phys, u64 len, u32 flags);
+			    u64 phys, u64 len, u64 phys_len, u32 flags);
 int fiemap_check_flags(struct fiemap_extent_info *fieinfo, u32 fs_flags);
 
 /*

include/uapi/linux/fiemap.h +5 -1

index 93abfcd9ac47..ed52d3f7821d 100644
@@ -19,7 +19,9 @@ struct fiemap_extent {
 	__u64 fe_physical; /* physical offset in bytes for the start
 			    * of the extent from the beginning of the disk */
 	__u64 fe_length;   /* length in bytes for this extent */
-	__u64 fe_reserved64[2];
+	__u64 fe_phys_length; /* physical length in bytes, may be different from
+                               * fe_length, is valid if PHYS_LENGTH flag set */
+	__u64 fe_reserved64;
 	__u32 fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
 	__u32 fe_reserved[3];
 };

@@ -50,6 +52,8 @@ struct fiemap {
 						    * Sets EXTENT_UNKNOWN. */
 #define FIEMAP_EXTENT_ENCODED		0x00000008 /* Data can not be read
 						    * while fs is unmounted */
+#define FIEMAP_EXTENT_PHYS_LENGTH	0x00000010 /* Physical length of extent
+						    * not the same as logical */
 #define FIEMAP_EXTENT_DATA_ENCRYPTED	0x00000080 /* Data is encrypted by fs.
 						    * Sets EXTENT_ENCODED */
 #define FIEMAP_EXTENT_NOT_ALIGNED	0x00000100 /* Extent offsets may not be

Signature · 1 line

1.8.4.5

A message that is not in the archive (sent off-list or not yet received): f257ce29393707293c0ce056bd8c6a7b3daa6082.1406739708.git.dsterba-AlSwsSmVLrQ@public.gmane.org

Andreas Dilger

2014-07-30 20:06 UTC #5 lore raw

Re: [PATCH 2/6] fiemap: add fe_phys_length and EXTENT_PHYS_LENGTH flag

Quoted 3 lines · David Sterba

Add a new member to fiemap_extent that represents the physical extent length. This value is undefined if the flag EXTENT_PHYS_LENGTH is not set.

The description here of PHYS_LENGTH makes sense...

The patch description should also mention the name of the new member, namely "fe_phys_length"

include/uapi/linux/fiemap.h struct fiemap { · line 56

Quoted 2 lines

include/uapi/linux/fiemap.h +2 -0

+#define FIEMAP_EXTENT_PHYS_LENGTH	0x00000010 /* Physical length of extent
+						    * not the same as logical */

But the comment doesn't match. This implies that if PHYS_LENGTH is set, fe_phys_length != fe_logi_length, but I don't think that is necessarily correct. I think it makes more sense to just set PHYS_LENGTH when fe_phys_length is valid, and if PHYS_LENGTH is not set then fe_phys_length aware applications should just use fe_phys_length = fe_logi_length, and older applications would just use fe_length for both as they would already today.

Cheers, Andreas

David Sterba

author

2014-07-30 17:18 UTC #6 lore raw

PATCH v5 3/6 fiemap: add FIEMAP_EXTENT_DATA_COMPRESSED flag

In reply to David Sterba

Add extent flag to indicate that the data are compressed. The users set the fe_phys_length member, and PHYS_LENGTH and ENCODED flags.

Signed-off-by David Sterba <[email protected]>

---
 fs/ioctl.c                  | 6 +++++-
 include/uapi/linux/fiemap.h | 4 ++++
 2 files changed, 9 insertions(+), 1 deletion(-)

fs/ioctl.c +5 -1

index 127551c70b0a..902b1d5a6316 100644
@@ -81,7 +81,8 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
  * extent that will fit in user array.
  */
 #define SET_UNKNOWN_FLAGS	(FIEMAP_EXTENT_DELALLOC)
-#define SET_NO_UNMOUNTED_IO_FLAGS	(FIEMAP_EXTENT_DATA_ENCRYPTED)
+#define SET_NO_UNMOUNTED_IO_FLAGS	(FIEMAP_EXTENT_DATA_ENCRYPTED | \
+					 FIEMAP_EXTENT_DATA_COMPRESSED)
 #define SET_NOT_ALIGNED_FLAGS	(FIEMAP_EXTENT_DATA_TAIL|FIEMAP_EXTENT_DATA_INLINE)
 int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 			    u64 phys, u64 len, u64 phys_len, u32 flags)

@@ -112,6 +113,9 @@ int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 	extent.fe_flags = flags;
 	extent.fe_phys_length = phys_len;
 
+	WARN_ON_ONCE((flags & FIEMAP_EXTENT_DATA_COMPRESSED)
+		&& !(flags & FIEMAP_EXTENT_ENCODED));
+
 	dest += fieinfo->fi_extents_mapped;
 	if (copy_to_user(dest, &extent, sizeof(extent)))
 		return -EFAULT;

include/uapi/linux/fiemap.h +4 -0

index ed52d3f7821d..64fb1b3a7e1f 100644
@@ -54,6 +54,10 @@ struct fiemap {
 						    * while fs is unmounted */
 #define FIEMAP_EXTENT_PHYS_LENGTH	0x00000010 /* Physical length of extent
 						    * not the same as logical */
+#define FIEMAP_EXTENT_DATA_COMPRESSED	0x00000040 /* Data is compressed by fs.
+						    * Sets EXTENT_ENCODED and
+						    * the compressed size is
+						    * stored in fe_phys_length */
 #define FIEMAP_EXTENT_DATA_ENCRYPTED	0x00000080 /* Data is encrypted by fs.
 						    * Sets EXTENT_ENCODED */
 #define FIEMAP_EXTENT_NOT_ALIGNED	0x00000100 /* Extent offsets may not be

Signature · 1 line

1.8.4.5

Andreas Dilger

2014-07-30 20:27 UTC #7 lore raw

Re: [PATCH 3/6] fiemap: add FIEMAP_EXTENT_DATA_COMPRESSED flag

Quoted 38 lines · David Sterba

Add extent flag to indicate that the data are compressed. The users set the fe_phys_length member, and PHYS_LENGTH and ENCODED flags.

Signed-off-by David Sterba <[email protected]>

---
fs/ioctl.c | 6 +++++-
include/uapi/linux/fiemap.h | 4 ++++

2 files changed, 9 insertions(+), 1 deletion(-)

fs/ioctl.c +5 -1

index 127551c70b0a..902b1d5a6316 100644
@@ -81,7 +81,8 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
 * extent that will fit in user array.
 */
 #define SET_UNKNOWN_FLAGS	(FIEMAP_EXTENT_DELALLOC)
-#define SET_NO_UNMOUNTED_IO_FLAGS	(FIEMAP_EXTENT_DATA_ENCRYPTED)
+#define SET_NO_UNMOUNTED_IO_FLAGS	(FIEMAP_EXTENT_DATA_ENCRYPTED | \
+					 FIEMAP_EXTENT_DATA_COMPRESSED)
 #define SET_NOT_ALIGNED_FLAGS	(FIEMAP_EXTENT_DATA_TAIL|FIEMAP_EXTENT_DATA_INLINE)
 int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 			    u64 phys, u64 len, u64 phys_len, u32 flags)

@@ -112,6 +113,9 @@ int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 	extent.fe_flags = flags;
 	extent.fe_phys_length = phys_len;
 
+	WARN_ON_ONCE((flags & FIEMAP_EXTENT_DATA_COMPRESSED)
+		&& !(flags & FIEMAP_EXTENT_ENCODED));
+
 	dest += fieinfo->fi_extents_mapped;
 	if (copy_to_user(dest, &extent, sizeof(extent)))
 		return -EFAULT;

include/uapi/linux/fiemap.h +0 -0

index ed52d3f7821d..64fb1b3a7e1f 100644

include/uapi/linux/fiemap.h struct fiemap { · line 60

Quoted 7 lines

include/uapi/linux/fiemap.h +4 -0

@@ -54,6 +54,10 @@ struct fiemap {
 						    * while fs is unmounted */
 #define FIEMAP_EXTENT_PHYS_LENGTH	0x00000010 /* Physical length of extent
 						    * not the same as logical */
+#define FIEMAP_EXTENT_DATA_COMPRESSED	0x00000040 /* Data is compressed by fs.
+						    * Sets EXTENT_ENCODED and
+						    * the compressed size is
+						    * stored in fe_phys_length */

This should also set PHYS_LENGTH.

Quoted 6 lines

include/uapi/linux/fiemap.h +0 -0

 #define FIEMAP_EXTENT_DATA_ENCRYPTED	0x00000080 /* Data is encrypted by fs.
 						    * Sets EXTENT_ENCODED */
 #define FIEMAP_EXTENT_NOT_ALIGNED	0x00000100 /* Extent offsets may not be

Signature · 1 line

1.8.4.5

Cheers, Andreas

David Sterba

author

2014-07-30 17:18 UTC #8 lore raw

PATCH v5 4/6 Documentation/fiemap: Document DATA_COMPRESSED and PHYS_LENGTH flags

In reply to David Sterba

CC [email protected]

CC Rob Landley <[email protected]>

Signed-off-by David Sterba <[email protected]>

---
 Documentation/filesystems/fiemap.txt | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

Documentation/filesystems/fiemap.txt +17 -2

index 1b805a0efbb0..1275dc27ab82 100644
@@ -83,7 +83,9 @@ struct fiemap_extent {
 	__u64	fe_physical; /* physical offset in bytes for the start
 			      * of the extent */
 	__u64	fe_length;   /* length in bytes for the extent */
-	__u64	fe_reserved64[2];
+	__u64   fe_phys_length; /* physical length in bytes, may be different from
+                                 * fe_length, is valid if PHYS_LENGTH flag set */
+	__u64	fe_reserved64;
 	__u32	fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
 	__u32	fe_reserved[3];
 };

@@ -93,7 +95,10 @@ for an extents logical offset to start before the request or its logical
 length to extend past the request.  Unless FIEMAP_EXTENT_NOT_ALIGNED is
 returned, fe_logical, fe_physical, and fe_length will be aligned to the
 block size of the file system.  With the exception of extents flagged as
-FIEMAP_EXTENT_MERGED, adjacent extents will not be merged.
+FIEMAP_EXTENT_MERGED, adjacent extents will not be merged.  The value of
+fe_phys_length is valid if FIEMAP_EXTENT_PHYS_LENGTH is set.  It may be
+different from fe_length and may set additional extent flags that indicate
+the reason, eg. for compression it's FIEMAP_EXTENT_DATA_COMPRESSED.
 
 The fe_flags field contains flags which describe the extent returned.
 A special flag, FIEMAP_EXTENT_LAST is always set on the last extent in

@@ -143,6 +148,16 @@ unmounted, and then only if the FIEMAP_EXTENT_ENCODED flag is
 clear; user applications must not try reading or writing to the
 filesystem via the block device under any other circumstances.
 
+* FIEMAP_EXTENT_PHYS_LENGTH
+If set then fe_phys_length is valid and contains the physical size of the
+extent.
+
+* FIEMAP_EXTENT_DATA_COMPRESSED
+  - This will also set FIEMAP_EXTENT_ENCODED
+The data in this extent has been compressed by the file system. When
+set, fe_phys_length contains the physical extent length in bytes. The
+value may be rounded depending on the filesystem implementation.
+
 * FIEMAP_EXTENT_DATA_ENCRYPTED
   - This will also set FIEMAP_EXTENT_ENCODED
 The data in this extent has been encrypted by the file system.

Signature · 1 line

1.8.4.5

Andreas Dilger

2014-07-30 20:33 UTC #9 lore raw

Re: [PATCH 4/6] Documentation/fiemap: Document DATA_COMPRESSED and PHYS_LENGTH flags

Documentation/filesystems/fiemap.txt unmounted, and then only if the FIEMAP_EXTENT_ENCODED flag is · line 159

Quoted 13 lines · David Sterba

(quoted diff) +0 -0

@@ -143,6 +148,16 @@ unmounted, and then only if the FIEMAP_EXTENT_ENCODED flag is

clear; user applications must not try reading or writing to the filesystem via the block device under any other circumstances.

Documentation/filesystems/fiemap.txt +9 -0

+* FIEMAP_EXTENT_PHYS_LENGTH
+If set then fe_phys_length is valid and contains the physical size of the
+extent.
+
+* FIEMAP_EXTENT_DATA_COMPRESSED
+  - This will also set FIEMAP_EXTENT_ENCODED
+The data in this extent has been compressed by the file system. When
+set, fe_phys_length contains the physical extent length in bytes. The
+value may be rounded depending on the filesystem implementation.

When DATA_COMPRESSED is set, then PHYS_LENGTH should always be set, since it is using fe_phys_length.

Cheers, Andreas

David Sterba

author

2014-07-30 17:18 UTC #10 lore raw

PATCH v5 5/6 fiemap: rename fe_length to fe_logi_length

In reply to David Sterba

Pick a more descriptive name for the logical length.

Signed-off-by David Sterba <[email protected]>

---
 Documentation/filesystems/fiemap.txt | 11 ++++++-----
 fs/ioctl.c                           |  2 +-
 include/uapi/linux/fiemap.h          |  6 ++++--
 3 files changed, 11 insertions(+), 8 deletions(-)

Documentation/filesystems/fiemap.txt +6 -5

index 1275dc27ab82..0e0b51e90cf4 100644
@@ -82,9 +82,10 @@ struct fiemap_extent {
 			      * the extent */
 	__u64	fe_physical; /* physical offset in bytes for the start
 			      * of the extent */
-	__u64	fe_length;   /* length in bytes for the extent */
+	__u64	fe_logi_length; /* logical length in bytes for the extent */
 	__u64   fe_phys_length; /* physical length in bytes, may be different from
-                                 * fe_length, is valid if PHYS_LENGTH flag set */
+                                 * fe_logi_length, is valid if PHYS_LENGTH flag
+				 * is set */
 	__u64	fe_reserved64;
 	__u32	fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
 	__u32	fe_reserved[3];

@@ -93,12 +94,12 @@ struct fiemap_extent {
 All offsets and lengths are in bytes and mirror those on disk.  It is valid
 for an extents logical offset to start before the request or its logical
 length to extend past the request.  Unless FIEMAP_EXTENT_NOT_ALIGNED is
-returned, fe_logical, fe_physical, and fe_length will be aligned to the
+returned, fe_logical, fe_physical, and fe_logi_length will be aligned to the
 block size of the file system.  With the exception of extents flagged as
 FIEMAP_EXTENT_MERGED, adjacent extents will not be merged.  The value of
 fe_phys_length is valid if FIEMAP_EXTENT_PHYS_LENGTH is set.  It may be
-different from fe_length and may set additional extent flags that indicate
-the reason, eg. for compression it's FIEMAP_EXTENT_DATA_COMPRESSED.
+different from fe_logi_length and may set additional extent flags that
+indicate the reason, eg. for compression it's FIEMAP_EXTENT_DATA_COMPRESSED.
 
 The fe_flags field contains flags which describe the extent returned.
 A special flag, FIEMAP_EXTENT_LAST is always set on the last extent in

fs/ioctl.c +1 -1

index 902b1d5a6316..442b5630d608 100644
@@ -109,7 +109,7 @@ int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 	memset(&extent, 0, sizeof(extent));
 	extent.fe_logical = logical;
 	extent.fe_physical = phys;
-	extent.fe_length = len;
+	extent.fe_logi_length = len;
 	extent.fe_flags = flags;
 	extent.fe_phys_length = phys_len;

include/uapi/linux/fiemap.h +4 -2

index 64fb1b3a7e1f..28ae5a0cbba0 100644
@@ -18,9 +18,11 @@ struct fiemap_extent {
 			    * the extent from the beginning of the file */
 	__u64 fe_physical; /* physical offset in bytes for the start
 			    * of the extent from the beginning of the disk */
-	__u64 fe_length;   /* length in bytes for this extent */
+#define fe_length fe_logi_length
+	__u64 fe_logi_length; /* logical length in bytes for this extent */
 	__u64 fe_phys_length; /* physical length in bytes, may be different from
-                               * fe_length, is valid if PHYS_LENGTH flag set */
+                               * fe_logi_length, is valid if PHYS_LENGTH flag
+			       * is set */
 	__u64 fe_reserved64;
 	__u32 fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
 	__u32 fe_reserved[3];

Signature · 1 line

1.8.4.5

Andreas Dilger

Reviewed-by on v5 5/6

2014-07-30 20:36 UTC #11 lore raw

Re: [PATCH 5/6] fiemap: rename fe_length to fe_logi_length

Quoted 3 lines · David Sterba

Pick a more descriptive name for the logical length.
Signed-off-by David Sterba <[email protected]>

Reviewed-by Andreas Dilger <[email protected]>

Quoted 73 lines

---

Documentation/filesystems/fiemap.txt | 11 ++++++-----
fs/ioctl.c                           |  2 +-
include/uapi/linux/fiemap.h          |  6 ++++--
3 files changed, 11 insertions(+), 8 deletions(-)

Documentation/filesystems/fiemap.txt +3 -2

index 1275dc27ab82..0e0b51e90cf4 100644
@@ -82,9 +82,10 @@ struct fiemap_extent {
 			      * the extent */
 	__u64	fe_physical; /* physical offset in bytes for the start
 			      * of the extent */
-	__u64	fe_length;   /* length in bytes for the extent */
+	__u64	fe_logi_length; /* logical length in bytes for the extent */
 	__u64   fe_phys_length; /* physical length in bytes, may be different from
-                                 * fe_length, is valid if PHYS_LENGTH flag set */
+                                 * fe_logi_length, is valid if PHYS_LENGTH flag
+				 * is set */
 	__u64	fe_reserved64;
 	__u32	fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
 	__u32	fe_reserved[3];

@@ -93,12 +94,12 @@ struct fiemap_extent {

All offsets and lengths are in bytes and mirror those on disk. It is valid for an extents logical offset to start before the request or its logical length to extend past the request. Unless FIEMAP_EXTENT_NOT_ALIGNED is

Documentation/filesystems/fiemap.txt +1 -1

-returned, fe_logical, fe_physical, and fe_length will be aligned to the
+returned, fe_logical, fe_physical, and fe_logi_length will be aligned to the

block size of the file system. With the exception of extents flagged as FIEMAP_EXTENT_MERGED, adjacent extents will not be merged. The value of fe_phys_length is valid if FIEMAP_EXTENT_PHYS_LENGTH is set. It may be

Documentation/filesystems/fiemap.txt +2 -2

-different from fe_length and may set additional extent flags that indicate
-the reason, eg. for compression it's FIEMAP_EXTENT_DATA_COMPRESSED.
+different from fe_logi_length and may set additional extent flags that
+indicate the reason, eg. for compression it's FIEMAP_EXTENT_DATA_COMPRESSED.

The fe_flags field contains flags which describe the extent returned. A special flag, FIEMAP_EXTENT_LAST is always set on the last extent in

fs/ioctl.c +1 -1

index 902b1d5a6316..442b5630d608 100644
@@ -109,7 +109,7 @@ int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 	memset(&extent, 0, sizeof(extent));
 	extent.fe_logical = logical;
 	extent.fe_physical = phys;
-	extent.fe_length = len;
+	extent.fe_logi_length = len;
 	extent.fe_flags = flags;
 	extent.fe_phys_length = phys_len;

include/uapi/linux/fiemap.h +4 -2

index 64fb1b3a7e1f..28ae5a0cbba0 100644
@@ -18,9 +18,11 @@ struct fiemap_extent {
 			    * the extent from the beginning of the file */
 	__u64 fe_physical; /* physical offset in bytes for the start
 			    * of the extent from the beginning of the disk */
-	__u64 fe_length;   /* length in bytes for this extent */
+#define fe_length fe_logi_length
+	__u64 fe_logi_length; /* logical length in bytes for this extent */
 	__u64 fe_phys_length; /* physical length in bytes, may be different from
-                               * fe_length, is valid if PHYS_LENGTH flag set */
+                               * fe_logi_length, is valid if PHYS_LENGTH flag
+			       * is set */
 	__u64 fe_reserved64;
 	__u32 fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
 	__u32 fe_reserved[3];

Signature · 1 line

1.8.4.5

Cheers, Andreas

David Sterba

author

2014-07-30 17:18 UTC #12 lore raw

PATCH v5 6/6 btrfs: set FIEMAP_EXTENT_DATA_COMPRESSED for compressed extents

In reply to David Sterba

Compressed extents set the physical length and the corresponging extent flags.

Signed-off-by David Sterba <[email protected]>

---
 fs/btrfs/extent_io.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

fs/btrfs/extent_io.c +8 -3

index c85da2e54ce7..12771d64b74e 100644
@@ -4272,6 +4272,7 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 
 	while (!end) {
 		u64 offset_in_extent = 0;
+		u64 em_phys_len = 0;
 
 		/* break if the extent we found is outside the range */
 		if (em->start >= max || extent_map_end(em) < off)

@@ -4334,8 +4335,12 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			if (ref_cnt > 1)
 				flags |= FIEMAP_EXTENT_SHARED;
 		}
-		if (test_bit(EXTENT_FLAG_COMPRESSED, &em->flags))
-			flags |= FIEMAP_EXTENT_ENCODED;
+		if (test_bit(EXTENT_FLAG_COMPRESSED, &em->flags)) {
+			flags |= FIEMAP_EXTENT_ENCODED |
+				 FIEMAP_EXTENT_DATA_COMPRESSED |
+				 FIEMAP_EXTENT_PHYS_LENGTH;
+			em_phys_len = em->block_len;
+		}
 
 		free_extent_map(em);
 		em = NULL;

@@ -4357,7 +4362,7 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			end = 1;
 		}
 		ret = fiemap_fill_next_extent(fieinfo, em_start, disko,
-					      em_len, 0, flags);
+					      em_len, em_phys_len, flags);
 		if (ret)
 			goto out_free;
 	}

Signature · 1 line

1.8.4.5