fiemap extension for more physical information

Some filesystems support compressed extents which have a larger logical size than physical, and for those filesystems, it can be useful for userspace to know how much space those extents actually use. For instance, the compsize [1] tool for btrfs currently uses btrfs-internal, root-only ioctl to find the actual disk space used by a file; it would be better and more useful for this information to require fewer privileges and to be usable on more filesystems. Therefore, use one of the padding u64s in the fiemap extent structure to return the actual physical length; and, for now, return this as equal to the logical length.

[1] https://github.com/kilobyte/compsize

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
 Documentation/filesystems/fiemap.rst | 28 +++++++++++++++++-------
 fs/ioctl.c                           |  3 ++-
 include/uapi/linux/fiemap.h          | 32 ++++++++++++++++++++++------
 3 files changed, 47 insertions(+), 16 deletions(-)

fs/ioctl.c +0 -0

 	extent.fe_flags = flags;
 
 	dest += fieinfo->fi_extents_mapped;

include/uapi/linux/fiemap.h +25 -7

index 24ca0c00cae3..3079159b8e94 100644
@@ -14,14 +14,30 @@
 
 #include <linux/types.h>
 
+/*
+ * For backward compatibility, where the member of the struct was called
+ * fe_length instead of fe_logical_length.
+ */
+#define fe_length fe_logical_length
+
 struct fiemap_extent {
-	__u64 fe_logical;  /* logical offset in bytes for the start of
-			    * the extent from the beginning of the file */
-	__u64 fe_physical; /* physical offset in bytes for the start
-			    * of the extent from the beginning of the disk */
-	__u64 fe_length;   /* length in bytes for this extent */
-	__u64 fe_reserved64[2];
-	__u32 fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
+	/*
+	 * logical offset in bytes for the start of
+	 * the extent from the beginning of the file
+	 */
+	__u64 fe_logical;
+	/*
+	 * physical offset in bytes for the start
+	 * of the extent from the beginning of the disk
+	 */
+	__u64 fe_physical;
+	/* logical length in bytes for this extent */
+	__u64 fe_logical_length;
+	/* physical length in bytes for this extent */
+	__u64 fe_physical_length;
+	__u64 fe_reserved64[1];
+	/* FIEMAP_EXTENT_* flags for this extent */
+	__u32 fe_flags;
 	__u32 fe_reserved[3];
 };
 

@@ -66,5 +82,7 @@ struct fiemap {
 						    * merged for efficiency. */
 #define FIEMAP_EXTENT_SHARED		0x00002000 /* Space shared with other
 						    * files. */
+#define FIEMAP_EXTENT_HAS_PHYS_LEN	0x00004000 /* Physical length is valid
+						    * and set by FS. */
 
 #endif /* _UAPI_LINUX_FIEMAP_H */

Signature · 1 line

2.43.0

Some filesystems support compressed extents which have a larger logical size than physical, and for those filesystems, it can be useful for userspace to know how much space those extents actually use. For instance, the compsize [1] tool for btrfs currently uses btrfs-internal, root-only ioctl to find the actual disk space used by a file; it would be better and more useful for this information to require fewer privileges and to be usable on more filesystems. Therefore, use one of the padding u64s in the fiemap extent structure to return the actual physical length; and, for now, return this as equal to the logical length.

[1] https://github.com/kilobyte/compsize

Signed-off-by Sweet Tea Dorminy <[email protected]>

---

Documentation/filesystems/fiemap.rst | 28 +++++++++++++++++-------
fs/ioctl.c                           |  3 ++-
include/uapi/linux/fiemap.h          | 32 ++++++++++++++++++++++------
3 files changed, 47 insertions(+), 16 deletions(-)

Documentation/filesystems/fiemap.rst +0 -0

index 93fc96f760aa..c2bfa107c8d7 100644
@@ -80,14 +80,24 @@ Each extent is described by a single fiemap_extent structure as

returned in fm_extents::

    struct fiemap_extent {

Documentation/filesystems/fiemap.rst +18 -8

-	    __u64	fe_logical;  /* logical offset in bytes for the start of
-				* the extent */
-	    __u64	fe_physical; /* physical offset in bytes for the start
-				* of the extent */
-	    __u64	fe_length;   /* length in bytes for the extent */
-	    __u64	fe_reserved64[2];
-	    __u32	fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
-	    __u32	fe_reserved[3];
+            /*
+             * logical offset in bytes for the start of
+             * the extent from the beginning of the file
+             */
+            __u64 fe_logical;
+            /*
+             * physical offset in bytes for the start
+             * of the extent from the beginning of the disk
+             */
+            __u64 fe_physical;
+            /* logical length in bytes for this extent */
+            __u64 fe_logical_length;
+            /* physical length in bytes for this extent */
+            __u64 fe_physical_length;
+            __u64 fe_reserved64[1];
+            /* FIEMAP_EXTENT_* flags for this extent */
+            __u32 fe_flags;
+            __u32 fe_reserved[3];
    };

All offsets and lengths are in bytes and mirror those on disk. It is valid

Documentation/filesystems/fiemap.rst +2 -0

@@ -175,6 +185,8 @@ FIEMAP_EXTENT_MERGED
  userspace would be highly inefficient, the kernel will try to merge most
  adjacent blocks into 'extents'.
 
+FIEMAP_EXTENT_HAS_PHYS_LEN
+  This will be set if the file system populated the physical length field.
 

VFS -> File System Implementation
---------------------------------

fs/ioctl.c +0 -0

index 661b46125669..8afd32e1a27a 100644

fs/ioctl.c +0 -0

 	extent.fe_flags = flags;
 
 	dest += fieinfo->fi_extents_mapped;

include/uapi/linux/fiemap.h +25 -8

index 24ca0c00cae3..3079159b8e94 100644
@@ -14,14 +14,30 @@
 
 #include <linux/types.h>
 
+/*
+ * For backward compatibility, where the member of the struct was called
+ * fe_length instead of fe_logical_length.
+ */
+#define fe_length fe_logical_length
+
 struct fiemap_extent {
-	__u64 fe_logical;  /* logical offset in bytes for the start of
-			    * the extent from the beginning of the file */
-	__u64 fe_physical; /* physical offset in bytes for the start
-			    * of the extent from the beginning of the disk */
-	__u64 fe_length;   /* length in bytes for this extent */
-	__u64 fe_reserved64[2];
-	__u32 fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
+	/*
+	 * logical offset in bytes for the start of
+	 * the extent from the beginning of the file
+	 */
+	__u64 fe_logical;
+	/*
+	 * physical offset in bytes for the start
+	 * of the extent from the beginning of the disk
+	 */
+	__u64 fe_physical;
+	/* logical length in bytes for this extent */
+	__u64 fe_logical_length;
+	/* physical length in bytes for this extent */
+	__u64 fe_physical_length;
+	__u64 fe_reserved64[1];
+	/* FIEMAP_EXTENT_* flags for this extent */
+	__u32 fe_flags;
 	__u32 fe_reserved[3];
 };
 

@@ -66,5 +82,7 @@ struct fiemap {
 						    * merged for efficiency. */
 #define FIEMAP_EXTENT_SHARED		0x00002000 /* Space shared with other
 						    * files. */
+#define FIEMAP_EXTENT_HAS_PHYS_LEN	0x00004000 /* Physical length is valid
+						    * and set by FS. */
 
 #endif /* _UAPI_LINUX_FIEMAP_H */
--

2.43.0

Some filesystems support compressed extents which have a larger logical size than physical, and for those filesystems, it can be useful for userspace to know how much space those extents actually use. For instance, the compsize [1] tool for btrfs currently uses btrfs-internal, root-only ioctl to find the actual disk space used by a file; it would be better and more useful for this information to require fewer privileges and to be usable on more filesystems. Therefore, use one of the padding u64s in the fiemap extent structure to return the actual physical length; and, for now, return this as equal to the logical length.

[1] https://github.com/kilobyte/compsize

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
 Documentation/filesystems/fiemap.rst | 28 +++++++++++++++++-------
 fs/ioctl.c                           |  3 ++-
 include/uapi/linux/fiemap.h          | 32 ++++++++++++++++++++++------
 3 files changed, 47 insertions(+), 16 deletions(-)

Documentation/filesystems/fiemap.rst +18 -8

index 93fc96f760aa..c2bfa107c8d7 100644
@@ -80,14 +80,24 @@ Each extent is described by a single fiemap_extent structure as
 returned in fm_extents::
 
     struct fiemap_extent {
-	    __u64	fe_logical;  /* logical offset in bytes for the start of
-				* the extent */
-	    __u64	fe_physical; /* physical offset in bytes for the start
-				* of the extent */
-	    __u64	fe_length;   /* length in bytes for the extent */
-	    __u64	fe_reserved64[2];
-	    __u32	fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
-	    __u32	fe_reserved[3];
+            /*
+             * logical offset in bytes for the start of
+             * the extent from the beginning of the file
+             */
+            __u64 fe_logical;
+            /*
+             * physical offset in bytes for the start
+             * of the extent from the beginning of the disk
+             */
+            __u64 fe_physical;
+            /* logical length in bytes for this extent */
+            __u64 fe_logical_length;
+            /* physical length in bytes for this extent */
+            __u64 fe_physical_length;
+            __u64 fe_reserved64[1];
+            /* FIEMAP_EXTENT_* flags for this extent */
+            __u32 fe_flags;
+            __u32 fe_reserved[3];
     };
 
 All offsets and lengths are in bytes and mirror those on disk.  It is valid

Documentation/filesystems/fiemap.rst +0 -0

 VFS -> File System Implementation
 ---------------------------------

fs/ioctl.c +2 -1

index 661b46125669..8afd32e1a27a 100644
@@ -138,7 +138,8 @@ int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 	memset(&extent, 0, sizeof(extent));
 	extent.fe_logical = logical;
 	extent.fe_physical = phys;
-	extent.fe_length = len;
+	extent.fe_logical_length = len;
+	extent.fe_physical_length = len;
 	extent.fe_flags = flags;
 
 	dest += fieinfo->fi_extents_mapped;

include/uapi/linux/fiemap.h +0 -0

index 24ca0c00cae3..3079159b8e94 100644

include/uapi/linux/fiemap.h +3 -7

+
 struct fiemap_extent {
-	__u64 fe_logical;  /* logical offset in bytes for the start of
-			    * the extent from the beginning of the file */
-	__u64 fe_physical; /* physical offset in bytes for the start
-			    * of the extent from the beginning of the disk */
-	__u64 fe_length;   /* length in bytes for this extent */
-	__u64 fe_reserved64[2];
-	__u32 fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
+	/*
+	 * logical offset in bytes for the start of

include/uapi/linux/fiemap.h +7 -0

+	/* physical length in bytes for this extent */
+	__u64 fe_physical_length;
+	__u64 fe_reserved64[1];
+	/* FIEMAP_EXTENT_* flags for this extent */
+	__u32 fe_flags;
 	__u32 fe_reserved[3];
 };
 

@@ -66,5 +82,7 @@ struct fiemap {
 						    * merged for efficiency. */
 #define FIEMAP_EXTENT_SHARED		0x00002000 /* Space shared with other
 						    * files. */
+#define FIEMAP_EXTENT_HAS_PHYS_LEN	0x00004000 /* Physical length is valid
+						    * and set by FS. */
 
 #endif /* _UAPI_LINUX_FIEMAP_H */

Signature · 1 line

2.43.0

Quoted 61 lines · Sweet Tea Dorminy

Some filesystems support compressed extents which have a larger logical size than physical, and for those filesystems, it can be useful for userspace to know how much space those extents actually use. For instance, the compsize [1] tool for btrfs currently uses btrfs-internal, root-only ioctl to find the actual disk space used by a file; it would be better and more useful for this information to require fewer privileges and to be usable on more filesystems. Therefore, use one of the padding u64s in the fiemap extent structure to return the actual physical length; and, for now, return this as equal to the logical length.

[1] https://github.com/kilobyte/compsize

Signed-off-by Sweet Tea Dorminy <[email protected]>

---

Documentation/filesystems/fiemap.rst | 28 +++++++++++++++++-------
fs/ioctl.c                           |  3 ++-
include/uapi/linux/fiemap.h          | 32 ++++++++++++++++++++++------
3 files changed, 47 insertions(+), 16 deletions(-)

Documentation/filesystems/fiemap.rst +0 -0

index 93fc96f760aa..c2bfa107c8d7 100644
@@ -80,14 +80,24 @@ Each extent is described by a single fiemap_extent structure as

returned in fm_extents::

    struct fiemap_extent {

Documentation/filesystems/fiemap.rst +18 -8

-	    __u64	fe_logical;  /* logical offset in bytes for the start of
-				* the extent */
-	    __u64	fe_physical; /* physical offset in bytes for the start
-				* of the extent */
-	    __u64	fe_length;   /* length in bytes for the extent */
-	    __u64	fe_reserved64[2];
-	    __u32	fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
-	    __u32	fe_reserved[3];
+            /*
+             * logical offset in bytes for the start of
+             * the extent from the beginning of the file
+             */
+            __u64 fe_logical;
+            /*
+             * physical offset in bytes for the start
+             * of the extent from the beginning of the disk
+             */
+            __u64 fe_physical;
+            /* logical length in bytes for this extent */
+            __u64 fe_logical_length;
+            /* physical length in bytes for this extent */
+            __u64 fe_physical_length;
+            __u64 fe_reserved64[1];
+            /* FIEMAP_EXTENT_* flags for this extent */
+            __u32 fe_flags;
+            __u32 fe_reserved[3];
    };

All offsets and lengths are in bytes and mirror those on disk. It is valid

Documentation/filesystems/fiemap.rst +0 -0

@@ -175,6 +185,8 @@ FIEMAP_EXTENT_MERGED
  userspace would be highly inefficient, the kernel will try to merge most
  adjacent blocks into 'extents'.
 

Documentation/filesystems/fiemap.rst FIEMAP_EXTENT_MERGED · line 189

Quoted 2 lines

Documentation/filesystems/fiemap.rst +2 -0

+FIEMAP_EXTENT_HAS_PHYS_LEN
+  This will be set if the file system populated the physical length field.

Just out of curiosity, should filesystems set this flag and fe_physical_length if fe_physical_length == fe_logical_length? Or just leave both blank?

Quoted 25 lines

VFS -> File System Implementation
---------------------------------

fs/ioctl.c +2 -1

index 661b46125669..8afd32e1a27a 100644
@@ -138,7 +138,8 @@ int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 	memset(&extent, 0, sizeof(extent));
 	extent.fe_logical = logical;
 	extent.fe_physical = phys;
-	extent.fe_length = len;
+	extent.fe_logical_length = len;
+	extent.fe_physical_length = len;
 	extent.fe_flags = flags;
 
 	dest += fieinfo->fi_extents_mapped;

include/uapi/linux/fiemap.h +0 -0

index 24ca0c00cae3..3079159b8e94 100644
@@ -14,14 +14,30 @@
 
 #include <linux/types.h>
 

include/uapi/linux/fiemap.h line 21

Quoted 5 lines

include/uapi/linux/fiemap.h +5 -0

+/*
+ * For backward compatibility, where the member of the struct was called
+ * fe_length instead of fe_logical_length.
+ */
+#define fe_length fe_logical_length

This #define has global scope; are you sure this isn't going to cause a weird build problem downstream with some program that declares an unrelated fe_length symbol?

Quoted 11 lines

include/uapi/linux/fiemap.h +3 -7

+
 struct fiemap_extent {
-	__u64 fe_logical;  /* logical offset in bytes for the start of
-			    * the extent from the beginning of the file */
-	__u64 fe_physical; /* physical offset in bytes for the start
-			    * of the extent from the beginning of the disk */
-	__u64 fe_length;   /* length in bytes for this extent */
-	__u64 fe_reserved64[2];
-	__u32 fe_flags;    /* FIEMAP_EXTENT_* flags for this extent */
+	/*
+	 * logical offset in bytes for the start of

include/uapi/linux/fiemap.h line 35

Quoted 10 lines

include/uapi/linux/fiemap.h +10 -0

+	 * the extent from the beginning of the file
+	 */
+	__u64 fe_logical;
+	/*
+	 * physical offset in bytes for the start
+	 * of the extent from the beginning of the disk
+	 */
+	__u64 fe_physical;
+	/* logical length in bytes for this extent */
+	__u64 fe_logical_length;

Or why not just leave the field name the same since the "logical length in bytes" comment is present both here in the header and again in the documentation?

--D

Quoted 15 lines

include/uapi/linux/fiemap.h +7 -0

+	/* physical length in bytes for this extent */
+	__u64 fe_physical_length;
+	__u64 fe_reserved64[1];
+	/* FIEMAP_EXTENT_* flags for this extent */
+	__u32 fe_flags;
 	__u32 fe_reserved[3];
 };
 

@@ -66,5 +82,7 @@ struct fiemap {
 						    * merged for efficiency. */
 #define FIEMAP_EXTENT_SHARED		0x00002000 /* Space shared with other
 						    * files. */
+#define FIEMAP_EXTENT_HAS_PHYS_LEN	0x00004000 /* Physical length is valid
+						    * and set by FS. */
 

Quoted 5 lines

include/uapi/linux/fiemap.h +0 -1

 #endif /* _UAPI_LINUX_FIEMAP_H */
--

2.43.0

Update the signature of fiemap_fill_next_extent() to allow passing a physical length. Update all callers to pass a 0 physical length -- since none set the EXTENT_HAS_PHYS_LEN flag, this value doesn't matter.

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
 Documentation/filesystems/fiemap.rst | 3 ++-
 fs/bcachefs/fs.c                     | 7 ++++---
 fs/btrfs/extent_io.c                 | 4 ++--
 fs/ext4/extents.c                    | 1 +
 fs/f2fs/data.c                       | 8 +++++---
 fs/f2fs/inline.c                     | 3 ++-
 fs/ioctl.c                           | 9 +++++----
 fs/iomap/fiemap.c                    | 2 +-
 fs/nilfs2/inode.c                    | 6 +++---
 fs/ntfs3/frecord.c                   | 7 ++++---
 fs/ocfs2/extent_map.c                | 4 ++--
 fs/smb/client/smb2ops.c              | 1 +
 include/linux/fiemap.h               | 2 +-
 13 files changed, 33 insertions(+), 24 deletions(-)

fs/ioctl.c +3 -2

index 8afd32e1a27a..1830baca532b 100644
@@ -99,7 +99,8 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
  * @fieinfo:	Fiemap context passed into ->fiemap
  * @logical:	Extent logical start offset, in bytes
  * @phys:	Extent physical start offset, in bytes
- * @len:	Extent length, in bytes
+ * @log_len:	Extent logical length, in bytes
+ * @phys_len:	Extent physical length, in bytes (optional)
  * @flags:	FIEMAP_EXTENT flags that describe this extent
  *
  * Called from file system ->fiemap callback. Will populate extent

@@ -110,7 +111,7 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
  * extent that will fit in user array.
  */
 int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
-			    u64 phys, u64 len, u32 flags)
+			    u64 phys, u64 log_len, u64 phys_len, u32 flags)
 {
 	struct fiemap_extent extent;
 	struct fiemap_extent __user *dest = fieinfo->fi_extents_start;

fs/ioctl.c +0 -0

 	extent.fe_flags = flags;
 
 	dest += fieinfo->fi_extents_mapped;

fs/iomap/fiemap.c +1 -1

index 610ca6f1ec9b..013e843c8d10 100644
@@ -36,7 +36,7 @@ static int iomap_to_fiemap(struct fiemap_extent_info *fi,
 
 	return fiemap_fill_next_extent(fi, iomap->offset,
 			iomap->addr != IOMAP_NULL_ADDR ? iomap->addr : 0,
-			iomap->length, flags);
+			iomap->length, 0, flags);
 }
 
 static loff_t iomap_fiemap_iter(const struct iomap_iter *iter,

fs/nilfs2/inode.c +3 -3

index 7340a01d80e1..4d3c347c982b 100644
@@ -1190,7 +1190,7 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			if (size) {
 				/* End of the current extent */
 				ret = fiemap_fill_next_extent(
-					fieinfo, logical, phys, size, flags);
+					fieinfo, logical, phys, size, 0, flags);
 				if (ret)
 					break;
 			}

@@ -1240,7 +1240,7 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 					flags |= FIEMAP_EXTENT_LAST;
 
 				ret = fiemap_fill_next_extent(
-					fieinfo, logical, phys, size, flags);
+					fieinfo, logical, phys, size, 0, flags);
 				if (ret)
 					break;
 				size = 0;

@@ -1256,7 +1256,7 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 					/* Terminate the current extent */
 					ret = fiemap_fill_next_extent(
 						fieinfo, logical, phys, size,
-						flags);
+						0, flags);
 					if (ret || blkoff > end_blkoff)
 						break;
 

fs/ntfs3/frecord.c +4 -3

index 7f27382e0ce2..ef0ed913428b 100644
@@ -1947,7 +1947,7 @@ int ni_fiemap(struct ntfs_inode *ni, struct fiemap_extent_info *fieinfo,
 	if (!attr || !attr->non_res) {
 		err = fiemap_fill_next_extent(
 			fieinfo, 0, 0,
-			attr ? le32_to_cpu(attr->res.data_size) : 0,
+			attr ? le32_to_cpu(attr->res.data_size) : 0, 0,
 			FIEMAP_EXTENT_DATA_INLINE | FIEMAP_EXTENT_LAST |
 				FIEMAP_EXTENT_MERGED);
 		goto out;

@@ -2042,7 +2042,7 @@ int ni_fiemap(struct ntfs_inode *ni, struct fiemap_extent_info *fieinfo,
 				flags |= FIEMAP_EXTENT_LAST;
 
 			err = fiemap_fill_next_extent(fieinfo, vbo, lbo, dlen,
-						      flags);
+						      0, flags);
 			if (err < 0)
 				break;
 			if (err == 1) {

@@ -2062,7 +2062,8 @@ int ni_fiemap(struct ntfs_inode *ni, struct fiemap_extent_info *fieinfo,
 		if (vbo + bytes >= end)
 			flags |= FIEMAP_EXTENT_LAST;
 
-		err = fiemap_fill_next_extent(fieinfo, vbo, lbo, bytes, flags);
+		err = fiemap_fill_next_extent(fieinfo, vbo, lbo, bytes, 0,
+					      flags);
 		if (err < 0)
 			break;
 		if (err == 1) {

fs/ocfs2/extent_map.c +2 -2

index 70a768b623cf..eabdf97cd685 100644
@@ -723,7 +723,7 @@ static int ocfs2_fiemap_inline(struct inode *inode, struct buffer_head *di_bh,
 					 id2.i_data.id_data);
 
 		ret = fiemap_fill_next_extent(fieinfo, 0, phys, id_count,
-					      flags);
+					      0, flags);
 		if (ret < 0)
 			return ret;
 	}

@@ -794,7 +794,7 @@ int ocfs2_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 		virt_bytes = (u64)le32_to_cpu(rec.e_cpos) << osb->s_clustersize_bits;
 
 		ret = fiemap_fill_next_extent(fieinfo, virt_bytes, phys_bytes,
-					      len_bytes, fe_flags);
+					      len_bytes, 0, fe_flags);
 		if (ret)
 			break;
 

fs/smb/client/smb2ops.c +1 -0

index 87b63f6ad2e2..23a193512f96 100644
@@ -3779,6 +3779,7 @@ static int smb3_fiemap(struct cifs_tcon *tcon,
 				le64_to_cpu(out_data[i].file_offset),
 				le64_to_cpu(out_data[i].file_offset),
 				le64_to_cpu(out_data[i].length),
+				0,
 				flags);
 		if (rc < 0)
 			goto out;

include/linux/fiemap.h +1 -1

index c50882f19235..17a6c32cdf3f 100644
@@ -16,6 +16,6 @@ struct fiemap_extent_info {
 int fiemap_prep(struct inode *inode, struct fiemap_extent_info *fieinfo,
 		u64 start, u64 *len, u32 supported_flags);
 int fiemap_fill_next_extent(struct fiemap_extent_info *info, u64 logical,
-			    u64 phys, u64 len, u32 flags);
+			    u64 phys, u64 log_len, u64 phys_len, u32 flags);
 
 #endif /* _LINUX_FIEMAP_H 1 */

Signature · 1 line

2.43.0

Update the signature of fiemap_fill_next_extent() to allow passing a physical length. Update all callers to pass a 0 physical length -- since none set the EXTENT_HAS_PHYS_LEN flag, this value doesn't matter.

Signed-off-by Sweet Tea Dorminy <[email protected]>

---

Documentation/filesystems/fiemap.rst | 3 ++-
fs/bcachefs/fs.c                     | 7 ++++---
fs/btrfs/extent_io.c                 | 4 ++--
fs/ext4/extents.c                    | 1 +
fs/f2fs/data.c                       | 8 +++++---
fs/f2fs/inline.c                     | 3 ++-
fs/ioctl.c                           | 9 +++++----
fs/iomap/fiemap.c                    | 2 +-
fs/nilfs2/inode.c                    | 6 +++---
fs/ntfs3/frecord.c                   | 7 ++++---
fs/ocfs2/extent_map.c                | 4 ++--
fs/smb/client/smb2ops.c              | 1 +
include/linux/fiemap.h               | 2 +-
13 files changed, 33 insertions(+), 24 deletions(-)

Documentation/filesystems/fiemap.rst +0 -0

index c2bfa107c8d7..c060bb83f5d8 100644
@@ -236,7 +236,8 @@ For each extent in the request range, the file system should call

the helper function, fiemap_fill_next_extent()::

int fiemap_fill_next_extent(struct fiemap_extent_info *info, u64 logical,

Documentation/filesystems/fiemap.rst +2 -1

-			      u64 phys, u64 len, u32 flags, u32 dev);
+			      u64 phys, u64 log_len, u64 phys_len, u32 flags,
+                              u32 dev);

fiemap_fill_next_extent() will use the passed values to populate the next free extent in the fm_extents array. 'General' extent flags will

fs/bcachefs/fs.c +4 -3

index 71013256fc39..f830578a9cd1 100644
@@ -931,7 +931,8 @@ static int bch2_fill_extent(struct bch_fs *c,
 			ret = fiemap_fill_next_extent(info,
 						bkey_start_offset(k.k) << 9,
 						offset << 9,
-						k.k->size << 9, flags|flags2);
+						k.k->size << 9, 0,
+						flags|flags2);
 			if (ret)
 				return ret;
 		}

@@ -940,13 +941,13 @@ static int bch2_fill_extent(struct bch_fs *c,
 	} else if (bkey_extent_is_inline_data(k.k)) {
 		return fiemap_fill_next_extent(info,
 					       bkey_start_offset(k.k) << 9,
-					       0, k.k->size << 9,
+					       0, k.k->size << 9, 0,
 					       flags|
 					       FIEMAP_EXTENT_DATA_INLINE);
 	} else if (k.k->type == KEY_TYPE_reservation) {
 		return fiemap_fill_next_extent(info,
 					       bkey_start_offset(k.k) << 9,
-					       0, k.k->size << 9,
+					       0, k.k->size << 9, 0,
 					       flags|
 					       FIEMAP_EXTENT_DELALLOC|
 					       FIEMAP_EXTENT_UNWRITTEN);

fs/btrfs/extent_io.c +2 -2

index eceef5ff780b..9e421d99fd5c 100644
@@ -2577,7 +2577,7 @@ static int flush_fiemap_cache(struct fiemap_extent_info *fieinfo,
 		int ret;
 
 		ret = fiemap_fill_next_extent(fieinfo, entry->offset,
-					      entry->phys, entry->len,
+					      entry->phys, entry->len, 0,
 					      entry->flags);
 		/*
 		 * Ignore 1 (reached max entries) because we keep track of that

@@ -2793,7 +2793,7 @@ static int emit_last_fiemap_cache(struct fiemap_extent_info *fieinfo,
 		return 0;
 
 	ret = fiemap_fill_next_extent(fieinfo, cache->offset, cache->phys,
-				      cache->len, cache->flags);
+				      cache->len, 0, cache->flags);
 	cache->cached = false;
 	if (ret > 0)
 		ret = 0;

fs/ext4/extents.c +1 -0

index e57054bdc5fd..2adade3c202a 100644
@@ -2215,6 +2215,7 @@ static int ext4_fill_es_cache_info(struct inode *inode,
 				(__u64)es.es_lblk << blksize_bits,
 				(__u64)es.es_pblk << blksize_bits,
 				(__u64)es.es_len << blksize_bits,
+				0,
 				flags);
 		if (next == 0)
 			break;

fs/f2fs/data.c +5 -3

index d9494b5fc7c1..87f8d828e038 100644
@@ -1834,7 +1834,8 @@ static int f2fs_xattr_fiemap(struct inode *inode,
 		if (!xnid)
 			flags |= FIEMAP_EXTENT_LAST;
 
-		err = fiemap_fill_next_extent(fieinfo, 0, phys, len, flags);
+		err = fiemap_fill_next_extent(
+				fieinfo, 0, phys, len, 0, flags);
 		trace_f2fs_fiemap(inode, 0, phys, len, flags, err);
 		if (err)
 			return err;

@@ -1860,7 +1861,8 @@ static int f2fs_xattr_fiemap(struct inode *inode,
 	}
 
 	if (phys) {
-		err = fiemap_fill_next_extent(fieinfo, 0, phys, len, flags);
+		err = fiemap_fill_next_extent(
+				fieinfo, 0, phys, len, 0, flags);
 		trace_f2fs_fiemap(inode, 0, phys, len, flags, err);
 	}
 

@@ -1979,7 +1981,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			flags |= FIEMAP_EXTENT_DATA_ENCRYPTED;
 
 		ret = fiemap_fill_next_extent(fieinfo, logical,
-				phys, size, flags);
+				phys, size, 0, flags);
 		trace_f2fs_fiemap(inode, logical, phys, size, flags, ret);
 		if (ret)
 			goto out;

fs/f2fs/inline.c +2 -1

index ac00423f117b..49d2f87fe048 100644
@@ -806,7 +806,8 @@ int f2fs_inline_data_fiemap(struct inode *inode,
 	byteaddr = (__u64)ni.blk_addr << inode->i_sb->s_blocksize_bits;
 	byteaddr += (char *)inline_data_addr(inode, ipage) -
 					(char *)F2FS_INODE(ipage);
-	err = fiemap_fill_next_extent(fieinfo, start, byteaddr, ilen, flags);
+	err = fiemap_fill_next_extent(
+			fieinfo, start, byteaddr, ilen, 0, flags);
 	trace_f2fs_fiemap(inode, start, byteaddr, ilen, flags, err);

out:

	f2fs_put_page(ipage, 1);

fs/ioctl.c +5 -4

index 8afd32e1a27a..1830baca532b 100644
@@ -99,7 +99,8 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
 * @fieinfo:	Fiemap context passed into ->fiemap
 * @logical:	Extent logical start offset, in bytes
 * @phys:	Extent physical start offset, in bytes
- * @len:	Extent length, in bytes
+ * @log_len:	Extent logical length, in bytes
+ * @phys_len:	Extent physical length, in bytes (optional)
 * @flags:	FIEMAP_EXTENT flags that describe this extent
 *
 * Called from file system ->fiemap callback. Will populate extent

@@ -110,7 +111,7 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
 * extent that will fit in user array.
 */
 int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
-			    u64 phys, u64 len, u32 flags)
+			    u64 phys, u64 log_len, u64 phys_len, u32 flags)
 {
 	struct fiemap_extent extent;
 	struct fiemap_extent __user *dest = fieinfo->fi_extents_start;

@@ -138,8 +139,8 @@ int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 	memset(&extent, 0, sizeof(extent));
 	extent.fe_logical = logical;
 	extent.fe_physical = phys;
-	extent.fe_logical_length = len;
-	extent.fe_physical_length = len;
+	extent.fe_logical_length = log_len;
+	extent.fe_physical_length = phys_len;
 	extent.fe_flags = flags;
 
 	dest += fieinfo->fi_extents_mapped;

fs/iomap/fiemap.c +1 -1

index 610ca6f1ec9b..013e843c8d10 100644
@@ -36,7 +36,7 @@ static int iomap_to_fiemap(struct fiemap_extent_info *fi,
 
 	return fiemap_fill_next_extent(fi, iomap->offset,
 			iomap->addr != IOMAP_NULL_ADDR ? iomap->addr : 0,
-			iomap->length, flags);
+			iomap->length, 0, flags);
 }
 
 static loff_t iomap_fiemap_iter(const struct iomap_iter *iter,

fs/nilfs2/inode.c +3 -3

index 7340a01d80e1..4d3c347c982b 100644
@@ -1190,7 +1190,7 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			if (size) {
 				/* End of the current extent */
 				ret = fiemap_fill_next_extent(
-					fieinfo, logical, phys, size, flags);
+					fieinfo, logical, phys, size, 0, flags);
 				if (ret)
 					break;
 			}

@@ -1240,7 +1240,7 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 					flags |= FIEMAP_EXTENT_LAST;
 
 				ret = fiemap_fill_next_extent(
-					fieinfo, logical, phys, size, flags);
+					fieinfo, logical, phys, size, 0, flags);
 				if (ret)
 					break;
 				size = 0;

@@ -1256,7 +1256,7 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 					/* Terminate the current extent */
 					ret = fiemap_fill_next_extent(
 						fieinfo, logical, phys, size,
-						flags);
+						0, flags);
 					if (ret || blkoff > end_blkoff)
 						break;
 

fs/ntfs3/frecord.c +4 -3

index 7f27382e0ce2..ef0ed913428b 100644
@@ -1947,7 +1947,7 @@ int ni_fiemap(struct ntfs_inode *ni, struct fiemap_extent_info *fieinfo,
 	if (!attr || !attr->non_res) {
 		err = fiemap_fill_next_extent(
 			fieinfo, 0, 0,
-			attr ? le32_to_cpu(attr->res.data_size) : 0,
+			attr ? le32_to_cpu(attr->res.data_size) : 0, 0,
 			FIEMAP_EXTENT_DATA_INLINE | FIEMAP_EXTENT_LAST |
 				FIEMAP_EXTENT_MERGED);
 		goto out;

@@ -2042,7 +2042,7 @@ int ni_fiemap(struct ntfs_inode *ni, struct fiemap_extent_info *fieinfo,
 				flags |= FIEMAP_EXTENT_LAST;
 
 			err = fiemap_fill_next_extent(fieinfo, vbo, lbo, dlen,
-						      flags);
+						      0, flags);
 			if (err < 0)
 				break;
 			if (err == 1) {

@@ -2062,7 +2062,8 @@ int ni_fiemap(struct ntfs_inode *ni, struct fiemap_extent_info *fieinfo,
 		if (vbo + bytes >= end)
 			flags |= FIEMAP_EXTENT_LAST;
 
-		err = fiemap_fill_next_extent(fieinfo, vbo, lbo, bytes, flags);
+		err = fiemap_fill_next_extent(fieinfo, vbo, lbo, bytes, 0,
+					      flags);
 		if (err < 0)
 			break;
 		if (err == 1) {

fs/ocfs2/extent_map.c +2 -2

index 70a768b623cf..eabdf97cd685 100644
@@ -723,7 +723,7 @@ static int ocfs2_fiemap_inline(struct inode *inode, struct buffer_head *di_bh,
 					 id2.i_data.id_data);
 
 		ret = fiemap_fill_next_extent(fieinfo, 0, phys, id_count,
-					      flags);
+					      0, flags);
 		if (ret < 0)
 			return ret;
 	}

@@ -794,7 +794,7 @@ int ocfs2_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 		virt_bytes = (u64)le32_to_cpu(rec.e_cpos) << osb->s_clustersize_bits;
 
 		ret = fiemap_fill_next_extent(fieinfo, virt_bytes, phys_bytes,
-					      len_bytes, fe_flags);
+					      len_bytes, 0, fe_flags);
 		if (ret)
 			break;
 

fs/smb/client/smb2ops.c +1 -0

index 87b63f6ad2e2..23a193512f96 100644
@@ -3779,6 +3779,7 @@ static int smb3_fiemap(struct cifs_tcon *tcon,
 				le64_to_cpu(out_data[i].file_offset),
 				le64_to_cpu(out_data[i].file_offset),
 				le64_to_cpu(out_data[i].length),
+				0,
 				flags);
 		if (rc < 0)
 			goto out;

include/linux/fiemap.h +1 -2

index c50882f19235..17a6c32cdf3f 100644
@@ -16,6 +16,6 @@ struct fiemap_extent_info {
 int fiemap_prep(struct inode *inode, struct fiemap_extent_info *fieinfo,
 		u64 start, u64 *len, u32 supported_flags);
 int fiemap_fill_next_extent(struct fiemap_extent_info *info, u64 logical,
-			    u64 phys, u64 len, u32 flags);
+			    u64 phys, u64 log_len, u64 phys_len, u32 flags);
 
 #endif /* _LINUX_FIEMAP_H 1 */
--

2.43.0

Quoted 4 lines · Sweet Tea Dorminy

Update the signature of fiemap_fill_next_extent() to allow passing a physical length. Update all callers to pass a 0 physical length -- since none set the EXTENT_HAS_PHYS_LEN flag, this value doesn't matter.

Patch-structure-wise, it doesn't make sense to me to change all of the callers to pass "0" as the argument to this function, and then submit a whole series of per-filesystem patches that sets only FIEMAP_EXTENT_HAS_PHYS_LEN (but also passes phys_len = 0, which is wrong AFAICS).

A cleaner approach would be to rename the existing fiemap_fill_next_extent() to fiemap_fill_next_extent_phys() that takes phys_len as an argument, and then add a simple wrapper until all of the filesystems are updated:

#define fiemap_fill_next_extent(info, logical, phys, log_len, flags, dev) \
fiemap_fill_next_extent_phys(info, logical, phys, log_len, 0, flags, dev)

Then the per-filesystem patches would involve changing over the callers to use fiemap_fill_next_extent_phys() and setting FIEMAP_EXTENT_HAS_PHYS_LEN.

This goes closely with the new physical length field in struct fiemap_extent, as when physical length is not equal to logical length the reason is frequently compression.

Signed-off-by Sweet Tea Dorminy <[email protected]>

---

Documentation/filesystems/fiemap.rst | 4 ++++
fs/ioctl.c                           | 3 ++-
include/uapi/linux/fiemap.h          | 2 ++
3 files changed, 8 insertions(+), 1 deletion(-)

Documentation/filesystems/fiemap.rst +4 -0

index c060bb83f5d8..16bd7faba5e0 100644
@@ -162,6 +162,10 @@ FIEMAP_EXTENT_DATA_ENCRYPTED
  This will also set FIEMAP_EXTENT_ENCODED
  The data in this extent has been encrypted by the file system.
 
+FIEMAP_EXTENT_DATA_COMPRESSED
+  This will also set FIEMAP_EXTENT_ENCODED
+  The data in this extent is compressed by the file system.
+

FIEMAP_EXTENT_NOT_ALIGNED
Extent offsets and length are not guaranteed to be block aligned.

fs/ioctl.c +2 -1

index 1830baca532b..b47e2da7ec17 100644
@@ -126,7 +126,8 @@ int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo, u64 logical,
 		return 1;
 
 #define SET_UNKNOWN_FLAGS	(FIEMAP_EXTENT_DELALLOC)
-#define SET_NO_UNMOUNTED_IO_FLAGS	(FIEMAP_EXTENT_DATA_ENCRYPTED)
+#define SET_NO_UNMOUNTED_IO_FLAGS	(FIEMAP_EXTENT_DATA_ENCRYPTED|\
+					 FIEMAP_EXTENT_DATA_COMPRESSED)
 #define SET_NOT_ALIGNED_FLAGS	(FIEMAP_EXTENT_DATA_TAIL|FIEMAP_EXTENT_DATA_INLINE)
 
 	if (flags & SET_UNKNOWN_FLAGS)

include/uapi/linux/fiemap.h +2 -1

index 3079159b8e94..ea97e33ddbb3 100644
@@ -67,6 +67,8 @@ struct fiemap {
 						    * Sets EXTENT_UNKNOWN. */
 #define FIEMAP_EXTENT_ENCODED		0x00000008 /* Data can not be read
 						    * while fs is unmounted */
+#define FIEMAP_EXTENT_DATA_COMPRESSED	0x00000040 /* Data is compressed by fs.
+						    * Sets EXTENT_ENCODED. */
 #define FIEMAP_EXTENT_DATA_ENCRYPTED	0x00000080 /* Data is encrypted by fs.
 						    * Sets EXTENT_NO_BYPASS. */
 #define FIEMAP_EXTENT_NOT_ALIGNED	0x00000100 /* Extent offsets may not be
--

2.43.0

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
fs/btrfs/extent_io.c | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

fs/btrfs/extent_io.c +2 -3

index 9e421d99fd5c..e9df670ef7d2 100644
@@ -2706,7 +2706,7 @@ static int emit_fiemap_extent(struct fiemap_extent_info *fieinfo,
 			if (range_end <= cache_end)
 				return 0;
 
-			if (!(flags & (FIEMAP_EXTENT_ENCODED | FIEMAP_EXTENT_DELALLOC)))
+			if (!(flags & (FIEMAP_EXTENT_DATA_COMPRESSED | FIEMAP_EXTENT_DELALLOC)))
 				phys += cache_end - offset;
 
 			offset = cache_end;

@@ -3236,7 +3236,7 @@ int extent_fiemap(struct btrfs_inode *inode, struct fiemap_extent_info *fieinfo,
 		}
 
 		if (compression != BTRFS_COMPRESS_NONE)
-			flags |= FIEMAP_EXTENT_ENCODED;
+			flags |= FIEMAP_EXTENT_DATA_COMPRESSED;
 
 		if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
 			flags |= FIEMAP_EXTENT_DATA_INLINE;
--

2.43.0

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
fs/nilfs2/inode.c | 12 ++++++++----

1 file changed, 8 insertions(+), 4 deletions(-)

fs/nilfs2/inode.c +1 -1

index 4d3c347c982b..e3108f2cead7 100644
@@ -1160,7 +1160,7 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 {
 	struct the_nilfs *nilfs = inode->i_sb->s_fs_info;
 	__u64 logical = 0, phys = 0, size = 0;
-	__u32 flags = 0;
+	__u32 flags = FIEMAP_EXTENT_HAS_PHYS_LEN;
 
 	loff_t isize;
 	sector_t blkoff, end_blkoff;

	sector_t delalloc_blkoff;

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
 fs/ext4/extents.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

fs/ext4/extents.c +0 -0

index 2adade3c202a..4874f757e1bd 100644

Signed-off-by Sweet Tea Dorminy <[email protected]>

---

fs/f2fs/data.c              |  6 +++---
fs/f2fs/inline.c            |  2 +-
include/trace/events/f2fs.h | 10 +++++++---
3 files changed, 11 insertions(+), 7 deletions(-)

fs/f2fs/data.c +3 -3

index 87f8d828e038..34af1673461b 100644
@@ -1836,7 +1836,7 @@ static int f2fs_xattr_fiemap(struct inode *inode,
 
 		err = fiemap_fill_next_extent(
 				fieinfo, 0, phys, len, 0, flags);
-		trace_f2fs_fiemap(inode, 0, phys, len, flags, err);
+		trace_f2fs_fiemap(inode, 0, phys, len, 0, flags, err);
 		if (err)
 			return err;
 	}

@@ -1863,7 +1863,7 @@ static int f2fs_xattr_fiemap(struct inode *inode,
 	if (phys) {
 		err = fiemap_fill_next_extent(
 				fieinfo, 0, phys, len, 0, flags);
-		trace_f2fs_fiemap(inode, 0, phys, len, flags, err);
+		trace_f2fs_fiemap(inode, 0, phys, len, 0, flags, err);
 	}
 
 	return (err < 0 ? err : 0);

@@ -1982,7 +1982,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 
 		ret = fiemap_fill_next_extent(fieinfo, logical,
 				phys, size, 0, flags);
-		trace_f2fs_fiemap(inode, logical, phys, size, flags, ret);
+		trace_f2fs_fiemap(inode, logical, phys, size, 0, flags, ret);
 		if (ret)
 			goto out;
 		size = 0;

fs/f2fs/inline.c +1 -1

index 49d2f87fe048..235b0d72f6fc 100644
@@ -808,7 +808,7 @@ int f2fs_inline_data_fiemap(struct inode *inode,
 					(char *)F2FS_INODE(ipage);
 	err = fiemap_fill_next_extent(
 			fieinfo, start, byteaddr, ilen, 0, flags);
-	trace_f2fs_fiemap(inode, start, byteaddr, ilen, flags, err);
+	trace_f2fs_fiemap(inode, start, byteaddr, ilen, 0, flags, err);

out:

	f2fs_put_page(ipage, 1);
	return err;

include/trace/events/f2fs.h +7 -4

index 7ed0fc430dc6..63706eb3a5db 100644
@@ -2276,9 +2276,10 @@ TRACE_EVENT(f2fs_bmap,
 TRACE_EVENT(f2fs_fiemap,
 
 	TP_PROTO(struct inode *inode, sector_t lblock, sector_t pblock,
-		unsigned long long len, unsigned int flags, int ret),
+		unsigned long long len, unsigned long long phys_len,
+		unsigned int flags, int ret),
 
-	TP_ARGS(inode, lblock, pblock, len, flags, ret),
+	TP_ARGS(inode, lblock, pblock, len, phys_len, flags, ret),
 
 	TP_STRUCT__entry(
 		__field(dev_t, dev)

@@ -2286,6 +2287,7 @@ TRACE_EVENT(f2fs_fiemap,
 		__field(sector_t, lblock)
 		__field(sector_t, pblock)
 		__field(unsigned long long, len)
+		__field(unsigned long long, phys_len)
 		__field(unsigned int, flags)
 		__field(int, ret)
 	),

@@ -2296,16 +2298,18 @@ TRACE_EVENT(f2fs_fiemap,
 		__entry->lblock		= lblock;
 		__entry->pblock		= pblock;
 		__entry->len		= len;
+		__entry->phys_len	= phys_len;
 		__entry->flags		= flags;
 		__entry->ret		= ret;
 	),
 
 	TP_printk("dev = (%d,%d), ino = %lu, lblock:%lld, pblock:%lld, "
-		"len:%llu, flags:%u, ret:%d",
+		"len:%llu, plen:%llu, flags:%u, ret:%d",
 		show_dev_ino(__entry),
 		(unsigned long long)__entry->lblock,
 		(unsigned long long)__entry->pblock,
 		__entry->len,
+		__entry->phys_len,
 		__entry->flags,
 		__entry->ret)
 );
--

2.43.0

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
 fs/ocfs2/extent_map.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

fs/ocfs2/extent_map.c +3 -1

index eabdf97cd685..229ea45df37b 100644
@@ -705,7 +705,9 @@ static int ocfs2_fiemap_inline(struct inode *inode, struct buffer_head *di_bh,
 	unsigned int id_count;
 	struct ocfs2_dinode *di;
 	u64 phys;
-	u32 flags = FIEMAP_EXTENT_DATA_INLINE|FIEMAP_EXTENT_LAST;
+	u32 flags = (FIEMAP_EXTENT_DATA_INLINE|
+		     FIEMAP_EXTENT_HAS_PHYS_LEN|
+		     FIEMAP_EXTENT_LAST);
 	struct ocfs2_inode_info *oi = OCFS2_I(inode);
 
 	di = (struct ocfs2_dinode *)di_bh->b_data;

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
 fs/bcachefs/fs.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

fs/bcachefs/fs.c +6 -4

index f830578a9cd1..d2793bae842d 100644
@@ -913,15 +913,17 @@ static int bch2_fill_extent(struct bch_fs *c,
 			flags |= FIEMAP_EXTENT_SHARED;
 
 		bkey_for_each_ptr_decode(k.k, ptrs, p, entry) {
-			int flags2 = 0;
+			int flags2 = FIEMAP_EXTENT_HAS_PHYS_LEN;
+			u64 phys_len = k.k->size << 9;
 			u64 offset = p.ptr.offset;
 
 			if (p.ptr.unwritten)
 				flags2 |= FIEMAP_EXTENT_UNWRITTEN;
 
-			if (p.crc.compression_type)
+			if (p.crc.compression_type) {
 				flags2 |= FIEMAP_EXTENT_ENCODED;
-			else
+				phys_len = p.crc.compressed_size << 9;
+			} else
 				offset += p.crc.offset;
 
 			if ((offset & (block_sectors(c) - 1)) ||

@@ -931,7 +933,7 @@ static int bch2_fill_extent(struct bch_fs *c,
 			ret = fiemap_fill_next_extent(info,
 						bkey_start_offset(k.k) << 9,
 						offset << 9,
-						k.k->size << 9, 0,
+						k.k->size << 9, phys_len,
 						flags|flags2);
 			if (ret)
 				return ret;

fs/bcachefs/fs.c +4 -1

 					       flags|
+					       FIEMAP_EXTENT_HAS_PHYS_LEN|
 					       FIEMAP_EXTENT_DATA_INLINE);
 	} else if (k.k->type == KEY_TYPE_reservation) {
 		return fiemap_fill_next_extent(info,
 					       bkey_start_offset(k.k) << 9,
-					       0, k.k->size << 9, 0,
+					       0, k.k->size << 9,
+					       k.k->size << 9,
 					       flags|
+					       FIEMAP_EXTENT_HAS_PHYS_LEN|
 					       FIEMAP_EXTENT_DELALLOC|
 					       FIEMAP_EXTENT_UNWRITTEN);
 	} else {

Signature · 1 line

2.43.0

Quoted 41 lines · Sweet Tea Dorminy

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
 fs/bcachefs/fs.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

fs/bcachefs/fs.c +6 -4

index f830578a9cd1..d2793bae842d 100644
@@ -913,15 +913,17 @@ static int bch2_fill_extent(struct bch_fs *c,
 			flags |= FIEMAP_EXTENT_SHARED;
 
 		bkey_for_each_ptr_decode(k.k, ptrs, p, entry) {
-			int flags2 = 0;
+			int flags2 = FIEMAP_EXTENT_HAS_PHYS_LEN;
+			u64 phys_len = k.k->size << 9;
 			u64 offset = p.ptr.offset;
 
 			if (p.ptr.unwritten)
 				flags2 |= FIEMAP_EXTENT_UNWRITTEN;
 
-			if (p.crc.compression_type)
+			if (p.crc.compression_type) {
 				flags2 |= FIEMAP_EXTENT_ENCODED;
-			else
+				phys_len = p.crc.compressed_size << 9;
+			} else
 				offset += p.crc.offset;
 
 			if ((offset & (block_sectors(c) - 1)) ||

@@ -931,7 +933,7 @@ static int bch2_fill_extent(struct bch_fs *c,
 			ret = fiemap_fill_next_extent(info,
 						bkey_start_offset(k.k) << 9,
 						offset << 9,
-						k.k->size << 9, 0,
+						k.k->size << 9, phys_len,
 						flags|flags2);
 			if (ret)
 				return ret;

@@ -941,14 +943,18 @@ static int bch2_fill_extent(struct bch_fs *c,
 	} else if (bkey_extent_is_inline_data(k.k)) {

fs/bcachefs/fs.c static int bch2_fill_extent(struct bch_fs *c, · line 947

Quoted 5 lines

fs/bcachefs/fs.c +2 -1

 		return fiemap_fill_next_extent(info,
 					       bkey_start_offset(k.k) << 9,
-					       0, k.k->size << 9, 0,
+					       0, k.k->size << 9,
+					       bkey_inline_data_bytes(k.k),

Question for Kent perhaps, but what's the functional difference between bkey_inline_data_bytes() and k->size in this particular case?

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
fs/bcachefs/fs.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

fs/bcachefs/fs.c +0 -0

index d2793bae842d..54f613f977b4 100644

Quoted 11 lines · Sweet Tea Dorminy

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
fs/bcachefs/fs.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

fs/bcachefs/fs.c +0 -0

index d2793bae842d..54f613f977b4 100644

fs/bcachefs/fs.c static int bch2_fill_extent(struct bch_fs *c, · line 924

Quoted 5 lines

fs/bcachefs/fs.c +1 -1

@@ -921,7 +921,7 @@ static int bch2_fill_extent(struct bch_fs *c,
 				flags2 |= FIEMAP_EXTENT_UNWRITTEN;
 
 			if (p.crc.compression_type) {
-				flags2 |= FIEMAP_EXTENT_ENCODED;
+				flags2 |= FIEMAP_EXTENT_DATA_COMPRESSED;

(defect) This should *also* set FIEMAP_EXTENT_ENCODED in this case, along with FIEMAP_EXTENT_DATA_COMPRESSED. Both for compatibility with older code that doesn't understand FIEMAP_EXTENT_DATA_COMPRESSED, and because the data still cannot be read directly from the volume when it is not mounted.

Probably Kent should chime in here with what needs to be done to set the phys_len properly for bcachefs, or leave this patch out of your series and let him submit it directly. With proposed wrapper in the first patch of the series there isn't a hard requirement to change all of the filesystems in one shot.

Quoted 11 lines · Sweet Tea Dorminy

Signed-off-by Sweet Tea Dorminy <[email protected]>

---
fs/bcachefs/fs.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

fs/bcachefs/fs.c +0 -0

index d2793bae842d..54f613f977b4 100644

fs/bcachefs/fs.c static int bch2_fill_extent(struct bch_fs *c, · line 924

Quoted 5 lines

fs/bcachefs/fs.c +1 -1

@@ -921,7 +921,7 @@ static int bch2_fill_extent(struct bch_fs *c,
 				flags2 |= FIEMAP_EXTENT_UNWRITTEN;
 
 			if (p.crc.compression_type) {
-				flags2 |= FIEMAP_EXTENT_ENCODED;
+				flags2 |= FIEMAP_EXTENT_DATA_COMPRESSED;

(defect) This should *also* set FIEMAP_EXTENT_ENCODED in this case, along with FIEMAP_EXTENT_DATA_COMPRESSED. Both for compatibility with older code that doesn't understand FIEMAP_EXTENT_DATA_COMPRESSED, and because the data still cannot be read directly from the volume when it is not mounted.

Probably Kent should chime in here with what needs to be done to set the phys_len properly for bcachefs, or leave this patch out of your series and let him submit it directly. With proposed wrapper in the first patch of the series there isn't a hard requirement to change all of the filesystems in one shot.

For many years, various btrfs users have written programs to discover the actual disk space used by files, using root-only interfaces. However, this information is a great fit for fiemap: it is inherently tied to extent information, all filesystems can use it, and the capabilities required for FIEMAP make sense for this additional information also.

Hence, this patchset adds various additional information to fiemap, and extends filesystems (but not iomap) to return it. This uses some of the reserved padding in the fiemap extent structure, so programs unaware of the changes will be unaffected.

This is based on next-20240403. I've tested the btrfs part of this with the standard btrfs testing matrix locally and manually, and done minimal testing of the non-btrfs parts.

I'm unsure whether btrfs should be returning the entire physical extent referenced by a particular logical range, or just the part of the physical extent referenced by that range. The v2 thread has a discussion of this.

I'm not sure why here iomap was excluded technically or I'm missing some previous comments?

Could you also make iomap support new FIEMAP physical extent information? since compressed EROFS uses iomap FIEMAP interface to report compressed extents ("z_erofs_iomap_report_ops") but there is no way to return correct compressed lengths, that is unexpected.

Quoted 6 lines

I'm not sure why here iomap was excluded technically or I'm missing some previous comments?

Quoted 5 lines

Could you also make iomap support new FIEMAP physical extent information? since compressed EROFS uses iomap FIEMAP interface to report compressed extents ("z_erofs_iomap_report_ops") but there is no way to return correct compressed lengths, that is unexpected.

I'll add iomap support in v4, I'd skipped it since I was worried it'd be an expansive additional part not necessary initially. Thank you for noting it!

Sweet Tea

For many years, various btrfs users have written programs to discover the actual disk space used by files, using root-only interfaces. However, this information is a great fit for fiemap: it is inherently tied to extent information, all filesystems can use it, and the capabilities required for FIEMAP make sense for this additional information also.

Hence, this patchset adds various additional information to fiemap, and extends filesystems (but not iomap) to return it. This uses some of the reserved padding in the fiemap extent structure, so programs unaware of the changes will be unaffected.

This is based on next-20240403. I've tested the btrfs part of this with the standard btrfs testing matrix locally and manually, and done minimal testing of the non-btrfs parts.

I'm unsure whether btrfs should be returning the entire physical extent referenced by a particular logical range, or just the part of the physical extent referenced by that range. The v2 thread has a discussion of this.

Changelog:

v3:
- Adapted all the direct users of fiemap, except iomap, to emit the new fiemap information, as far as I understand the other filesystems.

v2:
- Adopted PHYS_LEN flag and COMPRESSED flag from the previous version,
as per Andreas Dilger' comment.
https://patchwork.ozlabs.org/project/linux-ext4/patch/4f8d5dc5b51a43efaf16c39398c23a6276e40a30.1386778303.git.dsterba@suse.cz/
- https://lore.kernel.org/linux-fsdevel/[email protected]/T/#t

v1: https://lore.kernel.org/linux-fsdevel/20240315030334.GQ6184@frogsfrogsfrogs/T/#t

Sweet Tea Dorminy (13):
fs: fiemap: add physical_length field to extents
fs: fiemap: update fiemap_fill_next_extent() signature
fs: fiemap: add new COMPRESSED extent state
btrfs: fiemap: emit new COMPRESSED state.
btrfs: fiemap: return extent physical size
nilfs2: fiemap: return correct extent physical length
ext4: fiemap: return correct extent physical length
f2fs: fiemap: add physical length to trace_f2fs_fiemap
f2fs: fiemap: return correct extent physical length
ocfs2: fiemap: return correct extent physical length
bcachefs: fiemap: return correct extent physical length
f2fs: fiemap: emit new COMPRESSED state
bcachefs: fiemap: emit new COMPRESSED state

 Documentation/filesystems/fiemap.rst | 35 ++++++++++----
 fs/bcachefs/fs.c                     | 17 +++++--
 fs/btrfs/extent_io.c                 | 72 ++++++++++++++++++----------
 fs/ext4/extents.c                    |  3 +-
 fs/f2fs/data.c                       | 36 +++++++++-----
 fs/f2fs/inline.c                     |  7 +--
 fs/ioctl.c                           | 11 +++--
 fs/iomap/fiemap.c                    |  2 +-
 fs/nilfs2/inode.c                    | 18 ++++---
 fs/ntfs3/frecord.c                   |  7 +--
 fs/ocfs2/extent_map.c                | 10 ++--
 fs/smb/client/smb2ops.c              |  1 +
 include/linux/fiemap.h               |  2 +-
 include/trace/events/f2fs.h          | 10 ++--
 include/uapi/linux/fiemap.h          | 34 ++++++++++---
 15 files changed, 178 insertions(+), 87 deletions(-)

base-commit: 75e31f66adc4c8d049e8aac1f079c1639294cd65

Signature · 1 line

2.43.0

Quoted 16 lines · Sweet Tea Dorminy

For many years, various btrfs users have written programs to discover the actual disk space used by files, using root-only interfaces. However, this information is a great fit for fiemap: it is inherently tied to extent information, all filesystems can use it, and the capabilities required for FIEMAP make sense for this additional information also.

Hence, this patchset adds various additional information to fiemap, and extends filesystems (but not iomap) to return it. This uses some of the reserved padding in the fiemap extent structure, so programs unaware of the changes will be unaffected.

This is based on next-20240403. I've tested the btrfs part of this with the standard btrfs testing matrix locally and manually, and done minimal testing of the non-btrfs parts.

Quoted 4 lines

I'm unsure whether btrfs should be returning the entire physical extent referenced by a particular logical range, or just the part of the physical extent referenced by that range. The v2 thread has a discussion of this.

I believe there was some talk of using the padding for a device ID, so that fiemap could properly support multi device filesystems. Are we sure this is the best use of those bytes?

Quoted 53 lines

Changelog:

v3:
- Adapted all the direct users of fiemap, except iomap, to emit the new fiemap information, as far as I understand the other filesystems.

v2:
- Adopted PHYS_LEN flag and COMPRESSED flag from the previous version,
as per Andreas Dilger' comment.
https://patchwork.ozlabs.org/project/linux-ext4/patch/4f8d5dc5b51a43efaf16c39398c23a6276e40a30.1386778303.git.dsterba@suse.cz/
- https://lore.kernel.org/linux-fsdevel/[email protected]/T/#t

v1: https://lore.kernel.org/linux-fsdevel/20240315030334.GQ6184@frogsfrogsfrogs/T/#t

Sweet Tea Dorminy (13):
fs: fiemap: add physical_length field to extents
fs: fiemap: update fiemap_fill_next_extent() signature
fs: fiemap: add new COMPRESSED extent state
btrfs: fiemap: emit new COMPRESSED state.
btrfs: fiemap: return extent physical size
nilfs2: fiemap: return correct extent physical length
ext4: fiemap: return correct extent physical length
f2fs: fiemap: add physical length to trace_f2fs_fiemap
f2fs: fiemap: return correct extent physical length
ocfs2: fiemap: return correct extent physical length
bcachefs: fiemap: return correct extent physical length
f2fs: fiemap: emit new COMPRESSED state
bcachefs: fiemap: emit new COMPRESSED state

 Documentation/filesystems/fiemap.rst | 35 ++++++++++----
 fs/bcachefs/fs.c                     | 17 +++++--
 fs/btrfs/extent_io.c                 | 72 ++++++++++++++++++----------
 fs/ext4/extents.c                    |  3 +-
 fs/f2fs/data.c                       | 36 +++++++++-----
 fs/f2fs/inline.c                     |  7 +--
 fs/ioctl.c                           | 11 +++--
 fs/iomap/fiemap.c                    |  2 +-
 fs/nilfs2/inode.c                    | 18 ++++---
 fs/ntfs3/frecord.c                   |  7 +--
 fs/ocfs2/extent_map.c                | 10 ++--
 fs/smb/client/smb2ops.c              |  1 +
 include/linux/fiemap.h               |  2 +-
 include/trace/events/f2fs.h          | 10 ++--
 include/uapi/linux/fiemap.h          | 34 ++++++++++---
 15 files changed, 178 insertions(+), 87 deletions(-)

base-commit: 75e31f66adc4c8d049e8aac1f079c1639294cd65

Signature · 1 line

2.43.0

Quoted 16 lines · Sweet Tea Dorminy

For many years, various btrfs users have written programs to discover the actual disk space used by files, using root-only interfaces. However, this information is a great fit for fiemap: it is inherently tied to extent information, all filesystems can use it, and the capabilities required for FIEMAP make sense for this additional information also.

Hence, this patchset adds various additional information to fiemap, and extends filesystems (but not iomap) to return it. This uses some of the reserved padding in the fiemap extent structure, so programs unaware of the changes will be unaffected.

This is based on next-20240403. I've tested the btrfs part of this with the standard btrfs testing matrix locally and manually, and done minimal testing of the non-btrfs parts.

Quoted 4 lines

I'm unsure whether btrfs should be returning the entire physical extent referenced by a particular logical range, or just the part of the physical extent referenced by that range. The v2 thread has a discussion of this.

I believe there was some talk of using the padding for a device ID, so that fiemap could properly support multi device filesystems. Are we sure this is the best use of those bytes?