Skip to contentSkip to conversation

erofs: add fiemap support with iomap

PATCH In review Gao Xiang 2 participants 9 messages Aug 13, 2021 – Aug 18, 2021 EROFS file system

lore
Share & tools
lore

Gao Xiang

author
#0 lore raw

cover erofs: add fiemap support with iomap

Hi folks,

This patchset mainly addresses fiemap support for EROFS. After this patchset, end users can observe (un)compressed extent distribution by hand.

The first patch is also useful for later LZMA support in order to decompress full LZMA extents if needed (according to specific strategy.)

Btw, the current development status for LZMA is in the following branches (yet these are not aimed for the next merge window since it's still some work to do):
https://git.kernel.org/pub/scm/linux/kernel/git/xiang/linux.git/log/?h=erofs/lzma
https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/log/?h=experimental-lzma

Thanks,
Gao Xiang

Gao Xiang (2):
erofs: add support for the full decompressed length
erofs: add fiemap support with iomap

 fs/erofs/data.c     |  15 ++++-
 fs/erofs/inode.c    |   1 +
 fs/erofs/internal.h |  10 ++++
 fs/erofs/namei.c    |   1 +
 fs/erofs/zmap.c     | 131 +++++++++++++++++++++++++++++++++++++++++---
 5 files changed, 149 insertions(+), 9 deletions(-)
Signature · 1 line
2.24.4
author
#1 lore raw

PATCH 1/2 erofs: add support for the full decompressed length

Previously, there is no need to get the full decompressed length since EROFS supports partial decompression. However for some other cases such as fiemap, the full decompressed length is necessary for iomap to make it work properly.

This patch adds a way to get the full decompressed length. Note that it takes more metadata overhead and it'd be avoided if possible in the performance sensitive scenario.

---
 fs/erofs/internal.h |  5 +++
 fs/erofs/zmap.c     | 93 +++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 90 insertions(+), 8 deletions(-)
fs/erofs/internal.h +5 -0
index 25b094085ca6..2a05b09e1c06 100644
@@ -356,6 +356,11 @@ struct erofs_map_blocks {
 
 /* Flags used by erofs_map_blocks_flatmode() */
 #define EROFS_GET_BLOCKS_RAW    0x0001
+/*
+ * Used to get the exact decompressed length, e.g. fiemap (consider lookback
+ * approach instead if possible since it's quite metadata expensive.)
+ */
+#define EROFS_GET_BLOCKS_FIEMAP	0x0002
 
 /* zmap.c */
 #ifdef CONFIG_EROFS_FS_ZIP
fs/erofs/zmap.c +85 -8

158 lines not on this page · raw message

Signature · 1 line
2.24.4
author
#2 lore raw

PATCH 1/2 erofs: add support for the full decompressed length

Previously, there is no need to get the full decompressed length since EROFS supports partial decompression. However for some other cases such as fiemap, the full decompressed length is necessary for iomap to make it work properly.

This patch adds a way to get the full decompressed length. Note that it takes more metadata overhead and it'd be avoided if possible in the performance sensitive scenario.

---
changes since v1:
- adjust some logic in get_compacted_la_distance() to avoid warning reported by 0day CI (even though it cannot really happen):
fs/erofs/zmap.c:233 get_compacted_la_distance() error: uninitialized symbol 'lo'.

 fs/erofs/internal.h |  5 +++
 fs/erofs/zmap.c     | 95 +++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 92 insertions(+), 8 deletions(-)
fs/erofs/internal.h +5 -0
index 25b094085ca6..2a05b09e1c06 100644
@@ -356,6 +356,11 @@ struct erofs_map_blocks {
 
 /* Flags used by erofs_map_blocks_flatmode() */
 #define EROFS_GET_BLOCKS_RAW    0x0001
+/*
+ * Used to get the exact decompressed length, e.g. fiemap (consider lookback
+ * approach instead if possible since it's more metadata lightweight.)
+ */
+#define EROFS_GET_BLOCKS_FIEMAP	0x0002
 
 /* zmap.c */
 #ifdef CONFIG_EROFS_FS_ZIP
fs/erofs/zmap.c +87 -8

160 lines not on this page · raw message

Signature · 1 line
2.24.4
maintainer Reviewed-by on v1 1/2
#3 lore raw

Re: [PATCH v1.1 1/2] erofs: add support for the full decompressed length

Quoted 10 lines · Gao Xiang

Previously, there is no need to get the full decompressed length since EROFS supports partial decompression. However for some other cases such as fiemap, the full decompressed length is necessary for iomap to make it work properly.

This patch adds a way to get the full decompressed length. Note that it takes more metadata overhead and it'd be avoided if possible in the performance sensitive scenario.

Thanks,

author
#4 lore raw

PATCH 1/1 erofs: add support for the full decompressed length

In reply to Gao Xiang

Previously, there is no need to get the full decompressed length since EROFS supports partial decompression. However for some other cases such as fiemap, the full decompressed length is necessary for iomap to make it work properly.

This patch adds a way to get the full decompressed length. Note that it takes more metadata overhead and it'd be avoided if possible in the performance sensitive scenario.

---
changes since v1.1:
- simplify a DBG_BUGON() suggested by Chao.

 fs/erofs/internal.h |  5 +++
 fs/erofs/zmap.c     | 95 +++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 92 insertions(+), 8 deletions(-)
fs/erofs/internal.h +5 -0
index 25b094085ca6..a9fc5245015b 100644
@@ -356,6 +356,11 @@ struct erofs_map_blocks {
 
 /* Flags used by erofs_map_blocks_flatmode() */
 #define EROFS_GET_BLOCKS_RAW    0x0001
+/*
+ * Used to get the exact decompressed length, e.g. fiemap (consider lookback
+ * approach instead if possible since it's more metadata lightweight.)
+ */
+#define EROFS_GET_BLOCKS_FIEMAP	0x0002
 
 /* zmap.c */
 #ifdef CONFIG_EROFS_FS_ZIP
fs/erofs/zmap.c +87 -8

160 lines not on this page · raw message

Signature · 1 line
2.24.4
maintainer
#5 lore raw

Re: [PATCH 1/2] erofs: add support for the full decompressed length

In reply to Gao Xiang
Quoted 38 lines · Gao Xiang

Previously, there is no need to get the full decompressed length since EROFS supports partial decompression. However for some other cases such as fiemap, the full decompressed length is necessary for iomap to make it work properly.

This patch adds a way to get the full decompressed length. Note that it takes more metadata overhead and it'd be avoided if possible in the performance sensitive scenario.

---
  fs/erofs/internal.h |  5 +++
  fs/erofs/zmap.c     | 93 +++++++++++++++++++++++++++++++++++++++++----
  2 files changed, 90 insertions(+), 8 deletions(-)
fs/erofs/internal.h +5 -0
index 25b094085ca6..2a05b09e1c06 100644
@@ -356,6 +356,11 @@ struct erofs_map_blocks {
 
  /* Flags used by erofs_map_blocks_flatmode() */
  #define EROFS_GET_BLOCKS_RAW    0x0001
+/*
+ * Used to get the exact decompressed length, e.g. fiemap (consider lookback
+ * approach instead if possible since it's quite metadata expensive.)
+ */
+#define EROFS_GET_BLOCKS_FIEMAP	0x0002
 
  /* zmap.c */
  #ifdef CONFIG_EROFS_FS_ZIP
fs/erofs/zmap.c +0 -0
index f68aea4baed7..12256ef12819 100644
@@ -212,9 +212,32 @@ static unsigned int decode_compactedbits(unsigned int lobits,
  	return lo;
  }

fs/erofs/zmap.c static unsigned int decode_compactedbits(unsigned int lobits, · line 223

Quoted 10 lines
fs/erofs/zmap.c +9 -0
 
+static int get_compacted_la_distance(unsigned int lclusterbits,
+				     unsigned int encodebits,
+				     unsigned int vcnt, u8 *in, int i)
+{
+	const unsigned int lomask = (1 << lclusterbits) - 1;
+	unsigned int lo, d1 = 0;
+	u8 type;
+
+	for (; i < vcnt; ++i) {

for (di = 0; i < vcnt; ++i, ++d1)

fs/erofs/zmap.c static unsigned int decode_compactedbits(unsigned int lobits, · line 228

Quoted 5 lines
fs/erofs/zmap.c +5 -0
+		lo = decode_compactedbits(lclusterbits, lomask,
+					  in, encodebits * i, &type);
+
+		if (type != Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD)
+			return d1;

[1]

Quoted 96 lines
fs/erofs/zmap.c +35 -7
+		++d1;
+	}
+
+	/* vcnt - 1 (Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD) item */
+	if (!(lo & Z_EROFS_VLE_DI_D0_CBLKCNT))
+		d1 += lo - 1;
+	return d1;
+}
+
  static int unpack_compacted_index(struct z_erofs_maprecorder *m,
  				  unsigned int amortizedshift,
-				  unsigned int eofs)
+				  unsigned int eofs, bool lookahead)
  {
  	struct erofs_inode *const vi = EROFS_I(m->inode);
  	const unsigned int lclusterbits = vi->z_logical_clusterbits;
@@ -243,6 +266,11 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
  	m->type = type;
  	if (type == Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD) {
  		m->clusterofs = 1 << lclusterbits;
+
+		/* figure out lookahead_distance: delta[1] if needed */
+		if (lookahead)
+			m->delta[1] = get_compacted_la_distance(lclusterbits,
+						encodebits, vcnt, in, i);
  		if (lo & Z_EROFS_VLE_DI_D0_CBLKCNT) {
  			if (!big_pcluster) {
  				DBG_BUGON(1);
@@ -313,7 +341,7 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
  }
 
  static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,
-					    unsigned long lcn)
+					    unsigned long lcn, bool lookahead)
  {
  	struct inode *const inode = m->inode;
  	struct erofs_inode *const vi = EROFS_I(inode);
@@ -364,11 +392,12 @@ static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,
  	err = z_erofs_reload_indexes(m, erofs_blknr(pos));
  	if (err)
  		return err;
-	return unpack_compacted_index(m, amortizedshift, erofs_blkoff(pos));
+	return unpack_compacted_index(m, amortizedshift, erofs_blkoff(pos),
+				      lookahead);
  }
 
  static int z_erofs_load_cluster_from_disk(struct z_erofs_maprecorder *m,
-					  unsigned int lcn)
+					  unsigned int lcn, bool lookahead)
  {
  	const unsigned int datamode = EROFS_I(m->inode)->datalayout;
 
@@ -376,7 +405,7 @@ static int z_erofs_load_cluster_from_disk(struct z_erofs_maprecorder *m,
  		return legacy_load_cluster_from_disk(m, lcn);
 
  	if (datamode == EROFS_INODE_FLAT_COMPRESSION)
-		return compacted_load_cluster_from_disk(m, lcn);
+		return compacted_load_cluster_from_disk(m, lcn, lookahead);
 
  	return -EINVAL;
  }
@@ -399,7 +428,7 @@ static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m,
 
  	/* load extent head logical cluster if needed */
  	lcn -= lookback_distance;
-	err = z_erofs_load_cluster_from_disk(m, lcn);
+	err = z_erofs_load_cluster_from_disk(m, lcn, false);
  	if (err)
  		return err;
 
@@ -450,7 +479,7 @@ static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m,
  	if (m->compressedlcs)
  		goto out;
 
-	err = z_erofs_load_cluster_from_disk(m, lcn);
+	err = z_erofs_load_cluster_from_disk(m, lcn, false);
  	if (err)
  		return err;
 
@@ -498,6 +527,48 @@ static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m,
  	return -EFSCORRUPTED;
  }
 
+static int z_erofs_get_extent_decompressedlen(struct z_erofs_maprecorder *m)
+{
+	struct inode *inode = m->inode;
+	struct erofs_inode *vi = EROFS_I(inode);
+	struct erofs_map_blocks *map = m->map;
+	unsigned int lclusterbits = vi->z_logical_clusterbits;
+	u64 lcn = m->lcn, headlcn = map->m_la >> lclusterbits;
+	int err;
+
+	do {
+		/* handle the last EOF pcluster (no next HEAD lcluster) */
+		if ((lcn << lclusterbits) >= inode->i_size) {
+			map->m_llen = inode->i_size - map->m_la;

fs/erofs/zmap.c static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m, · line 552

Quoted 10 lines
fs/erofs/zmap.c +10 -0
+			return 0;
+		}
+
+		err = z_erofs_load_cluster_from_disk(m, lcn, true);
+		if (err)
+			return err;
+
+		if (m->type == Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD) {
+			if (!m->delta[1])
+				DBG_BUGON(m->clusterofs != 1 << lclusterbits);
			DBG_BUGON(!m->delta[1] &&
				m->clusterofs != 1 << lclusterbits);

fs/erofs/zmap.c static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m, · line 558

Quoted 6 lines
fs/erofs/zmap.c +6 -0
+		} else if (m->type == Z_EROFS_VLE_CLUSTER_TYPE_PLAIN ||
+			   m->type == Z_EROFS_VLE_CLUSTER_TYPE_HEAD) {
+			/* go on until the next HEAD lcluster */
+			if (lcn != headlcn)
+				break;
+			m->delta[1] = 1;

If I didn't miss anything, return value [1] of get_compacted_la_distance() won't be used anyway here? right?

Thanks,

Quoted 39 lines
fs/erofs/zmap.c +20 -1
+		} else {
+			erofs_err(inode->i_sb, "unknown type %u @ lcn %llu of nid %llu",
+				  m->type, lcn, vi->nid);
+			DBG_BUGON(1);
+			return -EOPNOTSUPP;
+		}
+		lcn += m->delta[1];
+	} while (m->delta[1]);
+
+	map->m_llen = (lcn << lclusterbits) + m->clusterofs - map->m_la;
+	return 0;
+}
+
  int z_erofs_map_blocks_iter(struct inode *inode,
  			    struct erofs_map_blocks *map,
  			    int flags)
@@ -531,7 +602,7 @@ int z_erofs_map_blocks_iter(struct inode *inode,
  	initial_lcn = ofs >> lclusterbits;
  	endoff = ofs & ((1 << lclusterbits) - 1);
 
-	err = z_erofs_load_cluster_from_disk(&m, initial_lcn);
+	err = z_erofs_load_cluster_from_disk(&m, initial_lcn, false);
  	if (err)
  		goto unmap_out;
 
@@ -581,6 +652,12 @@ int z_erofs_map_blocks_iter(struct inode *inode,
  	err = z_erofs_get_extent_compressedlen(&m, initial_lcn);
  	if (err)
  		goto out;
+
+	if (flags & EROFS_GET_BLOCKS_FIEMAP) {
+		err = z_erofs_get_extent_decompressedlen(&m);
+		if (!err)
+			map->m_flags |= EROFS_MAP_FULL_MAPPED;
+	}
  unmap_out:
  	if (m.kaddr)
  		kunmap_atomic(m.kaddr);
author
#6 lore raw

Re: [PATCH 1/2] erofs: add support for the full decompressed lengthy

Quoted 40 lines · Chao Yu
Quoted 39 lines · Gao Xiang

Previously, there is no need to get the full decompressed length since EROFS supports partial decompression. However for some other cases such as fiemap, the full decompressed length is necessary for iomap to make it work properly.

This patch adds a way to get the full decompressed length. Note that it takes more metadata overhead and it'd be avoided if possible in the performance sensitive scenario.

---
  fs/erofs/internal.h |  5 +++
  fs/erofs/zmap.c     | 93 +++++++++++++++++++++++++++++++++++++++++----
  2 files changed, 90 insertions(+), 8 deletions(-)
fs/erofs/internal.h +5 -0
index 25b094085ca6..2a05b09e1c06 100644
@@ -356,6 +356,11 @@ struct erofs_map_blocks {
  /* Flags used by erofs_map_blocks_flatmode() */
  #define EROFS_GET_BLOCKS_RAW    0x0001
+/*
+ * Used to get the exact decompressed length, e.g. fiemap (consider lookback
+ * approach instead if possible since it's quite metadata expensive.)
+ */
+#define EROFS_GET_BLOCKS_FIEMAP	0x0002
  /* zmap.c */
  #ifdef CONFIG_EROFS_FS_ZIP
fs/erofs/zmap.c +3 -0
index f68aea4baed7..12256ef12819 100644
@@ -212,9 +212,32 @@ static unsigned int decode_compactedbits(unsigned int lobits,
  	return lo;
  }
+static int get_compacted_la_distance(unsigned int lclusterbits,
+				     unsigned int encodebits,
+				     unsigned int vcnt, u8 *in, int i)
Quoted 8 lines

fs/erofs/zmap.c static unsigned int decode_compactedbits(unsigned int lobits, · line 223

Quoted 6 lines
fs/erofs/zmap.c +6 -0
+{
+	const unsigned int lomask = (1 << lclusterbits) - 1;
+	unsigned int lo, d1 = 0;
+	u8 type;
+
+	for (; i < vcnt; ++i) {

for (di = 0; i < vcnt; ++i, ++d1)

Hi Chao,

Thanks for the review

this could cause lo potential uninitialized warning (actually it's unpossible). So I resent [PATCH v1.1] as

https://lore.kernel.org/linux-erofs/[email protected]/

Please kindly help check out...

Quoted 100 lines

fs/erofs/zmap.c static unsigned int decode_compactedbits(unsigned int lobits, · line 228

Quoted 6 lines
fs/erofs/zmap.c +5 -0
+		lo = decode_compactedbits(lclusterbits, lomask,
+					  in, encodebits * i, &type);
+
+		if (type != Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD)
+			return d1;

[1]

Quoted 91 lines
fs/erofs/zmap.c +40 -7
+		++d1;
+	}
+
+	/* vcnt - 1 (Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD) item */
+	if (!(lo & Z_EROFS_VLE_DI_D0_CBLKCNT))
+		d1 += lo - 1;
+	return d1;
+}
+
  static int unpack_compacted_index(struct z_erofs_maprecorder *m,
  				  unsigned int amortizedshift,
-				  unsigned int eofs)
+				  unsigned int eofs, bool lookahead)
  {
  	struct erofs_inode *const vi = EROFS_I(m->inode);
  	const unsigned int lclusterbits = vi->z_logical_clusterbits;
@@ -243,6 +266,11 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
  	m->type = type;
  	if (type == Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD) {
  		m->clusterofs = 1 << lclusterbits;
+
+		/* figure out lookahead_distance: delta[1] if needed */
+		if (lookahead)
+			m->delta[1] = get_compacted_la_distance(lclusterbits,
+						encodebits, vcnt, in, i);
  		if (lo & Z_EROFS_VLE_DI_D0_CBLKCNT) {
  			if (!big_pcluster) {
  				DBG_BUGON(1);
@@ -313,7 +341,7 @@ static int unpack_compacted_index(struct z_erofs_maprecorder *m,
  }
  static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,
-					    unsigned long lcn)
+					    unsigned long lcn, bool lookahead)
  {
  	struct inode *const inode = m->inode;
  	struct erofs_inode *const vi = EROFS_I(inode);
@@ -364,11 +392,12 @@ static int compacted_load_cluster_from_disk(struct z_erofs_maprecorder *m,
  	err = z_erofs_reload_indexes(m, erofs_blknr(pos));
  	if (err)
  		return err;
-	return unpack_compacted_index(m, amortizedshift, erofs_blkoff(pos));
+	return unpack_compacted_index(m, amortizedshift, erofs_blkoff(pos),
+				      lookahead);
  }
  static int z_erofs_load_cluster_from_disk(struct z_erofs_maprecorder *m,
-					  unsigned int lcn)
+					  unsigned int lcn, bool lookahead)
  {
  	const unsigned int datamode = EROFS_I(m->inode)->datalayout;
@@ -376,7 +405,7 @@ static int z_erofs_load_cluster_from_disk(struct z_erofs_maprecorder *m,
  		return legacy_load_cluster_from_disk(m, lcn);
  	if (datamode == EROFS_INODE_FLAT_COMPRESSION)
-		return compacted_load_cluster_from_disk(m, lcn);
+		return compacted_load_cluster_from_disk(m, lcn, lookahead);
  	return -EINVAL;
  }
@@ -399,7 +428,7 @@ static int z_erofs_extent_lookback(struct z_erofs_maprecorder *m,
  	/* load extent head logical cluster if needed */
  	lcn -= lookback_distance;
-	err = z_erofs_load_cluster_from_disk(m, lcn);
+	err = z_erofs_load_cluster_from_disk(m, lcn, false);
  	if (err)
  		return err;
@@ -450,7 +479,7 @@ static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m,
  	if (m->compressedlcs)
  		goto out;
-	err = z_erofs_load_cluster_from_disk(m, lcn);
+	err = z_erofs_load_cluster_from_disk(m, lcn, false);
  	if (err)
  		return err;
@@ -498,6 +527,48 @@ static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m,
  	return -EFSCORRUPTED;
  }
+static int z_erofs_get_extent_decompressedlen(struct z_erofs_maprecorder *m)
+{
+	struct inode *inode = m->inode;
+	struct erofs_inode *vi = EROFS_I(inode);
+	struct erofs_map_blocks *map = m->map;
+	unsigned int lclusterbits = vi->z_logical_clusterbits;
+	u64 lcn = m->lcn, headlcn = map->m_la >> lclusterbits;
+	int err;
+
+	do {
+		/* handle the last EOF pcluster (no next HEAD lcluster) */
+		if ((lcn << lclusterbits) >= inode->i_size) {
+			map->m_llen = inode->i_size - map->m_la;
+			return 0;
+		}
+
+		err = z_erofs_load_cluster_from_disk(m, lcn, true);
+		if (err)
Quoted 9 lines

fs/erofs/zmap.c static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m, · line 552

Quoted 5 lines
fs/erofs/zmap.c +5 -0
+			return err;
+
+		if (m->type == Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD) {
+			if (!m->delta[1])
+				DBG_BUGON(m->clusterofs != 1 << lclusterbits);
			DBG_BUGON(!m->delta[1] &&
				m->clusterofs != 1 << lclusterbits);

Ok, will update.

Quoted 9 lines

fs/erofs/zmap.c static int z_erofs_get_extent_compressedlen(struct z_erofs_maprecorder *m, · line 558

Quoted 6 lines
fs/erofs/zmap.c +6 -0
+		} else if (m->type == Z_EROFS_VLE_CLUSTER_TYPE_PLAIN ||
+			   m->type == Z_EROFS_VLE_CLUSTER_TYPE_HEAD) {
+			/* go on until the next HEAD lcluster */
+			if (lcn != headlcn)
+				break;
+			m->delta[1] = 1;

If I didn't miss anything, return value [1] of get_compacted_la_distance() won't be used anyway here? right?

These are two different things, first, return value [1] is used to calculate length to the next lcluster, e.g.

NONHEAD NONHEAD HEAD

     ^
     m->lcn

so here in the first loop, delta[1] = 2, and m->type == NONHEAD, the second loop, m->type == HEAD, lcn != headlcn, so we are done.

Yet here the logic is instead
if m->lcn == m->headlcn, so m->lcn points to this lcluster HEAD, but we need to find the next lcluster, e.g.

HEAD NONHEAD NONHEAD NONHEAD HEAD

    ^                             ^
    m->lcn, m->headlcn            we need to get this

so we lcn == headlcn, we need to go on and get the next HEAD lcluster one.

It will use since m->lcn could be originally equal to headlcn (point to this HEAD lcluster)

Thanks,
Gao Xiang

Quoted 40 lines

Thanks,

Quoted 37 lines
fs/erofs/zmap.c +20 -1
+		} else {
+			erofs_err(inode->i_sb, "unknown type %u @ lcn %llu of nid %llu",
+				  m->type, lcn, vi->nid);
+			DBG_BUGON(1);
+			return -EOPNOTSUPP;
+		}
+		lcn += m->delta[1];
+	} while (m->delta[1]);
+
+	map->m_llen = (lcn << lclusterbits) + m->clusterofs - map->m_la;
+	return 0;
+}
+
  int z_erofs_map_blocks_iter(struct inode *inode,
  			    struct erofs_map_blocks *map,
  			    int flags)
@@ -531,7 +602,7 @@ int z_erofs_map_blocks_iter(struct inode *inode,
  	initial_lcn = ofs >> lclusterbits;
  	endoff = ofs & ((1 << lclusterbits) - 1);
-	err = z_erofs_load_cluster_from_disk(&m, initial_lcn);
+	err = z_erofs_load_cluster_from_disk(&m, initial_lcn, false);
  	if (err)
  		goto unmap_out;
@@ -581,6 +652,12 @@ int z_erofs_map_blocks_iter(struct inode *inode,
  	err = z_erofs_get_extent_compressedlen(&m, initial_lcn);
  	if (err)
  		goto out;
+
+	if (flags & EROFS_GET_BLOCKS_FIEMAP) {
+		err = z_erofs_get_extent_decompressedlen(&m);
+		if (!err)
+			map->m_flags |= EROFS_MAP_FULL_MAPPED;
+	}
  unmap_out:
  	if (m.kaddr)
  		kunmap_atomic(m.kaddr);
author
#7 lore raw

PATCH 2/2 erofs: add fiemap support with iomap

In reply to Gao Xiang

This adds fiemap support for both uncompressed files and compressed files by using iomap infrastructure.

---
 fs/erofs/data.c     | 15 ++++++++++++++-
 fs/erofs/inode.c    |  1 +
 fs/erofs/internal.h |  5 +++++
 fs/erofs/namei.c    |  1 +
 fs/erofs/zmap.c     | 38 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 59 insertions(+), 1 deletion(-)
fs/erofs/data.c +14 -1
index b2a22aabc9bc..09c46fbdb9b2 100644
@@ -5,7 +5,6 @@
  */
 #include "internal.h"
 #include <linux/prefetch.h>
-#include <linux/iomap.h>
 #include <linux/dax.h>
 #include <trace/events/erofs.h>
 
@@ -152,6 +151,20 @@ static const struct iomap_ops erofs_iomap_ops = {
 	.iomap_end = erofs_iomap_end,
 };
 
+int erofs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
+		 u64 start, u64 len)
+{
+	if (erofs_inode_is_data_compressed(EROFS_I(inode)->datalayout)) {
+#ifdef CONFIG_EROFS_FS_ZIP
+		return iomap_fiemap(inode, fieinfo, start, len,
+				    &z_erofs_iomap_report_ops);
+#else
+		return -EOPNOTSUPP;
+#endif
+	}
+	return iomap_fiemap(inode, fieinfo, start, len, &erofs_iomap_ops);
+}
+
 /*
  * since we dont have write or truncate flows, so no inode
  * locking needs to be held at the moment.
fs/erofs/inode.c +1 -0
index 92728da1d206..d13e0709599c 100644
@@ -365,6 +365,7 @@ const struct inode_operations erofs_generic_iops = {
 	.getattr = erofs_getattr,
 	.listxattr = erofs_listxattr,
 	.get_acl = erofs_get_acl,
+	.fiemap = erofs_fiemap,
 };
 
 const struct inode_operations erofs_symlink_iops = {
fs/erofs/internal.h +5 -0
index 2a05b09e1c06..ae33a28c8669 100644
@@ -15,6 +15,7 @@
 #include <linux/magic.h>
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
+#include <linux/iomap.h>
 #include "erofs_fs.h"
 
 /* redefine pr_fmt "erofs: " */
@@ -363,6 +364,8 @@ struct erofs_map_blocks {
 #define EROFS_GET_BLOCKS_FIEMAP	0x0002
 
 /* zmap.c */
+extern const struct iomap_ops z_erofs_iomap_report_ops;
+
 #ifdef CONFIG_EROFS_FS_ZIP
 int z_erofs_fill_inode(struct inode *inode);
 int z_erofs_map_blocks_iter(struct inode *inode,
@@ -381,6 +384,8 @@ static inline int z_erofs_map_blocks_iter(struct inode *inode,
 /* data.c */
 extern const struct file_operations erofs_file_fops;
 struct page *erofs_get_meta_page(struct super_block *sb, erofs_blk_t blkaddr);
+int erofs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
+		 u64 start, u64 len);
 
 /* inode.c */
 static inline unsigned long erofs_inode_hash(erofs_nid_t nid)
fs/erofs/namei.c +1 -0
index a8271ce5e13f..8629e616028c 100644
@@ -245,4 +245,5 @@ const struct inode_operations erofs_dir_iops = {
 	.getattr = erofs_getattr,
 	.listxattr = erofs_listxattr,
 	.get_acl = erofs_get_acl,
+	.fiemap = erofs_fiemap,
 };
fs/erofs/zmap.c +38 -0
index 12256ef12819..6af31ef6f13f 100644
@@ -673,3 +673,41 @@ int z_erofs_map_blocks_iter(struct inode *inode,
 	DBG_BUGON(err < 0 && err != -ENOMEM);
 	return err;
 }
+
+static int z_erofs_iomap_begin_report(struct inode *inode, loff_t offset,
+				loff_t length, unsigned int flags,
+				struct iomap *iomap, struct iomap *srcmap)
+{
+	int ret;
+	struct erofs_map_blocks map = { .m_la = offset };
+
+	ret = z_erofs_map_blocks_iter(inode, &map, EROFS_GET_BLOCKS_FIEMAP);
+	if (map.mpage)
+		put_page(map.mpage);
+	if (ret < 0)
+		return ret;
+
+	iomap->bdev = inode->i_sb->s_bdev;
+	iomap->offset = map.m_la;
+	iomap->length = map.m_llen;
+	if (map.m_flags & EROFS_MAP_MAPPED) {
+		iomap->type = IOMAP_MAPPED;
+		iomap->addr = map.m_pa;
+	} else {
+		iomap->type = IOMAP_HOLE;
+		iomap->addr = IOMAP_NULL_ADDR;
+		/*
+		 * No strict rule how to describe extents for post EOF, yet
+		 * we need do like below. Otherwise, iomap itself will get
+		 * into an endless loop on post EOF.
+		 */
+		if (iomap->offset >= inode->i_size)
+			iomap->length = length + map.m_la - offset;
+	}
+	iomap->flags = 0;
+	return 0;
+}
+
+const struct iomap_ops z_erofs_iomap_report_ops = {
+	.iomap_begin = z_erofs_iomap_begin_report,
+};
Signature · 1 line
2.24.4