Skip to contentSkip to conversation

btrfs: extent-map: unify the members with btrfs_ordered_extent

PATCH Superseded Qu Wenruo 4 participants 38 messages May 3, 2024 – May 23, 2024

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

v2 cover btrfs: extent-map: unify the members with btrfs_ordered_extent

[CHANGELOG]
v2:
- Rebased to the latest for-next
There is a conflicts with extent locking, and maybe some other hidden conflicts for NOCOW/PREALLOC?
As previously the patchset passes fstests auto group, but after the merging with other patches, it always crashes as btrfs/060.

- Fix an error in the final cleanup patch
It's the NOCOW/PREALLOC shenanigans again, in the buffered NOCOW path, that we have to use the old inaccurate numbers for NOCOW/PREALLOC OEs.

- Split the final cleanup into 4 patches
Most cleanups are very straightforward, but the cleanup for btrfs_alloc_ordered_extent() needs extra special handling for NOCOW/PREALLOC.

v1:
- Rebased to the latest for-next
To resolve the conflicts with the recently introduced extent map shrinker

- A new cleanup patch to remove the recursive header inclusion

- Use a new structure to pass the file extent item related members around

- Add a new comment on why we're intentionally passing incorrect numbers for NOCOW/PREALLOC ordered extents inside btrfs_create_dio_extent()

[REPO]
https://github.com/adam900710/linux/tree/em_cleanup

This series introduce two new members (disk_bytenr/offset) to extent_map, and removes three old members (block_start/block_len/offset), finally rename one member (orig_block_len -> disk_num_bytes).

This should save us one u64 for extent_map, although with the recent extent map shrinker, the saving is not that useful.

But to make things safe to migrate, I introduce extra sanity checks for extent_map, and do cross check for both old and new members.

The extra sanity checks already exposed one bug (thankfully harmless) causing em::block_start to be incorrect.

But so far, the patchset is fine for default fstests run.

Furthermore, since we're already having too long parameter lists for extent_map/ordered_extent/can_nocow_extent, here is a new structure, btrfs_file_extent, a memory-access-friendly structure to represent a btrfs_file_extent_item.

With the help of that structure, we can use that to represent a file extent item without a super long parameter list.

The patchset would rename orig_block_len to disk_num_bytes first. Then introduce the new member, the extra sanity checks, and introduce the new btrfs_file_extent structure and use that to remove the older 3 members from extent_map.

After all above works done, use btrfs_file_extent to further cleanup can_nocow_file_extent_args()/btrfs_alloc_ordered_extent()/create_io_em()/ btrfs_create_dio_extent().

The cleanup is in fact pretty tricky, the current code base never expects correct numbers for NOCOW/PREALLOC OEs, thus we have to keep the old but incorrect numbers just for NOCOW/PREALLOC.

I will address the NOCOW/PREALLOC shenanigans the future, but after the huge cleanup across multiple core structures.

Qu Wenruo (11):
btrfs: rename extent_map::orig_block_len to disk_num_bytes
btrfs: export the expected file extent through can_nocow_extent()
btrfs: introduce new members for extent_map
btrfs: introduce extra sanity checks for extent maps
btrfs: remove extent_map::orig_start member
btrfs: remove extent_map::block_len member
btrfs: remove extent_map::block_start member
btrfs: cleanup duplicated parameters related to
can_nocow_file_extent_args
btrfs: cleanup duplicated parameters related to
btrfs_alloc_ordered_extent
btrfs: cleanup duplicated parameters related to create_io_em()
btrfs: cleanup duplicated parameters related to
btrfs_create_dio_extent()

 fs/btrfs/btrfs_inode.h            |   4 +-
 fs/btrfs/compression.c            |   7 +-
 fs/btrfs/defrag.c                 |  14 +-
 fs/btrfs/extent_io.c              |  10 +-
 fs/btrfs/extent_map.c             | 187 ++++++++++++------
 fs/btrfs/extent_map.h             |  51 +++--
 fs/btrfs/file-item.c              |  23 +--
 fs/btrfs/file.c                   |  18 +-
 fs/btrfs/inode.c                  | 308 +++++++++++++-----------------
 fs/btrfs/ordered-data.c           |  36 +++-
 fs/btrfs/ordered-data.h           |  22 ++-
 fs/btrfs/relocation.c             |   5 +-
 fs/btrfs/tests/extent-map-tests.c | 114 ++++++-----
 fs/btrfs/tests/inode-tests.c      | 177 ++++++++---------
 fs/btrfs/tree-log.c               |  25 +--
 fs/btrfs/zoned.c                  |   4 +-
 include/trace/events/btrfs.h      |  26 +--
 17 files changed, 548 insertions(+), 483 deletions(-)
Signature · 1 line
2.45.0
author
#1 lore raw

PATCH v2 1/11 btrfs: rename extent_map::orig_block_len to disk_num_bytes

This would make it very obvious that the member just matches btrfs_file_extent_item::disk_num_bytes.

---
 fs/btrfs/extent_map.c | 16 ++++++++--------
 fs/btrfs/extent_map.h |  2 +-
 fs/btrfs/file-item.c  |  4 ++--
 fs/btrfs/file.c       |  2 +-
 fs/btrfs/inode.c      | 10 +++++-----
 fs/btrfs/tree-log.c   |  6 +++---
 6 files changed, 20 insertions(+), 20 deletions(-)
fs/btrfs/extent_map.c +8 -8
index 744e8952abb0..4230dd0f34cc 100644
@@ -783,14 +783,14 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
 					split->block_len = em->block_len;
 				else
 					split->block_len = split->len;
-				split->orig_block_len = max(split->block_len,
-						em->orig_block_len);
+				split->disk_num_bytes = max(split->block_len,
+							    em->disk_num_bytes);
 				split->ram_bytes = em->ram_bytes;
 			} else {
 				split->orig_start = split->start;
 				split->block_len = 0;
 				split->block_start = em->block_start;
-				split->orig_block_len = 0;
+				split->disk_num_bytes = 0;
 				split->ram_bytes = split->len;
 			}
 
@@ -815,8 +815,8 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
 			split->generation = gen;
 
 			if (em->block_start < EXTENT_MAP_LAST_BYTE) {
-				split->orig_block_len = max(em->block_len,
-						    em->orig_block_len);
+				split->disk_num_bytes = max(em->block_len,
+							    em->disk_num_bytes);
 
 				split->ram_bytes = em->ram_bytes;
 				if (compressed) {
@@ -833,7 +833,7 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
 				split->ram_bytes = split->len;
 				split->orig_start = split->start;
 				split->block_len = 0;
-				split->orig_block_len = 0;
+				split->disk_num_bytes = 0;
 			}
 
 			if (extent_map_in_tree(em)) {
@@ -990,7 +990,7 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
 	split_pre->orig_start = split_pre->start;
 	split_pre->block_start = new_logical;
 	split_pre->block_len = split_pre->len;
-	split_pre->orig_block_len = split_pre->block_len;
+	split_pre->disk_num_bytes = split_pre->block_len;
 	split_pre->ram_bytes = split_pre->len;
 	split_pre->flags = flags;
 	split_pre->generation = em->generation;
@@ -1008,7 +1008,7 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
 	split_mid->orig_start = split_mid->start;
 	split_mid->block_start = em->block_start + pre;
 	split_mid->block_len = split_mid->len;
-	split_mid->orig_block_len = split_mid->block_len;
+	split_mid->disk_num_bytes = split_mid->block_len;
 	split_mid->ram_bytes = split_mid->len;
 	split_mid->flags = flags;
 	split_mid->generation = em->generation;
fs/btrfs/extent_map.h +1 -1
index 6d587111f73a..6ea0287b0d61 100644
@@ -74,7 +74,7 @@ struct extent_map {
 	 * The full on-disk extent length, matching
 	 * btrfs_file_extent_item::disk_num_bytes.
 	 */
-	u64 orig_block_len;
+	u64 disk_num_bytes;
 
 	/*
 	 * The decompressed size of the whole on-disk extent, matching
fs/btrfs/file-item.c +2 -2
index bce95f871750..2197cfe5443b 100644
@@ -1294,7 +1294,7 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		em->len = btrfs_file_extent_end(path) - extent_start;
 		em->orig_start = extent_start -
 			btrfs_file_extent_offset(leaf, fi);
-		em->orig_block_len = btrfs_file_extent_disk_num_bytes(leaf, fi);
+		em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
 		bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
 		if (bytenr == 0) {
 			em->block_start = EXTENT_MAP_HOLE;
@@ -1303,7 +1303,7 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		if (compress_type != BTRFS_COMPRESS_NONE) {
 			extent_map_set_compression(em, compress_type);
 			em->block_start = bytenr;
-			em->block_len = em->orig_block_len;
+			em->block_len = em->disk_num_bytes;
 		} else {
 			bytenr += btrfs_file_extent_offset(leaf, fi);
 			em->block_start = bytenr;
fs/btrfs/file.c +1 -1
index 0c7c1b42028e..d3cbd161cd90 100644
@@ -2338,7 +2338,7 @@ static int fill_holes(struct btrfs_trans_handle *trans,
 
 		hole_em->block_start = EXTENT_MAP_HOLE;
 		hole_em->block_len = 0;
-		hole_em->orig_block_len = 0;
+		hole_em->disk_num_bytes = 0;
 		hole_em->generation = trans->transid;
 
 		ret = btrfs_replace_extent_map_range(inode, hole_em, true);
fs/btrfs/inode.c +5 -5
index d0274324c75a..0e5d4517af0e 100644
@@ -4971,7 +4971,7 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
 
 			hole_em->block_start = EXTENT_MAP_HOLE;
 			hole_em->block_len = 0;
-			hole_em->orig_block_len = 0;
+			hole_em->disk_num_bytes = 0;
 			hole_em->ram_bytes = hole_size;
 			hole_em->generation = btrfs_get_fs_generation(fs_info);
 
@@ -7292,7 +7292,7 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
 /* The callers of this must take lock_extent() */
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 				       u64 len, u64 orig_start, u64 block_start,
-				       u64 block_len, u64 orig_block_len,
+				       u64 block_len, u64 disk_num_bytes,
 				       u64 ram_bytes, int compress_type,
 				       int type)
 {
@@ -7324,7 +7324,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 		ASSERT(block_len == len);
 
 		/* COW results a new extent matching our file extent size. */
-		ASSERT(orig_block_len == len);
+		ASSERT(disk_num_bytes == len);
 		ASSERT(ram_bytes == len);
 
 		/* Since it's a new extent, we should not have any offset. */
@@ -7351,7 +7351,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 	em->len = len;
 	em->block_len = block_len;
 	em->block_start = block_start;
-	em->orig_block_len = orig_block_len;
+	em->disk_num_bytes = disk_num_bytes;
 	em->ram_bytes = ram_bytes;
 	em->generation = -1;
 	em->flags |= EXTENT_FLAG_PINNED;
@@ -9587,7 +9587,7 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
 		em->len = ins.offset;
 		em->block_start = ins.objectid;
 		em->block_len = ins.offset;
-		em->orig_block_len = ins.offset;
+		em->disk_num_bytes = ins.offset;
 		em->ram_bytes = ins.offset;
 		em->flags |= EXTENT_FLAG_PREALLOC;
 		em->generation = trans->transid;
fs/btrfs/tree-log.c +3 -3
index 5146387b416b..83dff4b06c84 100644
@@ -2874,7 +2874,7 @@ static inline void btrfs_remove_log_ctx(struct btrfs_root *root,
 	mutex_unlock(&root->log_mutex);
 }
 
-/* 
+/*
  * Invoked in log mutex context, or be sure there is no other task which
  * can access the list.
  */
@@ -4648,7 +4648,7 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
 	/* If we're compressed we have to save the entire range of csums. */
 	if (extent_map_is_compressed(em)) {
 		csum_offset = 0;
-		csum_len = max(em->block_len, em->orig_block_len);
+		csum_len = max(em->block_len, em->disk_num_bytes);
 	} else {
 		csum_offset = mod_start - em->start;
 		csum_len = mod_len;
@@ -4698,7 +4698,7 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
 	else
 		btrfs_set_stack_file_extent_type(&fi, BTRFS_FILE_EXTENT_REG);
 
-	block_len = max(em->block_len, em->orig_block_len);
+	block_len = max(em->block_len, em->disk_num_bytes);
 	compress_type = extent_map_compression(em);
 	if (compress_type != BTRFS_COMPRESS_NONE) {
 		btrfs_set_stack_file_extent_disk_bytenr(&fi, em->block_start);
Signature · 1 line
2.45.0
#2 lore raw

Re: [PATCH v2 01/11] btrfs: rename extent_map::orig_block_len to disk_num_bytes

Quoted 177 lines · Qu Wenruo

This would make it very obvious that the member just matches btrfs_file_extent_item::disk_num_bytes.

---
 fs/btrfs/extent_map.c | 16 ++++++++--------
 fs/btrfs/extent_map.h |  2 +-
 fs/btrfs/file-item.c  |  4 ++--
 fs/btrfs/file.c       |  2 +-
 fs/btrfs/inode.c      | 10 +++++-----
 fs/btrfs/tree-log.c   |  6 +++---
 6 files changed, 20 insertions(+), 20 deletions(-)
fs/btrfs/extent_map.c +8 -8
index 744e8952abb0..4230dd0f34cc 100644
@@ -783,14 +783,14 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                        split->block_len = em->block_len;
                                else
                                        split->block_len = split->len;
-                               split->orig_block_len = max(split->block_len,
-                                               em->orig_block_len);
+                               split->disk_num_bytes = max(split->block_len,
+                                                           em->disk_num_bytes);
                                split->ram_bytes = em->ram_bytes;
                        } else {
                                split->orig_start = split->start;
                                split->block_len = 0;
                                split->block_start = em->block_start;
-                               split->orig_block_len = 0;
+                               split->disk_num_bytes = 0;
                                split->ram_bytes = split->len;
                        }
 
@@ -815,8 +815,8 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        split->generation = gen;
 
                        if (em->block_start < EXTENT_MAP_LAST_BYTE) {
-                               split->orig_block_len = max(em->block_len,
-                                                   em->orig_block_len);
+                               split->disk_num_bytes = max(em->block_len,
+                                                           em->disk_num_bytes);
 
                                split->ram_bytes = em->ram_bytes;
                                if (compressed) {
@@ -833,7 +833,7 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                split->ram_bytes = split->len;
                                split->orig_start = split->start;
                                split->block_len = 0;
-                               split->orig_block_len = 0;
+                               split->disk_num_bytes = 0;
                        }
 
                        if (extent_map_in_tree(em)) {
@@ -990,7 +990,7 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        split_pre->orig_start = split_pre->start;
        split_pre->block_start = new_logical;
        split_pre->block_len = split_pre->len;
-       split_pre->orig_block_len = split_pre->block_len;
+       split_pre->disk_num_bytes = split_pre->block_len;
        split_pre->ram_bytes = split_pre->len;
        split_pre->flags = flags;
        split_pre->generation = em->generation;
@@ -1008,7 +1008,7 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        split_mid->orig_start = split_mid->start;
        split_mid->block_start = em->block_start + pre;
        split_mid->block_len = split_mid->len;
-       split_mid->orig_block_len = split_mid->block_len;
+       split_mid->disk_num_bytes = split_mid->block_len;
        split_mid->ram_bytes = split_mid->len;
        split_mid->flags = flags;
        split_mid->generation = em->generation;
fs/btrfs/extent_map.h +1 -1
index 6d587111f73a..6ea0287b0d61 100644
@@ -74,7 +74,7 @@ struct extent_map {
         * The full on-disk extent length, matching
         * btrfs_file_extent_item::disk_num_bytes.
         */
-       u64 orig_block_len;
+       u64 disk_num_bytes;
 
        /*
         * The decompressed size of the whole on-disk extent, matching
fs/btrfs/file-item.c +2 -2
index bce95f871750..2197cfe5443b 100644
@@ -1294,7 +1294,7 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                em->len = btrfs_file_extent_end(path) - extent_start;
                em->orig_start = extent_start -
                        btrfs_file_extent_offset(leaf, fi);
-               em->orig_block_len = btrfs_file_extent_disk_num_bytes(leaf, fi);
+               em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
                bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
                if (bytenr == 0) {
                        em->block_start = EXTENT_MAP_HOLE;
@@ -1303,7 +1303,7 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                if (compress_type != BTRFS_COMPRESS_NONE) {
                        extent_map_set_compression(em, compress_type);
                        em->block_start = bytenr;
-                       em->block_len = em->orig_block_len;
+                       em->block_len = em->disk_num_bytes;
                } else {
                        bytenr += btrfs_file_extent_offset(leaf, fi);
                        em->block_start = bytenr;
fs/btrfs/file.c +1 -1
index 0c7c1b42028e..d3cbd161cd90 100644
@@ -2338,7 +2338,7 @@ static int fill_holes(struct btrfs_trans_handle *trans,
 
                hole_em->block_start = EXTENT_MAP_HOLE;
                hole_em->block_len = 0;
-               hole_em->orig_block_len = 0;
+               hole_em->disk_num_bytes = 0;
                hole_em->generation = trans->transid;
 
                ret = btrfs_replace_extent_map_range(inode, hole_em, true);
fs/btrfs/inode.c +5 -5
index d0274324c75a..0e5d4517af0e 100644
@@ -4971,7 +4971,7 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
 
                        hole_em->block_start = EXTENT_MAP_HOLE;
                        hole_em->block_len = 0;
-                       hole_em->orig_block_len = 0;
+                       hole_em->disk_num_bytes = 0;
                        hole_em->ram_bytes = hole_size;
                        hole_em->generation = btrfs_get_fs_generation(fs_info);
 
@@ -7292,7 +7292,7 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
 /* The callers of this must take lock_extent() */
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 orig_start, u64 block_start,
-                                      u64 block_len, u64 orig_block_len,
+                                      u64 block_len, u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
                                       int type)
 {
@@ -7324,7 +7324,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                ASSERT(block_len == len);
 
                /* COW results a new extent matching our file extent size. */
-               ASSERT(orig_block_len == len);
+               ASSERT(disk_num_bytes == len);
                ASSERT(ram_bytes == len);
 
                /* Since it's a new extent, we should not have any offset. */
@@ -7351,7 +7351,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
        em->len = len;
        em->block_len = block_len;
        em->block_start = block_start;
-       em->orig_block_len = orig_block_len;
+       em->disk_num_bytes = disk_num_bytes;
        em->ram_bytes = ram_bytes;
        em->generation = -1;
        em->flags |= EXTENT_FLAG_PINNED;
@@ -9587,7 +9587,7 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
                em->len = ins.offset;
                em->block_start = ins.objectid;
                em->block_len = ins.offset;
-               em->orig_block_len = ins.offset;
+               em->disk_num_bytes = ins.offset;
                em->ram_bytes = ins.offset;
                em->flags |= EXTENT_FLAG_PREALLOC;
                em->generation = trans->transid;
fs/btrfs/tree-log.c +0 -0
index 5146387b416b..83dff4b06c84 100644

fs/btrfs/tree-log.c static inline void btrfs_remove_log_ctx(struct btrfs_root *root, · line 2877

Quoted 5 lines
fs/btrfs/tree-log.c +1 -1
@@ -2874,7 +2874,7 @@ static inline void btrfs_remove_log_ctx(struct btrfs_root *root,
        mutex_unlock(&root->log_mutex);
 }
 
-/*
+/*

Unrelated white space only changed. This was pointed out before IIRC.

Other than that it looks fine, but I have to read the rest of the patchset.

Thanks.

Quoted 25 lines
fs/btrfs/tree-log.c +2 -3
  * Invoked in log mutex context, or be sure there is no other task which
  * can access the list.
  */
@@ -4648,7 +4648,7 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
        /* If we're compressed we have to save the entire range of csums. */
        if (extent_map_is_compressed(em)) {
                csum_offset = 0;
-               csum_len = max(em->block_len, em->orig_block_len);
+               csum_len = max(em->block_len, em->disk_num_bytes);
        } else {
                csum_offset = mod_start - em->start;
                csum_len = mod_len;
@@ -4698,7 +4698,7 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
        else
                btrfs_set_stack_file_extent_type(&fi, BTRFS_FILE_EXTENT_REG);
 
-       block_len = max(em->block_len, em->orig_block_len);
+       block_len = max(em->block_len, em->disk_num_bytes);
        compress_type = extent_map_compression(em);
        if (compress_type != BTRFS_COMPRESS_NONE) {
                btrfs_set_stack_file_extent_disk_bytenr(&fi, em->block_start);
--

2.45.0

author
#3 lore raw

PATCH v2 2/11 btrfs: export the expected file extent through can_nocow_extent()

In reply to Qu Wenruo

Currently function can_nocow_extent() only returns members needed for extent_map.

However since we will soon change the extent_map structure to be more like btrfs_file_extent_item, we want to expose the expected file extent caused by the NOCOW write for future usage.

This would introduce a new structure, btrfs_file_extent, to be a more memory-access-friendly representation of btrfs_file_extent_item. And use that structure to expose the expected file extent caused by the NOCOW write.

For now there is no user of the new structure yet.

---
 fs/btrfs/btrfs_inode.h | 20 +++++++++++++++++++-
 fs/btrfs/file.c        |  2 +-
 fs/btrfs/inode.c       | 22 +++++++++++++++++++---
 3 files changed, 39 insertions(+), 5 deletions(-)
fs/btrfs/btrfs_inode.h +19 -1
index de918d89a582..18678762615a 100644
@@ -443,9 +443,27 @@ int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page,
 			    u32 pgoff, u8 *csum, const u8 * const csum_expected);
 bool btrfs_data_csum_ok(struct btrfs_bio *bbio, struct btrfs_device *dev,
 			u32 bio_offset, struct bio_vec *bv);
+
+/*
+ * A more access-friendly representation of btrfs_file_extent_item.
+ *
+ * Unused members are excluded.
+ */
+struct btrfs_file_extent {
+	u64 disk_bytenr;
+	u64 disk_num_bytes;
+
+	u64 num_bytes;
+	u64 ram_bytes;
+	u64 offset;
+
+	u8 compression;
+};
+
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
 			      u64 *orig_start, u64 *orig_block_len,
-			      u64 *ram_bytes, bool nowait, bool strict);
+			      u64 *ram_bytes, struct btrfs_file_extent *file_extent,
+			      bool nowait, bool strict);
 
 void btrfs_del_delalloc_inode(struct btrfs_inode *inode);
 struct inode *btrfs_lookup_dentry(struct inode *dir, struct dentry *dentry);
fs/btrfs/file.c +1 -1
index d3cbd161cd90..63a13a4cace0 100644
@@ -1104,7 +1104,7 @@ int btrfs_check_nocow_lock(struct btrfs_inode *inode, loff_t pos,
 						   &cached_state);
 	}
 	ret = can_nocow_extent(&inode->vfs_inode, lockstart, &num_bytes,
-			NULL, NULL, NULL, nowait, false);
+			NULL, NULL, NULL, NULL, nowait, false);
 	if (ret <= 0)
 		btrfs_drew_write_unlock(&root->snapshot_lock);
 	else
fs/btrfs/inode.c +19 -3
index 0e5d4517af0e..2815b72f2d85 100644
@@ -1857,6 +1857,9 @@ struct can_nocow_file_extent_args {
 	u64 extent_offset;
 	/* Number of bytes that can be written to in NOCOW mode. */
 	u64 num_bytes;
+
+	/* The expected file extent for the NOCOW write. */
+	struct btrfs_file_extent file_extent;
 };
 
 /*
@@ -1921,6 +1924,12 @@ static int can_nocow_file_extent(struct btrfs_path *path,
 
 	extent_end = btrfs_file_extent_end(path);
 
+	args->file_extent.disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+	args->file_extent.disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
+	args->file_extent.ram_bytes = btrfs_file_extent_ram_bytes(leaf, fi);
+	args->file_extent.offset = btrfs_file_extent_offset(leaf, fi);
+	args->file_extent.compression = btrfs_file_extent_compression(leaf, fi);
+
 	/*
 	 * The following checks can be expensive, as they need to take other
 	 * locks and do btree or rbtree searches, so release the path to avoid
@@ -1955,6 +1964,9 @@ static int can_nocow_file_extent(struct btrfs_path *path,
 	args->disk_bytenr += args->start - key->offset;
 	args->num_bytes = min(args->end + 1, extent_end) - args->start;
 
+	args->file_extent.num_bytes = args->num_bytes;
+	args->file_extent.offset += args->start - key->offset;
+
 	/*
 	 * Force COW if csums exist in the range. This ensures that csums for a
 	 * given extent are either valid or do not exist.
@@ -7099,7 +7111,8 @@ static bool btrfs_extent_readonly(struct btrfs_fs_info *fs_info, u64 bytenr)
  */
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
 			      u64 *orig_start, u64 *orig_block_len,
-			      u64 *ram_bytes, bool nowait, bool strict)
+			      u64 *ram_bytes, struct btrfs_file_extent *file_extent,
+			      bool nowait, bool strict)
 {
 	struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
 	struct can_nocow_file_extent_args nocow_args = { 0 };
@@ -7188,6 +7201,9 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
 		*orig_start = key.offset - nocow_args.extent_offset;
 	if (orig_block_len)
 		*orig_block_len = nocow_args.disk_num_bytes;
+	if (file_extent)
+		memcpy(file_extent, &nocow_args.file_extent,
+		       sizeof(struct btrfs_file_extent));
 
 	*len = nocow_args.num_bytes;
 	ret = 1;
@@ -7407,7 +7423,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 		block_start = em->block_start + (start - em->start);
 
 		if (can_nocow_extent(inode, start, &len, &orig_start,
-				     &orig_block_len, &ram_bytes, false, false) == 1) {
+				     &orig_block_len, &ram_bytes, NULL, false, false) == 1) {
 			bg = btrfs_inc_nocow_writers(fs_info, block_start);
 			if (bg)
 				can_nocow = true;
@@ -10660,7 +10676,7 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file,
 		free_extent_map(em);
 		em = NULL;
 
-		ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, false, true);
+		ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, NULL, false, true);
 		if (ret < 0) {
 			goto out;
 		} else if (ret) {
Signature · 1 line
2.45.0
#4 lore raw

Re: [PATCH v2 02/11] btrfs: export the expected file extent through can_nocow_extent()

Quoted 9 lines · Qu Wenruo

Currently function can_nocow_extent() only returns members needed for extent_map.

However since we will soon change the extent_map structure to be more like btrfs_file_extent_item, we want to expose the expected file extent caused by the NOCOW write for future usage.

This would introduce a new structure, btrfs_file_extent, to be a more

"would introduce" -> introduces

Quoted 19 lines

memory-access-friendly representation of btrfs_file_extent_item. And use that structure to expose the expected file extent caused by the NOCOW write.

For now there is no user of the new structure yet.

---
 fs/btrfs/btrfs_inode.h | 20 +++++++++++++++++++-
 fs/btrfs/file.c        |  2 +-
 fs/btrfs/inode.c       | 22 +++++++++++++++++++---
 3 files changed, 39 insertions(+), 5 deletions(-)
fs/btrfs/btrfs_inode.h +0 -0
index de918d89a582..18678762615a 100644

fs/btrfs/btrfs_inode.h int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page, · line 450

Quoted 8 lines
fs/btrfs/btrfs_inode.h +5 -0
@@ -443,9 +443,27 @@ int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page,
                            u32 pgoff, u8 *csum, const u8 * const csum_expected);
 bool btrfs_data_csum_ok(struct btrfs_bio *bbio, struct btrfs_device *dev,
                        u32 bio_offset, struct bio_vec *bv);
+
+/*
+ * A more access-friendly representation of btrfs_file_extent_item.
+ *
+ * Unused members are excluded.

A bit confusing this, you want to say this is a subset of btrfs_file_extent_item.
So just say something like:

This represents details about the target file extent item of a write operation.

fs/btrfs/btrfs_inode.h int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page, · line 460

Quoted 10 lines
fs/btrfs/btrfs_inode.h +10 -0
+ */
+struct btrfs_file_extent {
+       u64 disk_bytenr;
+       u64 disk_num_bytes;
+
+       u64 num_bytes;
+       u64 ram_bytes;
+       u64 offset;
+
+       u8 compression;

The blank lines between the members seem kind of ad-hoc, redundant to me - i.e. no logical grouping.
Maybe I missed something?

Quoted 71 lines
fs/btrfs/btrfs_inode.h +4 -1
+};
+
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
                              u64 *orig_start, u64 *orig_block_len,
-                             u64 *ram_bytes, bool nowait, bool strict);
+                             u64 *ram_bytes, struct btrfs_file_extent *file_extent,
+                             bool nowait, bool strict);
 
 void btrfs_del_delalloc_inode(struct btrfs_inode *inode);
 struct inode *btrfs_lookup_dentry(struct inode *dir, struct dentry *dentry);
fs/btrfs/file.c +1 -1
index d3cbd161cd90..63a13a4cace0 100644
@@ -1104,7 +1104,7 @@ int btrfs_check_nocow_lock(struct btrfs_inode *inode, loff_t pos,
                                                   &cached_state);
        }
        ret = can_nocow_extent(&inode->vfs_inode, lockstart, &num_bytes,
-                       NULL, NULL, NULL, nowait, false);
+                       NULL, NULL, NULL, NULL, nowait, false);
        if (ret <= 0)
                btrfs_drew_write_unlock(&root->snapshot_lock);
        else
fs/btrfs/inode.c +14 -1
index 0e5d4517af0e..2815b72f2d85 100644
@@ -1857,6 +1857,9 @@ struct can_nocow_file_extent_args {
        u64 extent_offset;
        /* Number of bytes that can be written to in NOCOW mode. */
        u64 num_bytes;
+
+       /* The expected file extent for the NOCOW write. */
+       struct btrfs_file_extent file_extent;
 };
 
 /*
@@ -1921,6 +1924,12 @@ static int can_nocow_file_extent(struct btrfs_path *path,
 
        extent_end = btrfs_file_extent_end(path);
 
+       args->file_extent.disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+       args->file_extent.disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
+       args->file_extent.ram_bytes = btrfs_file_extent_ram_bytes(leaf, fi);
+       args->file_extent.offset = btrfs_file_extent_offset(leaf, fi);
+       args->file_extent.compression = btrfs_file_extent_compression(leaf, fi);
+
        /*
         * The following checks can be expensive, as they need to take other
         * locks and do btree or rbtree searches, so release the path to avoid
@@ -1955,6 +1964,9 @@ static int can_nocow_file_extent(struct btrfs_path *path,
        args->disk_bytenr += args->start - key->offset;
        args->num_bytes = min(args->end + 1, extent_end) - args->start;
 
+       args->file_extent.num_bytes = args->num_bytes;
+       args->file_extent.offset += args->start - key->offset;
+
        /*
         * Force COW if csums exist in the range. This ensures that csums for a
         * given extent are either valid or do not exist.
@@ -7099,7 +7111,8 @@ static bool btrfs_extent_readonly(struct btrfs_fs_info *fs_info, u64 bytenr)
  */
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
                              u64 *orig_start, u64 *orig_block_len,
-                             u64 *ram_bytes, bool nowait, bool strict)
+                             u64 *ram_bytes, struct btrfs_file_extent *file_extent,
+                             bool nowait, bool strict)
 {
        struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
        struct can_nocow_file_extent_args nocow_args = { 0 };

fs/btrfs/inode.c noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len, · line 7206

Quoted 6 lines
fs/btrfs/inode.c +3 -0
@@ -7188,6 +7201,9 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
                *orig_start = key.offset - nocow_args.extent_offset;
        if (orig_block_len)
                *orig_block_len = nocow_args.disk_num_bytes;
+       if (file_extent)
+               memcpy(file_extent, &nocow_args.file_extent,
+                      sizeof(struct btrfs_file_extent));

Can use instead sizeof(*file_extent), and it will allow to have everything in the same line without being over 80 characters.

The rest looks fine, but I have to read the rest of the patchset to see how this is actually used.

Thanks.

Quoted 25 lines
fs/btrfs/inode.c +2 -3
        *len = nocow_args.num_bytes;
        ret = 1;
@@ -7407,7 +7423,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                block_start = em->block_start + (start - em->start);
 
                if (can_nocow_extent(inode, start, &len, &orig_start,
-                                    &orig_block_len, &ram_bytes, false, false) == 1) {
+                                    &orig_block_len, &ram_bytes, NULL, false, false) == 1) {
                        bg = btrfs_inc_nocow_writers(fs_info, block_start);
                        if (bg)
                                can_nocow = true;
@@ -10660,7 +10676,7 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file,
                free_extent_map(em);
                em = NULL;
 
-               ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, false, true);
+               ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, NULL, false, true);
                if (ret < 0) {
                        goto out;
                } else if (ret) {
--

2.45.0

#5 lore raw

Re: [PATCH v2 02/11] btrfs: export the expected file extent through can_nocow_extent()

在 2024/5/10 01:52, Filipe Manana 写道:
[...]

Quoted 6 lines
Quoted 6 lines

+ */

fs/btrfs/btrfs_inode.h +5 -0
+struct btrfs_file_extent {
+       u64 disk_bytenr;
+       u64 disk_num_bytes;
+
+       u64 num_bytes;
Quoted 8 lines

fs/btrfs/btrfs_inode.h int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page, · line 460

Quoted 4 lines
fs/btrfs/btrfs_inode.h +4 -0
+       u64 ram_bytes;
+       u64 offset;
+
+       u8 compression;

The blank lines between the members seem kind of ad-hoc, redundant to me - i.e. no logical grouping.
Maybe I missed something?

This is to separate the members so that, the disk_* ones are only referring to a data extent uniquely.

The ram_bytes/num_bytes/offset are the ones to specify the referred range inside the uncompressed extent.
Finally a u8 because it's not u64.

But for sure, I can remove all the new lines, because the grouping is not that strong.

Thanks,
Qu

author
#6 lore raw

PATCH v2 3/11 btrfs: introduce new members for extent_map

In reply to Qu Wenruo

Introduce two new members for extent_map:

- disk_bytenr
- offset

Both are matching the members with the same name inside btrfs_file_extent_items.

For now this patch only touches those members when:

- Reading btrfs_file_extent_items from disk
- Inserting new holes
- Merging two extent maps
With the new disk_bytenr and disk_num_bytes, doing merging would be a little complex, as we have 3 different cases:

* Both extent maps are referring to the same data extent
* Both extent maps are referring to different data extents, but those data extents are adjacent, and extent maps are at head/tail of each data extents
* One of the extent map is referring to an merged and larger data extent that covers both extent maps

The 3rd case seems only valid in selftest (test_case_3()), but a new helper merge_ondisk_extents() should be able to handle all of them.

To properly assign values for those new members, a new btrfs_file_extent parameter is introduced to all the involved call sites.

- For NOCOW writes the btrfs_file_extent would be exposed from can_nocow_file_extent().

- For other writes, the members can be easily calculated
As most of them have 0 offset and utilizing the whole on-disk data extent.
The exception is encoded write, but thankfully that interface provided offset directly and all other needed info.

For now, both the old members (block_start/block_len/orig_start) are co-existing with the new members (disk_bytenr/offset), meanwhile all the critical code is still using the old members only.

The cleanup would happen later after all the older and newer members are properly validated.

---
 fs/btrfs/defrag.c     |  4 +++
 fs/btrfs/extent_map.c | 75 +++++++++++++++++++++++++++++++++++++++++--
 fs/btrfs/extent_map.h | 17 ++++++++++
 fs/btrfs/file-item.c  |  9 +++++-
 fs/btrfs/file.c       |  1 +
 fs/btrfs/inode.c      | 60 ++++++++++++++++++++++++++++++----
 6 files changed, 155 insertions(+), 11 deletions(-)
fs/btrfs/defrag.c +4 -0
index 407ccec3e57e..242c5469f4ba 100644
@@ -709,6 +709,10 @@ static struct extent_map *defrag_get_extent(struct btrfs_inode *inode,
 			em->start = start;
 			em->orig_start = start;
 			em->block_start = EXTENT_MAP_HOLE;
+			em->disk_bytenr = EXTENT_MAP_HOLE;
+			em->disk_num_bytes = 0;
+			em->ram_bytes = 0;
+			em->offset = 0;
 			em->len = key.offset - start;
 			break;
 		}
fs/btrfs/extent_map.c +72 -3

149 lines not on this page · raw message

fs/btrfs/extent_map.h +17 -0
index 6ea0287b0d61..cc9c8092b704 100644
@@ -70,12 +70,29 @@ struct extent_map {
 	 */
 	u64 orig_start;
 
+	/*
+	 * The bytenr for of the full on-disk extent.
+	 *
+	 * For regular extents it's btrfs_file_extent_item::disk_bytenr.
+	 * For holes it's EXTENT_MAP_HOLE and for inline extents it's
+	 * EXTENT_MAP_INLINE.
+	 */
+	u64 disk_bytenr;
+
 	/*
 	 * The full on-disk extent length, matching
 	 * btrfs_file_extent_item::disk_num_bytes.
 	 */
 	u64 disk_num_bytes;
 
+	/*
+	 * Offset inside the decompressed extent.
+	 *
+	 * For regular extents it's btrfs_file_extent_item::offset.
+	 * For holes and inline extents it's 0.
+	 */
+	u64 offset;
+
 	/*
 	 * The decompressed size of the whole on-disk extent, matching
 	 * btrfs_file_extent_item::ram_bytes.
fs/btrfs/file-item.c +8 -1
index 2197cfe5443b..47bd4fe0a44b 100644
@@ -1294,12 +1294,17 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		em->len = btrfs_file_extent_end(path) - extent_start;
 		em->orig_start = extent_start -
 			btrfs_file_extent_offset(leaf, fi);
-		em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
 		bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
 		if (bytenr == 0) {
 			em->block_start = EXTENT_MAP_HOLE;
+			em->disk_bytenr = EXTENT_MAP_HOLE;
+			em->disk_num_bytes = 0;
+			em->offset = 0;
 			return;
 		}
+		em->disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+		em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
+		em->offset = btrfs_file_extent_offset(leaf, fi);
 		if (compress_type != BTRFS_COMPRESS_NONE) {
 			extent_map_set_compression(em, compress_type);
 			em->block_start = bytenr;
@@ -1316,8 +1321,10 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		ASSERT(extent_start == 0);
 
 		em->block_start = EXTENT_MAP_INLINE;
+		em->disk_bytenr = EXTENT_MAP_INLINE;
 		em->start = 0;
 		em->len = fs_info->sectorsize;
+		em->offset = 0;
 		/*
 		 * Initialize orig_start and block_len with the same values
 		 * as in inode.c:btrfs_get_extent().
fs/btrfs/file.c +1 -0
index 63a13a4cace0..8931eeee199d 100644
@@ -2337,6 +2337,7 @@ static int fill_holes(struct btrfs_trans_handle *trans,
 		hole_em->orig_start = offset;
 
 		hole_em->block_start = EXTENT_MAP_HOLE;
+		hole_em->disk_bytenr = EXTENT_MAP_HOLE;
 		hole_em->block_len = 0;
 		hole_em->disk_num_bytes = 0;
 		hole_em->generation = trans->transid;
fs/btrfs/inode.c +53 -7

207 lines not on this page · raw message

Signature · 1 line
2.45.0
#7 lore raw

Re: [PATCH v2 03/11] btrfs: introduce new members for extent_map

Quoted 17 lines · Qu Wenruo

Introduce two new members for extent_map:

- disk_bytenr
- offset

Both are matching the members with the same name inside btrfs_file_extent_items.

For now this patch only touches those members when:

- Reading btrfs_file_extent_items from disk
- Inserting new holes
- Merging two extent maps
With the new disk_bytenr and disk_num_bytes, doing merging would be a little complex, as we have 3 different cases:

Quoted 4 lines

* Both extent maps are referring to the same data extent
* Both extent maps are referring to different data extents, but those data extents are adjacent, and extent maps are at head/tail of each data extents

I have no idea what this last part of the sentence means:

"and extent maps are at head/tail of each data extents"

Quoted 1 line

* One of the extent map is referring to an merged and larger data

map -> maps
an -> a

Quoted 1 line

extent that covers both extent maps

Not sure if I can understand this. How can one of the extent maps already cover the range of the other extent map?
This suggests some overlapping, or most likely I misunderstood what this is trying to say.

Quoted 69 lines

The 3rd case seems only valid in selftest (test_case_3()), but a new helper merge_ondisk_extents() should be able to handle all of them.

To properly assign values for those new members, a new btrfs_file_extent parameter is introduced to all the involved call sites.

- For NOCOW writes the btrfs_file_extent would be exposed from can_nocow_file_extent().

- For other writes, the members can be easily calculated
As most of them have 0 offset and utilizing the whole on-disk data extent.
The exception is encoded write, but thankfully that interface provided offset directly and all other needed info.

For now, both the old members (block_start/block_len/orig_start) are co-existing with the new members (disk_bytenr/offset), meanwhile all the critical code is still using the old members only.

The cleanup would happen later after all the older and newer members are properly validated.

---
 fs/btrfs/defrag.c     |  4 +++
 fs/btrfs/extent_map.c | 75 +++++++++++++++++++++++++++++++++++++++++--
 fs/btrfs/extent_map.h | 17 ++++++++++
 fs/btrfs/file-item.c  |  9 +++++-
 fs/btrfs/file.c       |  1 +
 fs/btrfs/inode.c      | 60 ++++++++++++++++++++++++++++++----
 6 files changed, 155 insertions(+), 11 deletions(-)
fs/btrfs/defrag.c +4 -0
index 407ccec3e57e..242c5469f4ba 100644
@@ -709,6 +709,10 @@ static struct extent_map *defrag_get_extent(struct btrfs_inode *inode,
                        em->start = start;
                        em->orig_start = start;
                        em->block_start = EXTENT_MAP_HOLE;
+                       em->disk_bytenr = EXTENT_MAP_HOLE;
+                       em->disk_num_bytes = 0;
+                       em->ram_bytes = 0;
+                       em->offset = 0;
                        em->len = key.offset - start;
                        break;
                }
fs/btrfs/extent_map.c +11 -0
index 4230dd0f34cc..4d4ac9fc43e2 100644
@@ -232,6 +232,58 @@ static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma
        return next->block_start == prev->block_start;
 }
 
+/*
+ * Handle the ondisk data extents merge for @prev and @next.
+ *
+ * Only touches disk_bytenr/disk_num_bytes/offset/ram_bytes.
+ * For now only uncompressed regular extent can be merged.
+ *
+ * @prev and @next will be both updated to point to the new merged range.
+ * Thus one of them should be removed by the caller.
+ */
+static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *next)
+{

fs/btrfs/extent_map.c static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma · line 255

Quoted 10 lines
fs/btrfs/extent_map.c +10 -0
+       u64 new_disk_bytenr;
+       u64 new_disk_num_bytes;
+       u64 new_offset;
+
+       /* @prev and @next should not be compressed. */
+       ASSERT(!extent_map_is_compressed(prev));
+       ASSERT(!extent_map_is_compressed(next));
+
+       /*
+        * There are several different cases that @prev and @next can be merged.

that -> where

fs/btrfs/extent_map.c static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma · line 259

Quoted 4 lines
fs/btrfs/extent_map.c +4 -0
+        *
+        * 1) They are referring to the same data extent
+        * 2) Their ondisk data extents are adjacent and @prev is the tail
+        *    and @next is the head of their data extents

This thing of head and tail is confusing.
Just say that they are adjacent on disk, it's clear enough and head/tail suggest some part of an extent only, which isn't the case.

fs/btrfs/extent_map.c static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma · line 260

Quoted 1 line
fs/btrfs/extent_map.c +1 -0
+        * 3) One of @prev/@next is referrring to a larger merged data extent.

referrring -> referring

fs/btrfs/extent_map.c static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma · line 261

Quoted 1 line
fs/btrfs/extent_map.c +1 -0
+        *    (test_case_3 of extent maps tests).

If something is only exercised by the self tests but can't happen in practice, I'd rather change the tests and assert such a case can't happen.

fs/btrfs/extent_map.c static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma · line 273

Quoted 12 lines
fs/btrfs/extent_map.c +12 -0
+        *
+        * The calculation here always merge the data extents first, then update
+        * @offset using the new data extents.
+        *
+        * For case 1), the merged data extent would be the same.
+        * For case 2), we just merge the two data extents into one.
+        * For case 3), we just got the larger data extent.
+        */
+       new_disk_bytenr = min(prev->disk_bytenr, next->disk_bytenr);
+       new_disk_num_bytes = max(prev->disk_bytenr + prev->disk_num_bytes,
+                                next->disk_bytenr + next->disk_num_bytes) -
+                            new_disk_bytenr;

So this is confusing, disk_num_bytes being a max between the two extent maps and not their sum.
I gather this is modelled after what we already do at btrfs_drop_extent_map_range() when splitting.

But the truth is we never used the disk_num_bytes of an extent map that was merged - we also didn't do it before this patch, for that reason.
It's only used for logging new extents - which can't be merged - they can be merged only after being logged.

We can set this to the sum, or leave with some value to signal it's invalid. Just leave a comment saying disk_num_bytes it's not used anywhere for merged extent maps.

In the splitting case at btrfs_drop_extent_map_range() it's what we need since in the case the extent is new and not logged (in the modified list), disk_num_bytes always represents the size of the original, before split, extent.

Quoted 120 lines
fs/btrfs/extent_map.c +33 -3
+       new_offset = prev->disk_bytenr + prev->offset - new_disk_bytenr;
+
+       prev->disk_bytenr = new_disk_bytenr;
+       prev->disk_num_bytes = new_disk_num_bytes;
+       prev->ram_bytes = new_disk_num_bytes;
+       prev->offset = new_offset;
+
+       next->disk_bytenr = new_disk_bytenr;
+       next->disk_num_bytes = new_disk_num_bytes;
+       next->ram_bytes = new_disk_num_bytes;
+       next->offset = new_offset;
+}
+
 static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
 {
        struct extent_map_tree *tree = &inode->extent_tree;
@@ -263,6 +315,9 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                        em->block_len += merge->block_len;
                        em->block_start = merge->block_start;
                        em->generation = max(em->generation, merge->generation);
+
+                       if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
+                               merge_ondisk_extents(merge, em);
                        em->flags |= EXTENT_FLAG_MERGED;
 
                        rb_erase_cached(&merge->rb_node, &tree->map);
@@ -278,6 +333,8 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
        if (rb && can_merge_extent_map(merge) && mergeable_maps(em, merge)) {
                em->len += merge->len;
                em->block_len += merge->block_len;
+               if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
+                       merge_ondisk_extents(em, merge);
                rb_erase_cached(&merge->rb_node, &tree->map);
                RB_CLEAR_NODE(&merge->rb_node);
                em->generation = max(em->generation, merge->generation);
@@ -565,6 +622,7 @@ static noinline int merge_extent_mapping(struct btrfs_inode *inode,
            !extent_map_is_compressed(em)) {
                em->block_start += start_diff;
                em->block_len = em->len;
+               em->offset += start_diff;
        }
        return add_extent_mapping(inode, em, 0);
 }
@@ -783,14 +841,18 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                        split->block_len = em->block_len;
                                else
                                        split->block_len = split->len;
+                               split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = max(split->block_len,
                                                            em->disk_num_bytes);
+                               split->offset = em->offset;
                                split->ram_bytes = em->ram_bytes;
                        } else {
                                split->orig_start = split->start;
                                split->block_len = 0;
                                split->block_start = em->block_start;
+                               split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = 0;
+                               split->offset = 0;
                                split->ram_bytes = split->len;
                        }
 
@@ -811,13 +873,14 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        split->start = end;
                        split->len = em_end - end;
                        split->block_start = em->block_start;
+                       split->disk_bytenr = em->disk_bytenr;
                        split->flags = flags;
                        split->generation = gen;
 
                        if (em->block_start < EXTENT_MAP_LAST_BYTE) {
                                split->disk_num_bytes = max(em->block_len,
                                                            em->disk_num_bytes);
-
+                               split->offset = em->offset + end - em->start;
                                split->ram_bytes = em->ram_bytes;
                                if (compressed) {
                                        split->block_len = em->block_len;
@@ -830,10 +893,11 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                        split->orig_start = em->orig_start;
                                }
                        } else {
+                               split->disk_num_bytes = 0;
+                               split->offset = 0;
                                split->ram_bytes = split->len;
                                split->orig_start = split->start;
                                split->block_len = 0;
-                               split->disk_num_bytes = 0;
                        }
 
                        if (extent_map_in_tree(em)) {
@@ -987,6 +1051,9 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        /* First, replace the em with a new extent_map starting from * em->start */
        split_pre->start = em->start;
        split_pre->len = pre;
+       split_pre->disk_bytenr = new_logical;
+       split_pre->disk_num_bytes = split_pre->len;
+       split_pre->offset = 0;
        split_pre->orig_start = split_pre->start;
        split_pre->block_start = new_logical;
        split_pre->block_len = split_pre->len;
@@ -1005,10 +1072,12 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        /* Insert the middle extent_map. */
        split_mid->start = em->start + pre;
        split_mid->len = em->len - pre;
+       split_mid->disk_bytenr = em->block_start + pre;
+       split_mid->disk_num_bytes = split_mid->len;
+       split_mid->offset = 0;
        split_mid->orig_start = split_mid->start;
        split_mid->block_start = em->block_start + pre;
        split_mid->block_len = split_mid->len;
-       split_mid->disk_num_bytes = split_mid->block_len;
        split_mid->ram_bytes = split_mid->len;
        split_mid->flags = flags;
        split_mid->generation = em->generation;
fs/btrfs/extent_map.h +0 -0
index 6ea0287b0d61..cc9c8092b704 100644

fs/btrfs/extent_map.h struct extent_map { · line 74

Quoted 5 lines
fs/btrfs/extent_map.h +2 -0
@@ -70,12 +70,29 @@ struct extent_map {
         */
        u64 orig_start;
 
+       /*
+        * The bytenr for of the full on-disk extent.

for of -> of

Quoted 116 lines
fs/btrfs/extent_map.h +15 -0
+        *
+        * For regular extents it's btrfs_file_extent_item::disk_bytenr.
+        * For holes it's EXTENT_MAP_HOLE and for inline extents it's
+        * EXTENT_MAP_INLINE.
+        */
+       u64 disk_bytenr;
+
        /*
         * The full on-disk extent length, matching
         * btrfs_file_extent_item::disk_num_bytes.
         */
        u64 disk_num_bytes;
 
+       /*
+        * Offset inside the decompressed extent.
+        *
+        * For regular extents it's btrfs_file_extent_item::offset.
+        * For holes and inline extents it's 0.
+        */
+       u64 offset;
+
        /*
         * The decompressed size of the whole on-disk extent, matching
         * btrfs_file_extent_item::ram_bytes.
fs/btrfs/file-item.c +8 -1
index 2197cfe5443b..47bd4fe0a44b 100644
@@ -1294,12 +1294,17 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                em->len = btrfs_file_extent_end(path) - extent_start;
                em->orig_start = extent_start -
                        btrfs_file_extent_offset(leaf, fi);
-               em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
                bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
                if (bytenr == 0) {
                        em->block_start = EXTENT_MAP_HOLE;
+                       em->disk_bytenr = EXTENT_MAP_HOLE;
+                       em->disk_num_bytes = 0;
+                       em->offset = 0;
                        return;
                }
+               em->disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+               em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
+               em->offset = btrfs_file_extent_offset(leaf, fi);
                if (compress_type != BTRFS_COMPRESS_NONE) {
                        extent_map_set_compression(em, compress_type);
                        em->block_start = bytenr;
@@ -1316,8 +1321,10 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                ASSERT(extent_start == 0);
 
                em->block_start = EXTENT_MAP_INLINE;
+               em->disk_bytenr = EXTENT_MAP_INLINE;
                em->start = 0;
                em->len = fs_info->sectorsize;
+               em->offset = 0;
                /*
                 * Initialize orig_start and block_len with the same values
                 * as in inode.c:btrfs_get_extent().
fs/btrfs/file.c +1 -0
index 63a13a4cace0..8931eeee199d 100644
@@ -2337,6 +2337,7 @@ static int fill_holes(struct btrfs_trans_handle *trans,
                hole_em->orig_start = offset;
 
                hole_em->block_start = EXTENT_MAP_HOLE;
+               hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                hole_em->block_len = 0;
                hole_em->disk_num_bytes = 0;
                hole_em->generation = trans->transid;
fs/btrfs/inode.c +11 -1
index 2815b72f2d85..42fea12d509f 100644
@@ -139,8 +139,9 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
                                     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 orig_start, u64 block_start,
-                                      u64 block_len, u64 orig_block_len,
+                                      u64 block_len, u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
+                                      struct btrfs_file_extent *file_extent,
                                       int type);
 
 static int data_reloc_print_warning_inode(u64 inum, u64 offset, u64 num_bytes,
@@ -1152,6 +1153,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        struct btrfs_root *root = inode->root;
        struct btrfs_fs_info *fs_info = root->fs_info;
        struct btrfs_ordered_extent *ordered;
+       struct btrfs_file_extent file_extent = { 0 };
        struct btrfs_key ins;
        struct page *locked_page = NULL;
        struct extent_state *cached = NULL;
@@ -1198,6 +1200,13 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        lock_extent(io_tree, start, end, &cached);
 
        /* Here we're doing allocation and writeback of the compressed pages */
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.ram_bytes = async_extent->ram_size;
+       file_extent.num_bytes = async_extent->ram_size;
+       file_extent.offset = 0;
+       file_extent.compression = async_extent->compress_type;
+
        em = create_io_em(inode, start,
                          async_extent->ram_size,       /* len */
                          start,                        /* orig_start */
@@ -1206,6 +1215,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
                          ins.offset,                   /* orig_block_len */
                          async_extent->ram_size,       /* ram_bytes */
                          async_extent->compress_type,
+                         &file_extent,
                          BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);

fs/btrfs/inode.c static noinline int cow_file_range(struct btrfs_inode *inode, · line 1408

Quoted 4 lines
fs/btrfs/inode.c +1 -0
@@ -1395,6 +1405,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
 
        while (num_bytes > 0) {
                struct btrfs_ordered_extent *ordered;
+               struct btrfs_file_extent file_extent = { 0 };

Unnecessary initialization, see below.

fs/btrfs/inode.c static noinline int cow_file_range(struct btrfs_inode *inode, · line 1450

Quoted 13 lines
fs/btrfs/inode.c +6 -0
                cur_alloc_size = num_bytes;
                ret = btrfs_reserve_extent(root, cur_alloc_size, cur_alloc_size,
@@ -1431,6 +1442,12 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                extent_reserved = true;
 
                ram_size = ins.offset;
+               file_extent.disk_bytenr = ins.objectid;
+               file_extent.disk_num_bytes = ins.offset;
+               file_extent.num_bytes = ins.offset;
+               file_extent.ram_bytes = ins.offset;
+               file_extent.offset = 0;
+               file_extent.compression = BTRFS_COMPRESS_NONE;

If we always have to initialize all the members of the structure, it's pointless to have it initialized to zeros when we declared it.

Quoted 12 lines
fs/btrfs/inode.c +1 -0
                lock_extent(&inode->io_tree, start, start + ram_size - 1,
                            &cached);
@@ -1442,6 +1459,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                                  ins.offset, /* orig_block_len */
                                  ram_size, /* ram_bytes */
                                  BTRFS_COMPRESS_NONE, /* compress_type */
+                                 &file_extent,
                                  BTRFS_ORDERED_REGULAR /* type */);
                if (IS_ERR(em)) {
                        unlock_extent(&inode->io_tree, start,

fs/btrfs/inode.c struct can_nocow_file_extent_args { · line 1876

Quoted 4 lines
fs/btrfs/inode.c +1 -0
@@ -1855,6 +1873,7 @@ struct can_nocow_file_extent_args {
        u64 disk_bytenr;
        u64 disk_num_bytes;
        u64 extent_offset;
+

Unrelated white space change.

Quoted 55 lines
fs/btrfs/inode.c +7 -2
        /* Number of bytes that can be written to in NOCOW mode. */
        u64 num_bytes;
 
@@ -2182,6 +2201,7 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                                          nocow_args.num_bytes, /* block_len */
                                          nocow_args.disk_num_bytes, /* orig_block_len */
                                          ram_bytes, BTRFS_COMPRESS_NONE,
+                                         &nocow_args.file_extent,
                                          BTRFS_ORDERED_PREALLOC);
                        if (IS_ERR(em)) {
                                unlock_extent(&inode->io_tree, cur_offset,
@@ -4982,6 +5002,7 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
                        hole_em->orig_start = cur_offset;
 
                        hole_em->block_start = EXTENT_MAP_HOLE;
+                       hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                        hole_em->block_len = 0;
                        hole_em->disk_num_bytes = 0;
                        hole_em->ram_bytes = hole_size;
@@ -6842,6 +6863,7 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
        }
        em->start = EXTENT_MAP_HOLE;
        em->orig_start = EXTENT_MAP_HOLE;
+       em->disk_bytenr = EXTENT_MAP_HOLE;
        em->len = (u64)-1;
        em->block_len = (u64)-1;
 
@@ -7007,7 +7029,8 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                                                  const u64 block_len,
                                                  const u64 orig_block_len,
                                                  const u64 ram_bytes,
-                                                 const int type)
+                                                 const int type,
+                                                 struct btrfs_file_extent *file_extent)
 {
        struct extent_map *em = NULL;
        struct btrfs_ordered_extent *ordered;
@@ -7016,7 +7039,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                em = create_io_em(inode, start, len, orig_start, block_start,
                                  block_len, orig_block_len, ram_bytes,
                                  BTRFS_COMPRESS_NONE, /* compress_type */
-                                 type);
+                                 file_extent, type);
                if (IS_ERR(em))
                        goto out;
        }
@@ -7047,6 +7070,7 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
 {
        struct btrfs_root *root = inode->root;
        struct btrfs_fs_info *fs_info = root->fs_info;
+       struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em;
        struct btrfs_key ins;
        u64 alloc_hint;

fs/btrfs/inode.c static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode, · line 7097

Quoted 9 lines
fs/btrfs/inode.c +6 -0
@@ -7065,9 +7089,16 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
        if (ret)
                return ERR_PTR(ret);
 
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = ins.offset;
+       file_extent.ram_bytes = ins.offset;
+       file_extent.offset = 0;
+       file_extent.compression = BTRFS_COMPRESS_NONE;

Same as before:

"If we always have to initialize all the members of the structure, it's pointless to have it initialized to zeros when we declared it."

Quoted 74 lines
fs/btrfs/inode.c +13 -3
        em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset, start,
                                     ins.objectid, ins.offset, ins.offset,
-                                    ins.offset, BTRFS_ORDERED_REGULAR);
+                                    ins.offset, BTRFS_ORDERED_REGULAR,
+                                    &file_extent);
        btrfs_dec_block_group_reservations(fs_info, ins.objectid);
        if (IS_ERR(em))
                btrfs_free_reserved_extent(fs_info, ins.objectid, ins.offset,
@@ -7310,6 +7341,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 orig_start, u64 block_start,
                                       u64 block_len, u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
+                                      struct btrfs_file_extent *file_extent,
                                       int type)
 {
        struct extent_map *em;
@@ -7367,9 +7399,11 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
        em->len = len;
        em->block_len = block_len;
        em->block_start = block_start;
+       em->disk_bytenr = file_extent->disk_bytenr;
        em->disk_num_bytes = disk_num_bytes;
        em->ram_bytes = ram_bytes;
        em->generation = -1;
+       em->offset = file_extent->offset;
        em->flags |= EXTENT_FLAG_PINNED;
        if (type == BTRFS_ORDERED_COMPRESSED)
                extent_map_set_compression(em, compress_type);
@@ -7393,6 +7427,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 {
        const bool nowait = (iomap_flags & IOMAP_NOWAIT);
        struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
+       struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em = *map;
        int type;
        u64 block_start, orig_start, orig_block_len, ram_bytes;
@@ -7423,7 +7458,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                block_start = em->block_start + (start - em->start);
 
                if (can_nocow_extent(inode, start, &len, &orig_start,
-                                    &orig_block_len, &ram_bytes, NULL, false, false) == 1) {
+                                    &orig_block_len, &ram_bytes,
+                                    &file_extent, false, false) == 1) {
                        bg = btrfs_inc_nocow_writers(fs_info, block_start);
                        if (bg)
                                can_nocow = true;
@@ -7451,7 +7487,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
                                              orig_start, block_start,
                                              len, orig_block_len,
-                                             ram_bytes, type);
+                                             ram_bytes, type,
+                                             &file_extent);
                btrfs_dec_nocow_writers(bg);
                if (type == BTRFS_ORDERED_PREALLOC) {
                        free_extent_map(em);
@@ -9602,6 +9639,8 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
                em->orig_start = cur_offset;
                em->len = ins.offset;
                em->block_start = ins.objectid;
+               em->disk_bytenr = ins.objectid;
+               em->offset = 0;
                em->block_len = ins.offset;
                em->disk_num_bytes = ins.offset;
                em->ram_bytes = ins.offset;
@@ -10168,6 +10207,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
        struct extent_changeset *data_reserved = NULL;
        struct extent_state *cached_state = NULL;
        struct btrfs_ordered_extent *ordered;
+       struct btrfs_file_extent file_extent = { 0 };
        int compression;
        size_t orig_count;
        u64 start, end;

fs/btrfs/inode.c ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from, · line 10391

Quoted 9 lines
fs/btrfs/inode.c +6 -0
@@ -10343,10 +10383,16 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
                goto out_delalloc_release;
        extent_reserved = true;
 
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = num_bytes;
+       file_extent.ram_bytes = ram_bytes;
+       file_extent.offset = encoded->unencoded_offset;
+       file_extent.compression = compression;

Same as before:

"If we always have to initialize all the members of the structure, it's pointless to have it initialized to zeros when we declared it."

The rest I suppose seems fine, but I have to look at the rest of the patchset.

Thanks.

Quoted 12 lines
fs/btrfs/inode.c +1 -2
        em = create_io_em(inode, start, num_bytes,
                          start - encoded->unencoded_offset, ins.objectid,
                          ins.offset, ins.offset, ram_bytes, compression,
-                         BTRFS_ORDERED_COMPRESSED);
+                         &file_extent, BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
                goto out_free_reserved;
--

2.45.0

#8 lore raw

Re: [PATCH v2 03/11] btrfs: introduce new members for extent_map

Quoted 18 lines · Filipe Manana
Quoted 17 lines · Qu Wenruo

Introduce two new members for extent_map:

- disk_bytenr
- offset

Both are matching the members with the same name inside btrfs_file_extent_items.

For now this patch only touches those members when:

- Reading btrfs_file_extent_items from disk
- Inserting new holes
- Merging two extent maps
With the new disk_bytenr and disk_num_bytes, doing merging would be a little complex, as we have 3 different cases:

Quoted 8 lines
Quoted 4 lines

* Both extent maps are referring to the same data extent
* Both extent maps are referring to different data extents, but those data extents are adjacent, and extent maps are at head/tail of each data extents

I have no idea what this last part of the sentence means:

"and extent maps are at head/tail of each data extents"

For this case:

        |<- data extent 1 ->|<- data extent 2 ->|
	|          |////////|////////|          |
                     FE A      FE B

In above case, FE A is only referring the tailing part of data extent 1, and FE B is referring to the heading part of data extent 2.

In that case, FE A and FE B can be merged into something like this:

        |<------ merged extent 1 + 2 --------->|
	|          |/////////////////|         |

FE A + B

In that case, merged file extent would have:

- disk_bytenr = fe_a->disk_bytenr
- disk_num_bytes = fe_a->disk_num_bytes + fe_b->disk_num_bytes
- ram_bytes = fe_a->ram_bytes + fe_b->ram_bytes
- offset = fe_a->offset
- num_bytes = fe_a->num_bytes + fe_b->num_bytes

Quoted 6 lines
Quoted 2 lines

* One of the extent map is referring to an merged and larger data

map -> maps
an -> a

Quoted 6 lines
Quoted 1 line

extent that covers both extent maps

Not sure if I can understand this. How can one of the extent maps already cover the range of the other extent map?
This suggests some overlapping, or most likely I misunderstood what this is trying to say.

That's the for impossible test case 3, but as you mentioned, it's more sane just to remove it.

[...]

Quoted 14 lines

fs/btrfs/extent_map.c static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma · line 273

Quoted 13 lines
fs/btrfs/extent_map.c +12 -0
+        *
+        * The calculation here always merge the data extents first, then update
+        * @offset using the new data extents.
+        *
+        * For case 1), the merged data extent would be the same.
+        * For case 2), we just merge the two data extents into one.
+        * For case 3), we just got the larger data extent.
+        */
+       new_disk_bytenr = min(prev->disk_bytenr, next->disk_bytenr);
+       new_disk_num_bytes = max(prev->disk_bytenr + prev->disk_num_bytes,
+                                next->disk_bytenr + next->disk_num_bytes) -
+                            new_disk_bytenr;
Quoted 2 lines

So this is confusing, disk_num_bytes being a max between the two extent maps and not their sum.

Check case 1).

Both file extents are referring to the same data extent.

Like this:

FE 1, file pos 0, num_bytes 4K, disk_bytenr X, disk_num_bytes 16K, offset 0, ram_bytes 16K, compression none

FE 2, file pos 4k, num_bytes 4K, disk_bytenr X, disk_num_bytes 16K, offset 4k, ram_bytes 16K, compression none.

In that case we should not just sum up the disk_num_bytes at all. Remember disk_num_bytes are for the data extent.

Quoted 17 lines

I gather this is modelled after what we already do at btrfs_drop_extent_map_range() when splitting.

But the truth is we never used the disk_num_bytes of an extent map that was merged - we also didn't do it before this patch, for that reason.
It's only used for logging new extents - which can't be merged - they can be merged only after being logged.

We can set this to the sum, or leave with some value to signal it's invalid. Just leave a comment saying disk_num_bytes it's not used anywhere for merged extent maps.

In the splitting case at btrfs_drop_extent_map_range() it's what we need since in the case the extent is new and not logged (in the modified list), disk_num_bytes always represents the size of the original, before split, extent.

[...]

Quoted 12 lines

fs/btrfs/inode.c static noinline int cow_file_range(struct btrfs_inode *inode, · line 1450

Quoted 6 lines
fs/btrfs/inode.c +6 -0
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = ins.offset;
+       file_extent.ram_bytes = ins.offset;
+       file_extent.offset = 0;
+       file_extent.compression = BTRFS_COMPRESS_NONE;

Same as before:

"If we always have to initialize all the members of the structure, it's pointless to have it initialized to zeros when we declared it."

[...]

Quoted 11 lines

fs/btrfs/inode.c ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from, · line 10391

Quoted 6 lines
fs/btrfs/inode.c +6 -0
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = num_bytes;
+       file_extent.ram_bytes = ram_bytes;
+       file_extent.offset = encoded->unencoded_offset;
+       file_extent.compression = compression;

Same as before:

"If we always have to initialize all the members of the structure, it's pointless to have it initialized to zeros when we declared it."

Fair enough, and an uninitilized structure member can also make compiler to warn us.

Thanks,
Qu

Quoted 18 lines

The rest I suppose seems fine, but I have to look at the rest of the patchset.

Thanks.

Quoted 12 lines

em = create_io_em(inode, start, num_bytes,
start - encoded->unencoded_offset, ins.objectid, ins.offset, ins.offset, ram_bytes, compression,

fs/btrfs/inode.c +1 -2
-                         BTRFS_ORDERED_COMPRESSED);
+                         &file_extent, BTRFS_ORDERED_COMPRESSED);
         if (IS_ERR(em)) {
                 ret = PTR_ERR(em);
                 goto out_free_reserved;
--

2.45.0

#9 lore raw

Re: [PATCH v2 03/11] btrfs: introduce new members for extent_map

Quoted 94 lines · Qu Wenruo
Quoted 18 lines · Filipe Manana
Quoted 17 lines · Qu Wenruo

Introduce two new members for extent_map:

- disk_bytenr
- offset

Both are matching the members with the same name inside btrfs_file_extent_items.

For now this patch only touches those members when:

- Reading btrfs_file_extent_items from disk
- Inserting new holes
- Merging two extent maps
With the new disk_bytenr and disk_num_bytes, doing merging would be a little complex, as we have 3 different cases:

Quoted 8 lines
Quoted 4 lines

* Both extent maps are referring to the same data extent
* Both extent maps are referring to different data extents, but those data extents are adjacent, and extent maps are at head/tail of each data extents

I have no idea what this last part of the sentence means:

"and extent maps are at head/tail of each data extents"

For this case:

         |<- data extent 1 ->|<- data extent 2 ->|
        |          |////////|////////|          |
                      FE A      FE B

In above case, FE A is only referring the tailing part of data extent 1, and FE B is referring to the heading part of data extent 2.

In that case, FE A and FE B can be merged into something like this:

         |<------ merged extent 1 + 2 --------->|
        |          |/////////////////|         |

FE A + B

In that case, merged file extent would have:

- disk_bytenr = fe_a->disk_bytenr
- disk_num_bytes = fe_a->disk_num_bytes + fe_b->disk_num_bytes
- ram_bytes = fe_a->ram_bytes + fe_b->ram_bytes
- offset = fe_a->offset
- num_bytes = fe_a->num_bytes + fe_b->num_bytes

Quoted 6 lines
Quoted 2 lines

* One of the extent map is referring to an merged and larger data

map -> maps
an -> a

Quoted 6 lines
Quoted 1 line

extent that covers both extent maps

Not sure if I can understand this. How can one of the extent maps already cover the range of the other extent map?
This suggests some overlapping, or most likely I misunderstood what this is trying to say.

That's the for impossible test case 3, but as you mentioned, it's more sane just to remove it.

[...]

Quoted 14 lines

fs/btrfs/extent_map.c static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma · line 273

Quoted 13 lines
fs/btrfs/extent_map.c +12 -0
+        *
+        * The calculation here always merge the data extents first, then update
+        * @offset using the new data extents.
+        *
+        * For case 1), the merged data extent would be the same.
+        * For case 2), we just merge the two data extents into one.
+        * For case 3), we just got the larger data extent.
+        */
+       new_disk_bytenr = min(prev->disk_bytenr, next->disk_bytenr);
+       new_disk_num_bytes = max(prev->disk_bytenr + prev->disk_num_bytes,
+                                next->disk_bytenr + next->disk_num_bytes) -
+                            new_disk_bytenr;
Quoted 2 lines

So this is confusing, disk_num_bytes being a max between the two extent maps and not their sum.

Check case 1).

Both file extents are referring to the same data extent.

Like this:

Quoted 8 lines

FE 1, file pos 0, num_bytes 4K, disk_bytenr X, disk_num_bytes 16K, offset 0, ram_bytes 16K, compression none

FE 2, file pos 4k, num_bytes 4K, disk_bytenr X, disk_num_bytes 16K, offset 4k, ram_bytes 16K, compression none.

In that case we should not just sum up the disk_num_bytes at all. Remember disk_num_bytes are for the data extent.

Yes, but for cases where they refer to different file extent items that are contiguous on disk, the max is confusing, and a sum is what makes sense. Example:

FE 1, file pos 0, num_bytes 4K, disk_bytenr X, disk_num_bytes 4K, offset 0, ram_bytes 4K, no compression

FE 2, file pos 4K, num_bytes 4K, disk_bytenr X + 4K, disk_num_bytes 4K, offset 0, ram_bytes 4K, no compression

When merging the corresponding extent maps it's natural to think disk_num_bytes is 8K and not 4K.

But as I mentioned before, after merging we really don't use disk_num_bytes anywhere.
It's only used during extent logging, which only happens for extents maps that were not merged and can not be before they are logged.

Before this patchset, when disk_num_bytes was named orig_block_len, that was never updated during merging, exactly because we don't use cases for it.

Quoted 69 lines
Quoted 18 lines

I gather this is modelled after what we already do at btrfs_drop_extent_map_range() when splitting.

But the truth is we never used the disk_num_bytes of an extent map that was merged - we also didn't do it before this patch, for that reason.
It's only used for logging new extents - which can't be merged - they can be merged only after being logged.

We can set this to the sum, or leave with some value to signal it's invalid. Just leave a comment saying disk_num_bytes it's not used anywhere for merged extent maps.

In the splitting case at btrfs_drop_extent_map_range() it's what we need since in the case the extent is new and not logged (in the modified list), disk_num_bytes always represents the size of the original, before split, extent.

[...]

Quoted 12 lines

fs/btrfs/inode.c static noinline int cow_file_range(struct btrfs_inode *inode, · line 1450

Quoted 6 lines
fs/btrfs/inode.c +6 -0
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = ins.offset;
+       file_extent.ram_bytes = ins.offset;
+       file_extent.offset = 0;
+       file_extent.compression = BTRFS_COMPRESS_NONE;

Same as before:

"If we always have to initialize all the members of the structure, it's pointless to have it initialized to zeros when we declared it."

[...]

Quoted 11 lines

fs/btrfs/inode.c ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from, · line 10391

Quoted 6 lines
fs/btrfs/inode.c +6 -0
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = num_bytes;
+       file_extent.ram_bytes = ram_bytes;
+       file_extent.offset = encoded->unencoded_offset;
+       file_extent.compression = compression;

Same as before:

"If we always have to initialize all the members of the structure, it's pointless to have it initialized to zeros when we declared it."

Fair enough, and an uninitilized structure member can also make compiler to warn us.

Thanks,
Qu

Quoted 18 lines

The rest I suppose seems fine, but I have to look at the rest of the patchset.

Thanks.

Quoted 12 lines

em = create_io_em(inode, start, num_bytes,
start - encoded->unencoded_offset, ins.objectid, ins.offset, ins.offset, ram_bytes, compression,

fs/btrfs/inode.c +1 -2
-                         BTRFS_ORDERED_COMPRESSED);
+                         &file_extent, BTRFS_ORDERED_COMPRESSED);
         if (IS_ERR(em)) {
                 ret = PTR_ERR(em);
                 goto out_free_reserved;
--

2.45.0

author
#10 lore raw

Re: [PATCH v2 03/11] btrfs: introduce new members for extent_map

在 2024/5/10 20:56, Filipe Manana 写道:
[...]

Quoted 20 lines
Quoted 7 lines

Check case 1).

Both file extents are referring to the same data extent.

Like this:

Quoted 8 lines

FE 1, file pos 0, num_bytes 4K, disk_bytenr X, disk_num_bytes 16K, offset 0, ram_bytes 16K, compression none

FE 2, file pos 4k, num_bytes 4K, disk_bytenr X, disk_num_bytes 16K, offset 4k, ram_bytes 16K, compression none.

In that case we should not just sum up the disk_num_bytes at all. Remember disk_num_bytes are for the data extent.

Yes, but for cases where they refer to different file extent items that are contiguous on disk, the max is confusing, and a sum is what makes sense. Example:

Quoted 8 lines

FE 1, file pos 0, num_bytes 4K, disk_bytenr X, disk_num_bytes 4K, offset 0, ram_bytes 4K, no compression

FE 2, file pos 4K, num_bytes 4K, disk_bytenr X + 4K, disk_num_bytes 4K, offset 0, ram_bytes 4K, no compression

When merging the corresponding extent maps it's natural to think disk_num_bytes is 8K and not 4K.

The max based solution is based on their end bytenr, not disk_num_bytes.

So the merged one would have:

disk_bytenr = min(X, X + 4K) = X
disk_num_bytes = max(X + 4K, X + 4K + 4K) - disk_bytenr = 8K.

So I didn't see why it's not natural.

Furthermore, the max/min based solution can handle both case 1) (same data extent, different parts) and case 2) (different data extents) well.

Quoted 9 lines

But as I mentioned before, after merging we really don't use disk_num_bytes anywhere.
It's only used during extent logging, which only happens for extents maps that were not merged and can not be before they are logged.

Before this patchset, when disk_num_bytes was named orig_block_len, that was never updated during merging, exactly because we don't use cases for it.

But since we want everything to match the definition of the same-named member, no matter if they get merged or not, we need to change the values.

If they do not get used, that's still fine.

Thanks,
Qu

#11 lore raw

Re: [PATCH v2 03/11] btrfs: introduce new members for extent_map

In reply to Qu Wenruo
Quoted 320 lines · Qu Wenruo

Introduce two new members for extent_map:

- disk_bytenr
- offset

Both are matching the members with the same name inside btrfs_file_extent_items.

For now this patch only touches those members when:

- Reading btrfs_file_extent_items from disk
- Inserting new holes
- Merging two extent maps
With the new disk_bytenr and disk_num_bytes, doing merging would be a little complex, as we have 3 different cases:

* Both extent maps are referring to the same data extent
* Both extent maps are referring to different data extents, but those data extents are adjacent, and extent maps are at head/tail of each data extents
* One of the extent map is referring to an merged and larger data extent that covers both extent maps

The 3rd case seems only valid in selftest (test_case_3()), but a new helper merge_ondisk_extents() should be able to handle all of them.

To properly assign values for those new members, a new btrfs_file_extent parameter is introduced to all the involved call sites.

- For NOCOW writes the btrfs_file_extent would be exposed from can_nocow_file_extent().

- For other writes, the members can be easily calculated
As most of them have 0 offset and utilizing the whole on-disk data extent.
The exception is encoded write, but thankfully that interface provided offset directly and all other needed info.

For now, both the old members (block_start/block_len/orig_start) are co-existing with the new members (disk_bytenr/offset), meanwhile all the critical code is still using the old members only.

The cleanup would happen later after all the older and newer members are properly validated.

---
 fs/btrfs/defrag.c     |  4 +++
 fs/btrfs/extent_map.c | 75 +++++++++++++++++++++++++++++++++++++++++--
 fs/btrfs/extent_map.h | 17 ++++++++++
 fs/btrfs/file-item.c  |  9 +++++-
 fs/btrfs/file.c       |  1 +
 fs/btrfs/inode.c      | 60 ++++++++++++++++++++++++++++++----
 6 files changed, 155 insertions(+), 11 deletions(-)
fs/btrfs/defrag.c +4 -0
index 407ccec3e57e..242c5469f4ba 100644
@@ -709,6 +709,10 @@ static struct extent_map *defrag_get_extent(struct btrfs_inode *inode,
                        em->start = start;
                        em->orig_start = start;
                        em->block_start = EXTENT_MAP_HOLE;
+                       em->disk_bytenr = EXTENT_MAP_HOLE;
+                       em->disk_num_bytes = 0;
+                       em->ram_bytes = 0;
+                       em->offset = 0;
                        em->len = key.offset - start;
                        break;
                }
fs/btrfs/extent_map.c +72 -3
index 4230dd0f34cc..4d4ac9fc43e2 100644
@@ -232,6 +232,58 @@ static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma
        return next->block_start == prev->block_start;
 }
 
+/*
+ * Handle the ondisk data extents merge for @prev and @next.
+ *
+ * Only touches disk_bytenr/disk_num_bytes/offset/ram_bytes.
+ * For now only uncompressed regular extent can be merged.
+ *
+ * @prev and @next will be both updated to point to the new merged range.
+ * Thus one of them should be removed by the caller.
+ */
+static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *next)
+{
+       u64 new_disk_bytenr;
+       u64 new_disk_num_bytes;
+       u64 new_offset;
+
+       /* @prev and @next should not be compressed. */
+       ASSERT(!extent_map_is_compressed(prev));
+       ASSERT(!extent_map_is_compressed(next));
+
+       /*
+        * There are several different cases that @prev and @next can be merged.
+        *
+        * 1) They are referring to the same data extent
+        * 2) Their ondisk data extents are adjacent and @prev is the tail
+        *    and @next is the head of their data extents
+        * 3) One of @prev/@next is referrring to a larger merged data extent.
+        *    (test_case_3 of extent maps tests).
+        *
+        * The calculation here always merge the data extents first, then update
+        * @offset using the new data extents.
+        *
+        * For case 1), the merged data extent would be the same.
+        * For case 2), we just merge the two data extents into one.
+        * For case 3), we just got the larger data extent.
+        */
+       new_disk_bytenr = min(prev->disk_bytenr, next->disk_bytenr);
+       new_disk_num_bytes = max(prev->disk_bytenr + prev->disk_num_bytes,
+                                next->disk_bytenr + next->disk_num_bytes) -
+                            new_disk_bytenr;
+       new_offset = prev->disk_bytenr + prev->offset - new_disk_bytenr;
+
+       prev->disk_bytenr = new_disk_bytenr;
+       prev->disk_num_bytes = new_disk_num_bytes;
+       prev->ram_bytes = new_disk_num_bytes;
+       prev->offset = new_offset;
+
+       next->disk_bytenr = new_disk_bytenr;
+       next->disk_num_bytes = new_disk_num_bytes;
+       next->ram_bytes = new_disk_num_bytes;
+       next->offset = new_offset;
+}
+
 static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
 {
        struct extent_map_tree *tree = &inode->extent_tree;
@@ -263,6 +315,9 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                        em->block_len += merge->block_len;
                        em->block_start = merge->block_start;
                        em->generation = max(em->generation, merge->generation);
+
+                       if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
+                               merge_ondisk_extents(merge, em);
                        em->flags |= EXTENT_FLAG_MERGED;
 
                        rb_erase_cached(&merge->rb_node, &tree->map);
@@ -278,6 +333,8 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
        if (rb && can_merge_extent_map(merge) && mergeable_maps(em, merge)) {
                em->len += merge->len;
                em->block_len += merge->block_len;
+               if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
+                       merge_ondisk_extents(em, merge);
                rb_erase_cached(&merge->rb_node, &tree->map);
                RB_CLEAR_NODE(&merge->rb_node);
                em->generation = max(em->generation, merge->generation);
@@ -565,6 +622,7 @@ static noinline int merge_extent_mapping(struct btrfs_inode *inode,
            !extent_map_is_compressed(em)) {
                em->block_start += start_diff;
                em->block_len = em->len;
+               em->offset += start_diff;
        }
        return add_extent_mapping(inode, em, 0);
 }
@@ -783,14 +841,18 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                        split->block_len = em->block_len;
                                else
                                        split->block_len = split->len;
+                               split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = max(split->block_len,
                                                            em->disk_num_bytes);
+                               split->offset = em->offset;
                                split->ram_bytes = em->ram_bytes;
                        } else {
                                split->orig_start = split->start;
                                split->block_len = 0;
                                split->block_start = em->block_start;
+                               split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = 0;
+                               split->offset = 0;
                                split->ram_bytes = split->len;
                        }
 
@@ -811,13 +873,14 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        split->start = end;
                        split->len = em_end - end;
                        split->block_start = em->block_start;
+                       split->disk_bytenr = em->disk_bytenr;
                        split->flags = flags;
                        split->generation = gen;
 
                        if (em->block_start < EXTENT_MAP_LAST_BYTE) {
                                split->disk_num_bytes = max(em->block_len,
                                                            em->disk_num_bytes);
-
+                               split->offset = em->offset + end - em->start;
                                split->ram_bytes = em->ram_bytes;
                                if (compressed) {
                                        split->block_len = em->block_len;
@@ -830,10 +893,11 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                        split->orig_start = em->orig_start;
                                }
                        } else {
+                               split->disk_num_bytes = 0;
+                               split->offset = 0;
                                split->ram_bytes = split->len;
                                split->orig_start = split->start;
                                split->block_len = 0;
-                               split->disk_num_bytes = 0;
                        }
 
                        if (extent_map_in_tree(em)) {
@@ -987,6 +1051,9 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        /* First, replace the em with a new extent_map starting from * em->start */
        split_pre->start = em->start;
        split_pre->len = pre;
+       split_pre->disk_bytenr = new_logical;
+       split_pre->disk_num_bytes = split_pre->len;
+       split_pre->offset = 0;
        split_pre->orig_start = split_pre->start;
        split_pre->block_start = new_logical;
        split_pre->block_len = split_pre->len;
@@ -1005,10 +1072,12 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        /* Insert the middle extent_map. */
        split_mid->start = em->start + pre;
        split_mid->len = em->len - pre;
+       split_mid->disk_bytenr = em->block_start + pre;
+       split_mid->disk_num_bytes = split_mid->len;
+       split_mid->offset = 0;
        split_mid->orig_start = split_mid->start;
        split_mid->block_start = em->block_start + pre;
        split_mid->block_len = split_mid->len;
-       split_mid->disk_num_bytes = split_mid->block_len;
        split_mid->ram_bytes = split_mid->len;
        split_mid->flags = flags;
        split_mid->generation = em->generation;
fs/btrfs/extent_map.h +17 -0
index 6ea0287b0d61..cc9c8092b704 100644
@@ -70,12 +70,29 @@ struct extent_map {
         */
        u64 orig_start;
 
+       /*
+        * The bytenr for of the full on-disk extent.
+        *
+        * For regular extents it's btrfs_file_extent_item::disk_bytenr.
+        * For holes it's EXTENT_MAP_HOLE and for inline extents it's
+        * EXTENT_MAP_INLINE.
+        */
+       u64 disk_bytenr;
+
        /*
         * The full on-disk extent length, matching
         * btrfs_file_extent_item::disk_num_bytes.
         */
        u64 disk_num_bytes;
 
+       /*
+        * Offset inside the decompressed extent.
+        *
+        * For regular extents it's btrfs_file_extent_item::offset.
+        * For holes and inline extents it's 0.
+        */
+       u64 offset;
+
        /*
         * The decompressed size of the whole on-disk extent, matching
         * btrfs_file_extent_item::ram_bytes.
fs/btrfs/file-item.c +8 -1
index 2197cfe5443b..47bd4fe0a44b 100644
@@ -1294,12 +1294,17 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                em->len = btrfs_file_extent_end(path) - extent_start;
                em->orig_start = extent_start -
                        btrfs_file_extent_offset(leaf, fi);
-               em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
                bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
                if (bytenr == 0) {
                        em->block_start = EXTENT_MAP_HOLE;
+                       em->disk_bytenr = EXTENT_MAP_HOLE;
+                       em->disk_num_bytes = 0;
+                       em->offset = 0;
                        return;
                }
+               em->disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+               em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
+               em->offset = btrfs_file_extent_offset(leaf, fi);
                if (compress_type != BTRFS_COMPRESS_NONE) {
                        extent_map_set_compression(em, compress_type);
                        em->block_start = bytenr;
@@ -1316,8 +1321,10 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                ASSERT(extent_start == 0);
 
                em->block_start = EXTENT_MAP_INLINE;
+               em->disk_bytenr = EXTENT_MAP_INLINE;
                em->start = 0;
                em->len = fs_info->sectorsize;
+               em->offset = 0;
                /*
                 * Initialize orig_start and block_len with the same values
                 * as in inode.c:btrfs_get_extent().
fs/btrfs/file.c +1 -0
index 63a13a4cace0..8931eeee199d 100644
@@ -2337,6 +2337,7 @@ static int fill_holes(struct btrfs_trans_handle *trans,
                hole_em->orig_start = offset;
 
                hole_em->block_start = EXTENT_MAP_HOLE;
+               hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                hole_em->block_len = 0;
                hole_em->disk_num_bytes = 0;
                hole_em->generation = trans->transid;
fs/btrfs/inode.c +0 -0
index 2815b72f2d85..42fea12d509f 100644

fs/btrfs/inode.c static noinline int run_delalloc_cow(struct btrfs_inode *inode, · line 142

Quoted 5 lines
fs/btrfs/inode.c +1 -1
@@ -139,8 +139,9 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
                                     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 orig_start, u64 block_start,
-                                      u64 block_len, u64 orig_block_len,
+                                      u64 block_len, u64 disk_num_bytes,

This hunk should have been in the previous patch, which renames orig_block_len to disk_num_bytes everywhere, including the prototype declaration of this function.

Quoted 228 lines
fs/btrfs/inode.c +52 -7
                                       u64 ram_bytes, int compress_type,
+                                      struct btrfs_file_extent *file_extent,
                                       int type);
 
 static int data_reloc_print_warning_inode(u64 inum, u64 offset, u64 num_bytes,
@@ -1152,6 +1153,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        struct btrfs_root *root = inode->root;
        struct btrfs_fs_info *fs_info = root->fs_info;
        struct btrfs_ordered_extent *ordered;
+       struct btrfs_file_extent file_extent = { 0 };
        struct btrfs_key ins;
        struct page *locked_page = NULL;
        struct extent_state *cached = NULL;
@@ -1198,6 +1200,13 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        lock_extent(io_tree, start, end, &cached);
 
        /* Here we're doing allocation and writeback of the compressed pages */
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.ram_bytes = async_extent->ram_size;
+       file_extent.num_bytes = async_extent->ram_size;
+       file_extent.offset = 0;
+       file_extent.compression = async_extent->compress_type;
+
        em = create_io_em(inode, start,
                          async_extent->ram_size,       /* len */
                          start,                        /* orig_start */
@@ -1206,6 +1215,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
                          ins.offset,                   /* orig_block_len */
                          async_extent->ram_size,       /* ram_bytes */
                          async_extent->compress_type,
+                         &file_extent,
                          BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
@@ -1395,6 +1405,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
 
        while (num_bytes > 0) {
                struct btrfs_ordered_extent *ordered;
+               struct btrfs_file_extent file_extent = { 0 };
 
                cur_alloc_size = num_bytes;
                ret = btrfs_reserve_extent(root, cur_alloc_size, cur_alloc_size,
@@ -1431,6 +1442,12 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                extent_reserved = true;
 
                ram_size = ins.offset;
+               file_extent.disk_bytenr = ins.objectid;
+               file_extent.disk_num_bytes = ins.offset;
+               file_extent.num_bytes = ins.offset;
+               file_extent.ram_bytes = ins.offset;
+               file_extent.offset = 0;
+               file_extent.compression = BTRFS_COMPRESS_NONE;
 
                lock_extent(&inode->io_tree, start, start + ram_size - 1,
                            &cached);
@@ -1442,6 +1459,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                                  ins.offset, /* orig_block_len */
                                  ram_size, /* ram_bytes */
                                  BTRFS_COMPRESS_NONE, /* compress_type */
+                                 &file_extent,
                                  BTRFS_ORDERED_REGULAR /* type */);
                if (IS_ERR(em)) {
                        unlock_extent(&inode->io_tree, start,
@@ -1855,6 +1873,7 @@ struct can_nocow_file_extent_args {
        u64 disk_bytenr;
        u64 disk_num_bytes;
        u64 extent_offset;
+
        /* Number of bytes that can be written to in NOCOW mode. */
        u64 num_bytes;
 
@@ -2182,6 +2201,7 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                                          nocow_args.num_bytes, /* block_len */
                                          nocow_args.disk_num_bytes, /* orig_block_len */
                                          ram_bytes, BTRFS_COMPRESS_NONE,
+                                         &nocow_args.file_extent,
                                          BTRFS_ORDERED_PREALLOC);
                        if (IS_ERR(em)) {
                                unlock_extent(&inode->io_tree, cur_offset,
@@ -4982,6 +5002,7 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
                        hole_em->orig_start = cur_offset;
 
                        hole_em->block_start = EXTENT_MAP_HOLE;
+                       hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                        hole_em->block_len = 0;
                        hole_em->disk_num_bytes = 0;
                        hole_em->ram_bytes = hole_size;
@@ -6842,6 +6863,7 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
        }
        em->start = EXTENT_MAP_HOLE;
        em->orig_start = EXTENT_MAP_HOLE;
+       em->disk_bytenr = EXTENT_MAP_HOLE;
        em->len = (u64)-1;
        em->block_len = (u64)-1;
 
@@ -7007,7 +7029,8 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                                                  const u64 block_len,
                                                  const u64 orig_block_len,
                                                  const u64 ram_bytes,
-                                                 const int type)
+                                                 const int type,
+                                                 struct btrfs_file_extent *file_extent)
 {
        struct extent_map *em = NULL;
        struct btrfs_ordered_extent *ordered;
@@ -7016,7 +7039,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                em = create_io_em(inode, start, len, orig_start, block_start,
                                  block_len, orig_block_len, ram_bytes,
                                  BTRFS_COMPRESS_NONE, /* compress_type */
-                                 type);
+                                 file_extent, type);
                if (IS_ERR(em))
                        goto out;
        }
@@ -7047,6 +7070,7 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
 {
        struct btrfs_root *root = inode->root;
        struct btrfs_fs_info *fs_info = root->fs_info;
+       struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em;
        struct btrfs_key ins;
        u64 alloc_hint;
@@ -7065,9 +7089,16 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
        if (ret)
                return ERR_PTR(ret);
 
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = ins.offset;
+       file_extent.ram_bytes = ins.offset;
+       file_extent.offset = 0;
+       file_extent.compression = BTRFS_COMPRESS_NONE;
        em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset, start,
                                     ins.objectid, ins.offset, ins.offset,
-                                    ins.offset, BTRFS_ORDERED_REGULAR);
+                                    ins.offset, BTRFS_ORDERED_REGULAR,
+                                    &file_extent);
        btrfs_dec_block_group_reservations(fs_info, ins.objectid);
        if (IS_ERR(em))
                btrfs_free_reserved_extent(fs_info, ins.objectid, ins.offset,
@@ -7310,6 +7341,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 orig_start, u64 block_start,
                                       u64 block_len, u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
+                                      struct btrfs_file_extent *file_extent,
                                       int type)
 {
        struct extent_map *em;
@@ -7367,9 +7399,11 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
        em->len = len;
        em->block_len = block_len;
        em->block_start = block_start;
+       em->disk_bytenr = file_extent->disk_bytenr;
        em->disk_num_bytes = disk_num_bytes;
        em->ram_bytes = ram_bytes;
        em->generation = -1;
+       em->offset = file_extent->offset;
        em->flags |= EXTENT_FLAG_PINNED;
        if (type == BTRFS_ORDERED_COMPRESSED)
                extent_map_set_compression(em, compress_type);
@@ -7393,6 +7427,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 {
        const bool nowait = (iomap_flags & IOMAP_NOWAIT);
        struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
+       struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em = *map;
        int type;
        u64 block_start, orig_start, orig_block_len, ram_bytes;
@@ -7423,7 +7458,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                block_start = em->block_start + (start - em->start);
 
                if (can_nocow_extent(inode, start, &len, &orig_start,
-                                    &orig_block_len, &ram_bytes, NULL, false, false) == 1) {
+                                    &orig_block_len, &ram_bytes,
+                                    &file_extent, false, false) == 1) {
                        bg = btrfs_inc_nocow_writers(fs_info, block_start);
                        if (bg)
                                can_nocow = true;
@@ -7451,7 +7487,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
                                              orig_start, block_start,
                                              len, orig_block_len,
-                                             ram_bytes, type);
+                                             ram_bytes, type,
+                                             &file_extent);
                btrfs_dec_nocow_writers(bg);
                if (type == BTRFS_ORDERED_PREALLOC) {
                        free_extent_map(em);
@@ -9602,6 +9639,8 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
                em->orig_start = cur_offset;
                em->len = ins.offset;
                em->block_start = ins.objectid;
+               em->disk_bytenr = ins.objectid;
+               em->offset = 0;
                em->block_len = ins.offset;
                em->disk_num_bytes = ins.offset;
                em->ram_bytes = ins.offset;
@@ -10168,6 +10207,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
        struct extent_changeset *data_reserved = NULL;
        struct extent_state *cached_state = NULL;
        struct btrfs_ordered_extent *ordered;
+       struct btrfs_file_extent file_extent = { 0 };
        int compression;
        size_t orig_count;
        u64 start, end;
@@ -10343,10 +10383,16 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
                goto out_delalloc_release;
        extent_reserved = true;
 
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = num_bytes;
+       file_extent.ram_bytes = ram_bytes;
+       file_extent.offset = encoded->unencoded_offset;
+       file_extent.compression = compression;
        em = create_io_em(inode, start, num_bytes,
                          start - encoded->unencoded_offset, ins.objectid,
                          ins.offset, ins.offset, ram_bytes, compression,
-                         BTRFS_ORDERED_COMPRESSED);
+                         &file_extent, BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
                goto out_free_reserved;
--

2.45.0

#12 lore raw

Re: [PATCH v2 03/11] btrfs: introduce new members for extent_map

Quoted 321 lines · Filipe Manana
Quoted 319 lines · Qu Wenruo

Introduce two new members for extent_map:

- disk_bytenr
- offset

Both are matching the members with the same name inside btrfs_file_extent_items.

For now this patch only touches those members when:

- Reading btrfs_file_extent_items from disk
- Inserting new holes
- Merging two extent maps
With the new disk_bytenr and disk_num_bytes, doing merging would be a little complex, as we have 3 different cases:

* Both extent maps are referring to the same data extent
* Both extent maps are referring to different data extents, but those data extents are adjacent, and extent maps are at head/tail of each data extents
* One of the extent map is referring to an merged and larger data extent that covers both extent maps

The 3rd case seems only valid in selftest (test_case_3()), but a new helper merge_ondisk_extents() should be able to handle all of them.

To properly assign values for those new members, a new btrfs_file_extent parameter is introduced to all the involved call sites.

- For NOCOW writes the btrfs_file_extent would be exposed from can_nocow_file_extent().

- For other writes, the members can be easily calculated
As most of them have 0 offset and utilizing the whole on-disk data extent.
The exception is encoded write, but thankfully that interface provided offset directly and all other needed info.

For now, both the old members (block_start/block_len/orig_start) are co-existing with the new members (disk_bytenr/offset), meanwhile all the critical code is still using the old members only.

The cleanup would happen later after all the older and newer members are properly validated.

---
 fs/btrfs/defrag.c     |  4 +++
 fs/btrfs/extent_map.c | 75 +++++++++++++++++++++++++++++++++++++++++--
 fs/btrfs/extent_map.h | 17 ++++++++++
 fs/btrfs/file-item.c  |  9 +++++-
 fs/btrfs/file.c       |  1 +
 fs/btrfs/inode.c      | 60 ++++++++++++++++++++++++++++++----
 6 files changed, 155 insertions(+), 11 deletions(-)
fs/btrfs/defrag.c +4 -0
index 407ccec3e57e..242c5469f4ba 100644
@@ -709,6 +709,10 @@ static struct extent_map *defrag_get_extent(struct btrfs_inode *inode,
                        em->start = start;
                        em->orig_start = start;
                        em->block_start = EXTENT_MAP_HOLE;
+                       em->disk_bytenr = EXTENT_MAP_HOLE;
+                       em->disk_num_bytes = 0;
+                       em->ram_bytes = 0;
+                       em->offset = 0;
                        em->len = key.offset - start;
                        break;
                }
fs/btrfs/extent_map.c +72 -3
index 4230dd0f34cc..4d4ac9fc43e2 100644
@@ -232,6 +232,58 @@ static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma
        return next->block_start == prev->block_start;
 }
 
+/*
+ * Handle the ondisk data extents merge for @prev and @next.
+ *
+ * Only touches disk_bytenr/disk_num_bytes/offset/ram_bytes.
+ * For now only uncompressed regular extent can be merged.
+ *
+ * @prev and @next will be both updated to point to the new merged range.
+ * Thus one of them should be removed by the caller.
+ */
+static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *next)
+{
+       u64 new_disk_bytenr;
+       u64 new_disk_num_bytes;
+       u64 new_offset;
+
+       /* @prev and @next should not be compressed. */
+       ASSERT(!extent_map_is_compressed(prev));
+       ASSERT(!extent_map_is_compressed(next));
+
+       /*
+        * There are several different cases that @prev and @next can be merged.
+        *
+        * 1) They are referring to the same data extent
+        * 2) Their ondisk data extents are adjacent and @prev is the tail
+        *    and @next is the head of their data extents
+        * 3) One of @prev/@next is referrring to a larger merged data extent.
+        *    (test_case_3 of extent maps tests).
+        *
+        * The calculation here always merge the data extents first, then update
+        * @offset using the new data extents.
+        *
+        * For case 1), the merged data extent would be the same.
+        * For case 2), we just merge the two data extents into one.
+        * For case 3), we just got the larger data extent.
+        */
+       new_disk_bytenr = min(prev->disk_bytenr, next->disk_bytenr);
+       new_disk_num_bytes = max(prev->disk_bytenr + prev->disk_num_bytes,
+                                next->disk_bytenr + next->disk_num_bytes) -
+                            new_disk_bytenr;
+       new_offset = prev->disk_bytenr + prev->offset - new_disk_bytenr;
+
+       prev->disk_bytenr = new_disk_bytenr;
+       prev->disk_num_bytes = new_disk_num_bytes;
+       prev->ram_bytes = new_disk_num_bytes;
+       prev->offset = new_offset;
+
+       next->disk_bytenr = new_disk_bytenr;
+       next->disk_num_bytes = new_disk_num_bytes;
+       next->ram_bytes = new_disk_num_bytes;
+       next->offset = new_offset;
+}
+
 static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
 {
        struct extent_map_tree *tree = &inode->extent_tree;
@@ -263,6 +315,9 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                        em->block_len += merge->block_len;
                        em->block_start = merge->block_start;
                        em->generation = max(em->generation, merge->generation);
+
+                       if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
+                               merge_ondisk_extents(merge, em);
                        em->flags |= EXTENT_FLAG_MERGED;
 
                        rb_erase_cached(&merge->rb_node, &tree->map);
@@ -278,6 +333,8 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
        if (rb && can_merge_extent_map(merge) && mergeable_maps(em, merge)) {
                em->len += merge->len;
                em->block_len += merge->block_len;
+               if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
+                       merge_ondisk_extents(em, merge);
                rb_erase_cached(&merge->rb_node, &tree->map);
                RB_CLEAR_NODE(&merge->rb_node);
                em->generation = max(em->generation, merge->generation);
@@ -565,6 +622,7 @@ static noinline int merge_extent_mapping(struct btrfs_inode *inode,
            !extent_map_is_compressed(em)) {
                em->block_start += start_diff;
                em->block_len = em->len;
+               em->offset += start_diff;
        }
        return add_extent_mapping(inode, em, 0);
 }
@@ -783,14 +841,18 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                        split->block_len = em->block_len;
                                else
                                        split->block_len = split->len;
+                               split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = max(split->block_len,
                                                            em->disk_num_bytes);
+                               split->offset = em->offset;
                                split->ram_bytes = em->ram_bytes;
                        } else {
                                split->orig_start = split->start;
                                split->block_len = 0;
                                split->block_start = em->block_start;
+                               split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = 0;
+                               split->offset = 0;
                                split->ram_bytes = split->len;
                        }
 
@@ -811,13 +873,14 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        split->start = end;
                        split->len = em_end - end;
                        split->block_start = em->block_start;
+                       split->disk_bytenr = em->disk_bytenr;
                        split->flags = flags;
                        split->generation = gen;
 
                        if (em->block_start < EXTENT_MAP_LAST_BYTE) {
                                split->disk_num_bytes = max(em->block_len,
                                                            em->disk_num_bytes);
-
+                               split->offset = em->offset + end - em->start;
                                split->ram_bytes = em->ram_bytes;
                                if (compressed) {
                                        split->block_len = em->block_len;
@@ -830,10 +893,11 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                        split->orig_start = em->orig_start;
                                }
                        } else {
+                               split->disk_num_bytes = 0;
+                               split->offset = 0;
                                split->ram_bytes = split->len;
                                split->orig_start = split->start;
                                split->block_len = 0;
-                               split->disk_num_bytes = 0;
                        }
 
                        if (extent_map_in_tree(em)) {
@@ -987,6 +1051,9 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        /* First, replace the em with a new extent_map starting from * em->start */
        split_pre->start = em->start;
        split_pre->len = pre;
+       split_pre->disk_bytenr = new_logical;
+       split_pre->disk_num_bytes = split_pre->len;
+       split_pre->offset = 0;
        split_pre->orig_start = split_pre->start;
        split_pre->block_start = new_logical;
        split_pre->block_len = split_pre->len;
@@ -1005,10 +1072,12 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        /* Insert the middle extent_map. */
        split_mid->start = em->start + pre;
        split_mid->len = em->len - pre;
+       split_mid->disk_bytenr = em->block_start + pre;
+       split_mid->disk_num_bytes = split_mid->len;
+       split_mid->offset = 0;
        split_mid->orig_start = split_mid->start;
        split_mid->block_start = em->block_start + pre;
        split_mid->block_len = split_mid->len;
-       split_mid->disk_num_bytes = split_mid->block_len;
        split_mid->ram_bytes = split_mid->len;
        split_mid->flags = flags;
        split_mid->generation = em->generation;
fs/btrfs/extent_map.h +17 -0
index 6ea0287b0d61..cc9c8092b704 100644
@@ -70,12 +70,29 @@ struct extent_map {
         */
        u64 orig_start;
 
+       /*
+        * The bytenr for of the full on-disk extent.
+        *
+        * For regular extents it's btrfs_file_extent_item::disk_bytenr.
+        * For holes it's EXTENT_MAP_HOLE and for inline extents it's
+        * EXTENT_MAP_INLINE.
+        */
+       u64 disk_bytenr;
+
        /*
         * The full on-disk extent length, matching
         * btrfs_file_extent_item::disk_num_bytes.
         */
        u64 disk_num_bytes;
 
+       /*
+        * Offset inside the decompressed extent.
+        *
+        * For regular extents it's btrfs_file_extent_item::offset.
+        * For holes and inline extents it's 0.
+        */
+       u64 offset;
+
        /*
         * The decompressed size of the whole on-disk extent, matching
         * btrfs_file_extent_item::ram_bytes.
fs/btrfs/file-item.c +8 -1
index 2197cfe5443b..47bd4fe0a44b 100644
@@ -1294,12 +1294,17 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                em->len = btrfs_file_extent_end(path) - extent_start;
                em->orig_start = extent_start -
                        btrfs_file_extent_offset(leaf, fi);
-               em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
                bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
                if (bytenr == 0) {
                        em->block_start = EXTENT_MAP_HOLE;
+                       em->disk_bytenr = EXTENT_MAP_HOLE;
+                       em->disk_num_bytes = 0;
+                       em->offset = 0;
                        return;
                }
+               em->disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+               em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
+               em->offset = btrfs_file_extent_offset(leaf, fi);
                if (compress_type != BTRFS_COMPRESS_NONE) {
                        extent_map_set_compression(em, compress_type);
                        em->block_start = bytenr;
@@ -1316,8 +1321,10 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                ASSERT(extent_start == 0);
 
                em->block_start = EXTENT_MAP_INLINE;
+               em->disk_bytenr = EXTENT_MAP_INLINE;
                em->start = 0;
                em->len = fs_info->sectorsize;
+               em->offset = 0;
                /*
                 * Initialize orig_start and block_len with the same values
                 * as in inode.c:btrfs_get_extent().
fs/btrfs/file.c +1 -0
index 63a13a4cace0..8931eeee199d 100644
@@ -2337,6 +2337,7 @@ static int fill_holes(struct btrfs_trans_handle *trans,
                hole_em->orig_start = offset;
 
                hole_em->block_start = EXTENT_MAP_HOLE;
+               hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                hole_em->block_len = 0;
                hole_em->disk_num_bytes = 0;
                hole_em->generation = trans->transid;
fs/btrfs/inode.c +0 -0
index 2815b72f2d85..42fea12d509f 100644
Quoted 8 lines

fs/btrfs/inode.c static noinline int run_delalloc_cow(struct btrfs_inode *inode, · line 142

Quoted 6 lines
fs/btrfs/inode.c +1 -1
@@ -139,8 +139,9 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
                                     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 orig_start, u64 block_start,
-                                      u64 block_len, u64 orig_block_len,
+                                      u64 block_len, u64 disk_num_bytes,

This hunk should have been in the previous patch, which renames

In the first patch actually.

Quoted 2 lines

orig_block_len to disk_num_bytes everywhere, including the prototype declaration of this function.

And in the declaration of the function, not the prototype.

Quoted 229 lines
Quoted 229 lines
fs/btrfs/inode.c +52 -7
                                       u64 ram_bytes, int compress_type,
+                                      struct btrfs_file_extent *file_extent,
                                       int type);
 
 static int data_reloc_print_warning_inode(u64 inum, u64 offset, u64 num_bytes,
@@ -1152,6 +1153,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        struct btrfs_root *root = inode->root;
        struct btrfs_fs_info *fs_info = root->fs_info;
        struct btrfs_ordered_extent *ordered;
+       struct btrfs_file_extent file_extent = { 0 };
        struct btrfs_key ins;
        struct page *locked_page = NULL;
        struct extent_state *cached = NULL;
@@ -1198,6 +1200,13 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        lock_extent(io_tree, start, end, &cached);
 
        /* Here we're doing allocation and writeback of the compressed pages */
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.ram_bytes = async_extent->ram_size;
+       file_extent.num_bytes = async_extent->ram_size;
+       file_extent.offset = 0;
+       file_extent.compression = async_extent->compress_type;
+
        em = create_io_em(inode, start,
                          async_extent->ram_size,       /* len */
                          start,                        /* orig_start */
@@ -1206,6 +1215,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
                          ins.offset,                   /* orig_block_len */
                          async_extent->ram_size,       /* ram_bytes */
                          async_extent->compress_type,
+                         &file_extent,
                          BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
@@ -1395,6 +1405,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
 
        while (num_bytes > 0) {
                struct btrfs_ordered_extent *ordered;
+               struct btrfs_file_extent file_extent = { 0 };
 
                cur_alloc_size = num_bytes;
                ret = btrfs_reserve_extent(root, cur_alloc_size, cur_alloc_size,
@@ -1431,6 +1442,12 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                extent_reserved = true;
 
                ram_size = ins.offset;
+               file_extent.disk_bytenr = ins.objectid;
+               file_extent.disk_num_bytes = ins.offset;
+               file_extent.num_bytes = ins.offset;
+               file_extent.ram_bytes = ins.offset;
+               file_extent.offset = 0;
+               file_extent.compression = BTRFS_COMPRESS_NONE;
 
                lock_extent(&inode->io_tree, start, start + ram_size - 1,
                            &cached);
@@ -1442,6 +1459,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                                  ins.offset, /* orig_block_len */
                                  ram_size, /* ram_bytes */
                                  BTRFS_COMPRESS_NONE, /* compress_type */
+                                 &file_extent,
                                  BTRFS_ORDERED_REGULAR /* type */);
                if (IS_ERR(em)) {
                        unlock_extent(&inode->io_tree, start,
@@ -1855,6 +1873,7 @@ struct can_nocow_file_extent_args {
        u64 disk_bytenr;
        u64 disk_num_bytes;
        u64 extent_offset;
+
        /* Number of bytes that can be written to in NOCOW mode. */
        u64 num_bytes;
 
@@ -2182,6 +2201,7 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                                          nocow_args.num_bytes, /* block_len */
                                          nocow_args.disk_num_bytes, /* orig_block_len */
                                          ram_bytes, BTRFS_COMPRESS_NONE,
+                                         &nocow_args.file_extent,
                                          BTRFS_ORDERED_PREALLOC);
                        if (IS_ERR(em)) {
                                unlock_extent(&inode->io_tree, cur_offset,
@@ -4982,6 +5002,7 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
                        hole_em->orig_start = cur_offset;
 
                        hole_em->block_start = EXTENT_MAP_HOLE;
+                       hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                        hole_em->block_len = 0;
                        hole_em->disk_num_bytes = 0;
                        hole_em->ram_bytes = hole_size;
@@ -6842,6 +6863,7 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
        }
        em->start = EXTENT_MAP_HOLE;
        em->orig_start = EXTENT_MAP_HOLE;
+       em->disk_bytenr = EXTENT_MAP_HOLE;
        em->len = (u64)-1;
        em->block_len = (u64)-1;
 
@@ -7007,7 +7029,8 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                                                  const u64 block_len,
                                                  const u64 orig_block_len,
                                                  const u64 ram_bytes,
-                                                 const int type)
+                                                 const int type,
+                                                 struct btrfs_file_extent *file_extent)
 {
        struct extent_map *em = NULL;
        struct btrfs_ordered_extent *ordered;
@@ -7016,7 +7039,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                em = create_io_em(inode, start, len, orig_start, block_start,
                                  block_len, orig_block_len, ram_bytes,
                                  BTRFS_COMPRESS_NONE, /* compress_type */
-                                 type);
+                                 file_extent, type);
                if (IS_ERR(em))
                        goto out;
        }
@@ -7047,6 +7070,7 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
 {
        struct btrfs_root *root = inode->root;
        struct btrfs_fs_info *fs_info = root->fs_info;
+       struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em;
        struct btrfs_key ins;
        u64 alloc_hint;
@@ -7065,9 +7089,16 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
        if (ret)
                return ERR_PTR(ret);
 
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = ins.offset;
+       file_extent.ram_bytes = ins.offset;
+       file_extent.offset = 0;
+       file_extent.compression = BTRFS_COMPRESS_NONE;
        em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset, start,
                                     ins.objectid, ins.offset, ins.offset,
-                                    ins.offset, BTRFS_ORDERED_REGULAR);
+                                    ins.offset, BTRFS_ORDERED_REGULAR,
+                                    &file_extent);
        btrfs_dec_block_group_reservations(fs_info, ins.objectid);
        if (IS_ERR(em))
                btrfs_free_reserved_extent(fs_info, ins.objectid, ins.offset,
@@ -7310,6 +7341,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 orig_start, u64 block_start,
                                       u64 block_len, u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
+                                      struct btrfs_file_extent *file_extent,
                                       int type)
 {
        struct extent_map *em;
@@ -7367,9 +7399,11 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
        em->len = len;
        em->block_len = block_len;
        em->block_start = block_start;
+       em->disk_bytenr = file_extent->disk_bytenr;
        em->disk_num_bytes = disk_num_bytes;
        em->ram_bytes = ram_bytes;
        em->generation = -1;
+       em->offset = file_extent->offset;
        em->flags |= EXTENT_FLAG_PINNED;
        if (type == BTRFS_ORDERED_COMPRESSED)
                extent_map_set_compression(em, compress_type);
@@ -7393,6 +7427,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 {
        const bool nowait = (iomap_flags & IOMAP_NOWAIT);
        struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
+       struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em = *map;
        int type;
        u64 block_start, orig_start, orig_block_len, ram_bytes;
@@ -7423,7 +7458,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                block_start = em->block_start + (start - em->start);
 
                if (can_nocow_extent(inode, start, &len, &orig_start,
-                                    &orig_block_len, &ram_bytes, NULL, false, false) == 1) {
+                                    &orig_block_len, &ram_bytes,
+                                    &file_extent, false, false) == 1) {
                        bg = btrfs_inc_nocow_writers(fs_info, block_start);
                        if (bg)
                                can_nocow = true;
@@ -7451,7 +7487,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
                                              orig_start, block_start,
                                              len, orig_block_len,
-                                             ram_bytes, type);
+                                             ram_bytes, type,
+                                             &file_extent);
                btrfs_dec_nocow_writers(bg);
                if (type == BTRFS_ORDERED_PREALLOC) {
                        free_extent_map(em);
@@ -9602,6 +9639,8 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
                em->orig_start = cur_offset;
                em->len = ins.offset;
                em->block_start = ins.objectid;
+               em->disk_bytenr = ins.objectid;
+               em->offset = 0;
                em->block_len = ins.offset;
                em->disk_num_bytes = ins.offset;
                em->ram_bytes = ins.offset;
@@ -10168,6 +10207,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
        struct extent_changeset *data_reserved = NULL;
        struct extent_state *cached_state = NULL;
        struct btrfs_ordered_extent *ordered;
+       struct btrfs_file_extent file_extent = { 0 };
        int compression;
        size_t orig_count;
        u64 start, end;
@@ -10343,10 +10383,16 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
                goto out_delalloc_release;
        extent_reserved = true;
 
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = num_bytes;
+       file_extent.ram_bytes = ram_bytes;
+       file_extent.offset = encoded->unencoded_offset;
+       file_extent.compression = compression;
        em = create_io_em(inode, start, num_bytes,
                          start - encoded->unencoded_offset, ins.objectid,
                          ins.offset, ins.offset, ram_bytes, compression,
-                         BTRFS_ORDERED_COMPRESSED);
+                         &file_extent, BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
                goto out_free_reserved;
--

2.45.0

#13 lore raw

Re: [PATCH v2 03/11] btrfs: introduce new members for extent_map

In reply to Qu Wenruo
Quoted 212 lines · Qu Wenruo

Introduce two new members for extent_map:

- disk_bytenr
- offset

Both are matching the members with the same name inside btrfs_file_extent_items.

For now this patch only touches those members when:

- Reading btrfs_file_extent_items from disk
- Inserting new holes
- Merging two extent maps
With the new disk_bytenr and disk_num_bytes, doing merging would be a little complex, as we have 3 different cases:

* Both extent maps are referring to the same data extent
* Both extent maps are referring to different data extents, but those data extents are adjacent, and extent maps are at head/tail of each data extents
* One of the extent map is referring to an merged and larger data extent that covers both extent maps

The 3rd case seems only valid in selftest (test_case_3()), but a new helper merge_ondisk_extents() should be able to handle all of them.

To properly assign values for those new members, a new btrfs_file_extent parameter is introduced to all the involved call sites.

- For NOCOW writes the btrfs_file_extent would be exposed from can_nocow_file_extent().

- For other writes, the members can be easily calculated
As most of them have 0 offset and utilizing the whole on-disk data extent.
The exception is encoded write, but thankfully that interface provided offset directly and all other needed info.

For now, both the old members (block_start/block_len/orig_start) are co-existing with the new members (disk_bytenr/offset), meanwhile all the critical code is still using the old members only.

The cleanup would happen later after all the older and newer members are properly validated.

---
 fs/btrfs/defrag.c     |  4 +++
 fs/btrfs/extent_map.c | 75 +++++++++++++++++++++++++++++++++++++++++--
 fs/btrfs/extent_map.h | 17 ++++++++++
 fs/btrfs/file-item.c  |  9 +++++-
 fs/btrfs/file.c       |  1 +
 fs/btrfs/inode.c      | 60 ++++++++++++++++++++++++++++++----
 6 files changed, 155 insertions(+), 11 deletions(-)
fs/btrfs/defrag.c +4 -0
index 407ccec3e57e..242c5469f4ba 100644
@@ -709,6 +709,10 @@ static struct extent_map *defrag_get_extent(struct btrfs_inode *inode,
                        em->start = start;
                        em->orig_start = start;
                        em->block_start = EXTENT_MAP_HOLE;
+                       em->disk_bytenr = EXTENT_MAP_HOLE;
+                       em->disk_num_bytes = 0;
+                       em->ram_bytes = 0;
+                       em->offset = 0;
                        em->len = key.offset - start;
                        break;
                }
fs/btrfs/extent_map.c +66 -2
index 4230dd0f34cc..4d4ac9fc43e2 100644
@@ -232,6 +232,58 @@ static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma
        return next->block_start == prev->block_start;
 }
 
+/*
+ * Handle the ondisk data extents merge for @prev and @next.
+ *
+ * Only touches disk_bytenr/disk_num_bytes/offset/ram_bytes.
+ * For now only uncompressed regular extent can be merged.
+ *
+ * @prev and @next will be both updated to point to the new merged range.
+ * Thus one of them should be removed by the caller.
+ */
+static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *next)
+{
+       u64 new_disk_bytenr;
+       u64 new_disk_num_bytes;
+       u64 new_offset;
+
+       /* @prev and @next should not be compressed. */
+       ASSERT(!extent_map_is_compressed(prev));
+       ASSERT(!extent_map_is_compressed(next));
+
+       /*
+        * There are several different cases that @prev and @next can be merged.
+        *
+        * 1) They are referring to the same data extent
+        * 2) Their ondisk data extents are adjacent and @prev is the tail
+        *    and @next is the head of their data extents
+        * 3) One of @prev/@next is referrring to a larger merged data extent.
+        *    (test_case_3 of extent maps tests).
+        *
+        * The calculation here always merge the data extents first, then update
+        * @offset using the new data extents.
+        *
+        * For case 1), the merged data extent would be the same.
+        * For case 2), we just merge the two data extents into one.
+        * For case 3), we just got the larger data extent.
+        */
+       new_disk_bytenr = min(prev->disk_bytenr, next->disk_bytenr);
+       new_disk_num_bytes = max(prev->disk_bytenr + prev->disk_num_bytes,
+                                next->disk_bytenr + next->disk_num_bytes) -
+                            new_disk_bytenr;
+       new_offset = prev->disk_bytenr + prev->offset - new_disk_bytenr;
+
+       prev->disk_bytenr = new_disk_bytenr;
+       prev->disk_num_bytes = new_disk_num_bytes;
+       prev->ram_bytes = new_disk_num_bytes;
+       prev->offset = new_offset;
+
+       next->disk_bytenr = new_disk_bytenr;
+       next->disk_num_bytes = new_disk_num_bytes;
+       next->ram_bytes = new_disk_num_bytes;
+       next->offset = new_offset;
+}
+
 static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
 {
        struct extent_map_tree *tree = &inode->extent_tree;
@@ -263,6 +315,9 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                        em->block_len += merge->block_len;
                        em->block_start = merge->block_start;
                        em->generation = max(em->generation, merge->generation);
+
+                       if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
+                               merge_ondisk_extents(merge, em);
                        em->flags |= EXTENT_FLAG_MERGED;
 
                        rb_erase_cached(&merge->rb_node, &tree->map);
@@ -278,6 +333,8 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
        if (rb && can_merge_extent_map(merge) && mergeable_maps(em, merge)) {
                em->len += merge->len;
                em->block_len += merge->block_len;
+               if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
+                       merge_ondisk_extents(em, merge);
                rb_erase_cached(&merge->rb_node, &tree->map);
                RB_CLEAR_NODE(&merge->rb_node);
                em->generation = max(em->generation, merge->generation);
@@ -565,6 +622,7 @@ static noinline int merge_extent_mapping(struct btrfs_inode *inode,
            !extent_map_is_compressed(em)) {
                em->block_start += start_diff;
                em->block_len = em->len;
+               em->offset += start_diff;
        }
        return add_extent_mapping(inode, em, 0);
 }
@@ -783,14 +841,18 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                        split->block_len = em->block_len;
                                else
                                        split->block_len = split->len;
+                               split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = max(split->block_len,
                                                            em->disk_num_bytes);
+                               split->offset = em->offset;
                                split->ram_bytes = em->ram_bytes;
                        } else {
                                split->orig_start = split->start;
                                split->block_len = 0;
                                split->block_start = em->block_start;
+                               split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = 0;
+                               split->offset = 0;
                                split->ram_bytes = split->len;
                        }
 
@@ -811,13 +873,14 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        split->start = end;
                        split->len = em_end - end;
                        split->block_start = em->block_start;
+                       split->disk_bytenr = em->disk_bytenr;
                        split->flags = flags;
                        split->generation = gen;
 
                        if (em->block_start < EXTENT_MAP_LAST_BYTE) {
                                split->disk_num_bytes = max(em->block_len,
                                                            em->disk_num_bytes);
-
+                               split->offset = em->offset + end - em->start;
                                split->ram_bytes = em->ram_bytes;
                                if (compressed) {
                                        split->block_len = em->block_len;
@@ -830,10 +893,11 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                        split->orig_start = em->orig_start;
                                }
                        } else {
+                               split->disk_num_bytes = 0;
+                               split->offset = 0;
                                split->ram_bytes = split->len;
                                split->orig_start = split->start;
                                split->block_len = 0;
-                               split->disk_num_bytes = 0;
                        }
 
                        if (extent_map_in_tree(em)) {

fs/btrfs/extent_map.c int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre, · line 1055

Quoted 5 lines
fs/btrfs/extent_map.c +2 -0
@@ -987,6 +1051,9 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        /* First, replace the em with a new extent_map starting from * em->start */
        split_pre->start = em->start;
        split_pre->len = pre;
+       split_pre->disk_bytenr = new_logical;
+       split_pre->disk_num_bytes = split_pre->len;

So this assignment of ->disk_num_bytes is redundant, because a few lines below (not shown in the patch diff), there's this:

split_pre->disk_num_bytes = split_pre->block_len;

Which overwrites this statement.

Quoted 336 lines
fs/btrfs/extent_map.c +4 -1
+       split_pre->offset = 0;
        split_pre->orig_start = split_pre->start;
        split_pre->block_start = new_logical;
        split_pre->block_len = split_pre->len;
@@ -1005,10 +1072,12 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        /* Insert the middle extent_map. */
        split_mid->start = em->start + pre;
        split_mid->len = em->len - pre;
+       split_mid->disk_bytenr = em->block_start + pre;
+       split_mid->disk_num_bytes = split_mid->len;
+       split_mid->offset = 0;
        split_mid->orig_start = split_mid->start;
        split_mid->block_start = em->block_start + pre;
        split_mid->block_len = split_mid->len;
-       split_mid->disk_num_bytes = split_mid->block_len;
        split_mid->ram_bytes = split_mid->len;
        split_mid->flags = flags;
        split_mid->generation = em->generation;
fs/btrfs/extent_map.h +17 -0
index 6ea0287b0d61..cc9c8092b704 100644
@@ -70,12 +70,29 @@ struct extent_map {
         */
        u64 orig_start;
 
+       /*
+        * The bytenr for of the full on-disk extent.
+        *
+        * For regular extents it's btrfs_file_extent_item::disk_bytenr.
+        * For holes it's EXTENT_MAP_HOLE and for inline extents it's
+        * EXTENT_MAP_INLINE.
+        */
+       u64 disk_bytenr;
+
        /*
         * The full on-disk extent length, matching
         * btrfs_file_extent_item::disk_num_bytes.
         */
        u64 disk_num_bytes;
 
+       /*
+        * Offset inside the decompressed extent.
+        *
+        * For regular extents it's btrfs_file_extent_item::offset.
+        * For holes and inline extents it's 0.
+        */
+       u64 offset;
+
        /*
         * The decompressed size of the whole on-disk extent, matching
         * btrfs_file_extent_item::ram_bytes.
fs/btrfs/file-item.c +8 -1
index 2197cfe5443b..47bd4fe0a44b 100644
@@ -1294,12 +1294,17 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                em->len = btrfs_file_extent_end(path) - extent_start;
                em->orig_start = extent_start -
                        btrfs_file_extent_offset(leaf, fi);
-               em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
                bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
                if (bytenr == 0) {
                        em->block_start = EXTENT_MAP_HOLE;
+                       em->disk_bytenr = EXTENT_MAP_HOLE;
+                       em->disk_num_bytes = 0;
+                       em->offset = 0;
                        return;
                }
+               em->disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
+               em->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
+               em->offset = btrfs_file_extent_offset(leaf, fi);
                if (compress_type != BTRFS_COMPRESS_NONE) {
                        extent_map_set_compression(em, compress_type);
                        em->block_start = bytenr;
@@ -1316,8 +1321,10 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                ASSERT(extent_start == 0);
 
                em->block_start = EXTENT_MAP_INLINE;
+               em->disk_bytenr = EXTENT_MAP_INLINE;
                em->start = 0;
                em->len = fs_info->sectorsize;
+               em->offset = 0;
                /*
                 * Initialize orig_start and block_len with the same values
                 * as in inode.c:btrfs_get_extent().
fs/btrfs/file.c +1 -0
index 63a13a4cace0..8931eeee199d 100644
@@ -2337,6 +2337,7 @@ static int fill_holes(struct btrfs_trans_handle *trans,
                hole_em->orig_start = offset;
 
                hole_em->block_start = EXTENT_MAP_HOLE;
+               hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                hole_em->block_len = 0;
                hole_em->disk_num_bytes = 0;
                hole_em->generation = trans->transid;
fs/btrfs/inode.c +53 -8
index 2815b72f2d85..42fea12d509f 100644
@@ -139,8 +139,9 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
                                     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 orig_start, u64 block_start,
-                                      u64 block_len, u64 orig_block_len,
+                                      u64 block_len, u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
+                                      struct btrfs_file_extent *file_extent,
                                       int type);
 
 static int data_reloc_print_warning_inode(u64 inum, u64 offset, u64 num_bytes,
@@ -1152,6 +1153,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        struct btrfs_root *root = inode->root;
        struct btrfs_fs_info *fs_info = root->fs_info;
        struct btrfs_ordered_extent *ordered;
+       struct btrfs_file_extent file_extent = { 0 };
        struct btrfs_key ins;
        struct page *locked_page = NULL;
        struct extent_state *cached = NULL;
@@ -1198,6 +1200,13 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        lock_extent(io_tree, start, end, &cached);
 
        /* Here we're doing allocation and writeback of the compressed pages */
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.ram_bytes = async_extent->ram_size;
+       file_extent.num_bytes = async_extent->ram_size;
+       file_extent.offset = 0;
+       file_extent.compression = async_extent->compress_type;
+
        em = create_io_em(inode, start,
                          async_extent->ram_size,       /* len */
                          start,                        /* orig_start */
@@ -1206,6 +1215,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
                          ins.offset,                   /* orig_block_len */
                          async_extent->ram_size,       /* ram_bytes */
                          async_extent->compress_type,
+                         &file_extent,
                          BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
@@ -1395,6 +1405,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
 
        while (num_bytes > 0) {
                struct btrfs_ordered_extent *ordered;
+               struct btrfs_file_extent file_extent = { 0 };
 
                cur_alloc_size = num_bytes;
                ret = btrfs_reserve_extent(root, cur_alloc_size, cur_alloc_size,
@@ -1431,6 +1442,12 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                extent_reserved = true;
 
                ram_size = ins.offset;
+               file_extent.disk_bytenr = ins.objectid;
+               file_extent.disk_num_bytes = ins.offset;
+               file_extent.num_bytes = ins.offset;
+               file_extent.ram_bytes = ins.offset;
+               file_extent.offset = 0;
+               file_extent.compression = BTRFS_COMPRESS_NONE;
 
                lock_extent(&inode->io_tree, start, start + ram_size - 1,
                            &cached);
@@ -1442,6 +1459,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                                  ins.offset, /* orig_block_len */
                                  ram_size, /* ram_bytes */
                                  BTRFS_COMPRESS_NONE, /* compress_type */
+                                 &file_extent,
                                  BTRFS_ORDERED_REGULAR /* type */);
                if (IS_ERR(em)) {
                        unlock_extent(&inode->io_tree, start,
@@ -1855,6 +1873,7 @@ struct can_nocow_file_extent_args {
        u64 disk_bytenr;
        u64 disk_num_bytes;
        u64 extent_offset;
+
        /* Number of bytes that can be written to in NOCOW mode. */
        u64 num_bytes;
 
@@ -2182,6 +2201,7 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                                          nocow_args.num_bytes, /* block_len */
                                          nocow_args.disk_num_bytes, /* orig_block_len */
                                          ram_bytes, BTRFS_COMPRESS_NONE,
+                                         &nocow_args.file_extent,
                                          BTRFS_ORDERED_PREALLOC);
                        if (IS_ERR(em)) {
                                unlock_extent(&inode->io_tree, cur_offset,
@@ -4982,6 +5002,7 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
                        hole_em->orig_start = cur_offset;
 
                        hole_em->block_start = EXTENT_MAP_HOLE;
+                       hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                        hole_em->block_len = 0;
                        hole_em->disk_num_bytes = 0;
                        hole_em->ram_bytes = hole_size;
@@ -6842,6 +6863,7 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
        }
        em->start = EXTENT_MAP_HOLE;
        em->orig_start = EXTENT_MAP_HOLE;
+       em->disk_bytenr = EXTENT_MAP_HOLE;
        em->len = (u64)-1;
        em->block_len = (u64)-1;
 
@@ -7007,7 +7029,8 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                                                  const u64 block_len,
                                                  const u64 orig_block_len,
                                                  const u64 ram_bytes,
-                                                 const int type)
+                                                 const int type,
+                                                 struct btrfs_file_extent *file_extent)
 {
        struct extent_map *em = NULL;
        struct btrfs_ordered_extent *ordered;
@@ -7016,7 +7039,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                em = create_io_em(inode, start, len, orig_start, block_start,
                                  block_len, orig_block_len, ram_bytes,
                                  BTRFS_COMPRESS_NONE, /* compress_type */
-                                 type);
+                                 file_extent, type);
                if (IS_ERR(em))
                        goto out;
        }
@@ -7047,6 +7070,7 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
 {
        struct btrfs_root *root = inode->root;
        struct btrfs_fs_info *fs_info = root->fs_info;
+       struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em;
        struct btrfs_key ins;
        u64 alloc_hint;
@@ -7065,9 +7089,16 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
        if (ret)
                return ERR_PTR(ret);
 
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = ins.offset;
+       file_extent.ram_bytes = ins.offset;
+       file_extent.offset = 0;
+       file_extent.compression = BTRFS_COMPRESS_NONE;
        em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset, start,
                                     ins.objectid, ins.offset, ins.offset,
-                                    ins.offset, BTRFS_ORDERED_REGULAR);
+                                    ins.offset, BTRFS_ORDERED_REGULAR,
+                                    &file_extent);
        btrfs_dec_block_group_reservations(fs_info, ins.objectid);
        if (IS_ERR(em))
                btrfs_free_reserved_extent(fs_info, ins.objectid, ins.offset,
@@ -7310,6 +7341,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 orig_start, u64 block_start,
                                       u64 block_len, u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
+                                      struct btrfs_file_extent *file_extent,
                                       int type)
 {
        struct extent_map *em;
@@ -7367,9 +7399,11 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
        em->len = len;
        em->block_len = block_len;
        em->block_start = block_start;
+       em->disk_bytenr = file_extent->disk_bytenr;
        em->disk_num_bytes = disk_num_bytes;
        em->ram_bytes = ram_bytes;
        em->generation = -1;
+       em->offset = file_extent->offset;
        em->flags |= EXTENT_FLAG_PINNED;
        if (type == BTRFS_ORDERED_COMPRESSED)
                extent_map_set_compression(em, compress_type);
@@ -7393,6 +7427,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 {
        const bool nowait = (iomap_flags & IOMAP_NOWAIT);
        struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
+       struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em = *map;
        int type;
        u64 block_start, orig_start, orig_block_len, ram_bytes;
@@ -7423,7 +7458,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                block_start = em->block_start + (start - em->start);
 
                if (can_nocow_extent(inode, start, &len, &orig_start,
-                                    &orig_block_len, &ram_bytes, NULL, false, false) == 1) {
+                                    &orig_block_len, &ram_bytes,
+                                    &file_extent, false, false) == 1) {
                        bg = btrfs_inc_nocow_writers(fs_info, block_start);
                        if (bg)
                                can_nocow = true;
@@ -7451,7 +7487,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
                                              orig_start, block_start,
                                              len, orig_block_len,
-                                             ram_bytes, type);
+                                             ram_bytes, type,
+                                             &file_extent);
                btrfs_dec_nocow_writers(bg);
                if (type == BTRFS_ORDERED_PREALLOC) {
                        free_extent_map(em);
@@ -9602,6 +9639,8 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
                em->orig_start = cur_offset;
                em->len = ins.offset;
                em->block_start = ins.objectid;
+               em->disk_bytenr = ins.objectid;
+               em->offset = 0;
                em->block_len = ins.offset;
                em->disk_num_bytes = ins.offset;
                em->ram_bytes = ins.offset;
@@ -10168,6 +10207,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
        struct extent_changeset *data_reserved = NULL;
        struct extent_state *cached_state = NULL;
        struct btrfs_ordered_extent *ordered;
+       struct btrfs_file_extent file_extent = { 0 };
        int compression;
        size_t orig_count;
        u64 start, end;
@@ -10343,10 +10383,16 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
                goto out_delalloc_release;
        extent_reserved = true;
 
+       file_extent.disk_bytenr = ins.objectid;
+       file_extent.disk_num_bytes = ins.offset;
+       file_extent.num_bytes = num_bytes;
+       file_extent.ram_bytes = ram_bytes;
+       file_extent.offset = encoded->unencoded_offset;
+       file_extent.compression = compression;
        em = create_io_em(inode, start, num_bytes,
                          start - encoded->unencoded_offset, ins.objectid,
                          ins.offset, ins.offset, ram_bytes, compression,
-                         BTRFS_ORDERED_COMPRESSED);
+                         &file_extent, BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
                goto out_free_reserved;
--

2.45.0

author
#14 lore raw

PATCH v2 4/11 btrfs: introduce extra sanity checks for extent maps

In reply to Qu Wenruo

Since extent_map structure has the all the needed members to represent a file extent directly, we can apply all the file extent sanity checks to an extent map.

The new sanity checks would cross check both the old members (block_start/block_len/orig_start) and the new members (disk_bytenr/disk_num_bytes/offset).

There is a special case for offset/orig_start/start cross check, we only do such sanity check for compressed extent:

- Only compressed read/encoded write really utilize orig_start This can be proved by the cleanup patch of orig_start.

- Merged data extents can lead to false alerts
The problem is, with disk_bytenr/disk_num_bytes, if we're merging two extent maps like this:

    |<- data extent A -->|<-- data extent B -->|
              |<- em 1 ->|<- em 2 ->|

Let's assume em2 has orig_offset of 0 and start of 0, and obvisouly offset 0.

But after merging, the merged em would have offset of em1, screwing up whatever the @orig_start cross check against @start.

The checks happens at the following timing:

- add_extent_mapping()
This is for newly added extent map

- replace_extent_mapping()
This is for btrfs_drop_extent_map_range() and split_extent_map()

- try_merge_map()

Since the check is way more strict than before, the following code has to be modified to pass the check:

- extent-map-tests
Previously the test case never populate ram_bytes, not to mention the newly introduced disk_bytenr/disk_num_bytes.
Populate the involved numbers mostly to follow the existing block_start/block_len values.

There are two special cases worth mentioning:
- test_case_3()
The test case is already way too invalid that tree-checker will reject almost all extents.

And there is a special unaligned regular extent which has mismatch disk_num_bytes (4096) and ram_bytes (4096 - 1).
Fix it by all assigned the disk_num_bytes and ram_bytes to 4096 - 1.

- test_case_7()
An extent is inserted with 16K length, but on-disk extent size is only 4K.
This means it must be a compressed extent, so set the compressed flag for it.

- setup_relocation_extent_mapping()
This is mostly utilized by relocation code to read the chunk like an inode.
So populate the extent map using a regular non-compressed extent.

In fact, the new cross checks already exposed a bug in btrfs_drop_extent_map_range(), and caught tons of bugs in the new members assignment.

---
 fs/btrfs/extent_map.c             | 66 +++++++++++++++++++++++++++++++
 fs/btrfs/relocation.c             |  4 ++
 fs/btrfs/tests/extent-map-tests.c | 56 +++++++++++++++++++++++++-
 fs/btrfs/tests/inode-tests.c      |  2 +-
 4 files changed, 126 insertions(+), 2 deletions(-)
fs/btrfs/extent_map.c +66 -0
index 4d4ac9fc43e2..8d0e257fc113 100644
@@ -284,6 +284,66 @@ static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *nex
 	next->offset = new_offset;
 }
 
+static void dump_extent_map(const char *prefix, struct extent_map *em)
+{
+	if (!IS_ENABLED(CONFIG_BTRFS_DEBUG))
+		return;
+	pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu orig_start=%llu block_start=%llu block_len=%llu flags=0x%x\n",
+		prefix, em->start, em->len, em->disk_bytenr, em->disk_num_bytes,
+		em->ram_bytes, em->offset, em->orig_start, em->block_start,
+		em->block_len, em->flags);
+	ASSERT(0);
+}
+
+/* Internal sanity checks for btrfs debug builds. */
+static void validate_extent_map(struct extent_map *em)
+{
+	if (!IS_ENABLED(CONFIG_BTRFS_DEBUG))
+		return;
+	if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE) {
+		if (em->disk_num_bytes == 0)
+			dump_extent_map("zero disk_num_bytes", em);
+		if (em->offset + em->len > em->ram_bytes)
+			dump_extent_map("ram_bytes too small", em);
+		if (em->offset + em->len > em->disk_num_bytes &&
+		    !extent_map_is_compressed(em))
+			dump_extent_map("disk_num_bytes too small", em);
+
+		if (extent_map_is_compressed(em)) {
+			if (em->block_start != em->disk_bytenr)
+				dump_extent_map(
+				"mismatch block_start/disk_bytenr/offset", em);
+			if (em->disk_num_bytes != em->block_len)
+				dump_extent_map(
+				"mismatch disk_num_bytes/block_len", em);
+			/*
+			 * Here we only check the start/orig_start/offset for
+			 * compressed extents.
+			 * This is because em::offset is always based on the
+			 * referred data extent, which can be merged.
+			 *
+			 * In that case, @offset would no longer match
+			 * em::start - em::orig_start, and cause false alert.
+			 *
+			 * Thankfully only compressed extent read/encoded write
+			 * really bothers @orig_start, so we can skip
+			 * the check for non-compressed extents.
+			 */
+			if (em->orig_start != em->start - em->offset)
+				dump_extent_map(
+				"mismatch orig_start/offset/start", em);
+
+		} else {
+			if (em->block_start != em->disk_bytenr + em->offset)
+				dump_extent_map(
+				"mismatch block_start/disk_bytenr/offset", em);
+		}
+	} else {
+		if (em->offset)
+			dump_extent_map("non-zero offset for hole/inline", em);
+	}
+}
+
 static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
 {
 	struct extent_map_tree *tree = &inode->extent_tree;
@@ -320,6 +380,7 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
 				merge_ondisk_extents(merge, em);
 			em->flags |= EXTENT_FLAG_MERGED;
 
+			validate_extent_map(em);
 			rb_erase_cached(&merge->rb_node, &tree->map);
 			RB_CLEAR_NODE(&merge->rb_node);
 			free_extent_map(merge);
@@ -335,6 +396,7 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
 		em->block_len += merge->block_len;
 		if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
 			merge_ondisk_extents(em, merge);
+		validate_extent_map(em);
 		rb_erase_cached(&merge->rb_node, &tree->map);
 		RB_CLEAR_NODE(&merge->rb_node);
 		em->generation = max(em->generation, merge->generation);
@@ -446,6 +508,7 @@ static int add_extent_mapping(struct btrfs_inode *inode,
 
 	lockdep_assert_held_write(&tree->lock);
 
+	validate_extent_map(em);
 	ret = tree_insert(&tree->map, em);
 	if (ret)
 		return ret;
@@ -553,6 +616,9 @@ static void replace_extent_mapping(struct btrfs_inode *inode,
 
 	lockdep_assert_held_write(&tree->lock);
 
+	validate_extent_map(cur);
+	validate_extent_map(new);
+
 	WARN_ON(cur->flags & EXTENT_FLAG_PINNED);
 	ASSERT(extent_map_in_tree(cur));
 	if (!(cur->flags & EXTENT_FLAG_LOGGING))
fs/btrfs/relocation.c +4 -0
index 8b24bb5a0aa1..0eb737507d12 100644
@@ -2911,9 +2911,13 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inod
 		return -ENOMEM;
 
 	em->start = start;
+	em->orig_start = start;
 	em->len = end + 1 - start;
 	em->block_len = em->len;
 	em->block_start = block_start;
+	em->disk_bytenr = block_start;
+	em->disk_num_bytes = em->len;
+	em->ram_bytes = em->len;
 	em->flags |= EXTENT_FLAG_PINNED;
 
 	lock_extent(&BTRFS_I(inode)->io_tree, start, end, &cached_state);
fs/btrfs/tests/extent-map-tests.c +55 -1

173 lines not on this page · raw message

fs/btrfs/tests/inode-tests.c +1 -1
index 99da9d34b77a..0895c6e06812 100644
@@ -117,7 +117,7 @@ static void setup_file_extents(struct btrfs_root *root, u32 sectorsize)
 
 	/* Now for a regular extent */
 	insert_extent(root, offset, sectorsize - 1, sectorsize - 1, 0,
-		      disk_bytenr, sectorsize, BTRFS_FILE_EXTENT_REG, 0, slot);
+		      disk_bytenr, sectorsize - 1, BTRFS_FILE_EXTENT_REG, 0, slot);
 	slot++;
 	disk_bytenr += sectorsize;
 	offset += sectorsize - 1;
Signature · 1 line
2.45.0
#15 lore raw

Re: [PATCH v2 04/11] btrfs: introduce extra sanity checks for extent maps

Quoted 19 lines · Qu Wenruo

Since extent_map structure has the all the needed members to represent a file extent directly, we can apply all the file extent sanity checks to an extent map.

The new sanity checks would cross check both the old members (block_start/block_len/orig_start) and the new members (disk_bytenr/disk_num_bytes/offset).

There is a special case for offset/orig_start/start cross check, we only do such sanity check for compressed extent:

- Only compressed read/encoded write really utilize orig_start This can be proved by the cleanup patch of orig_start.

- Merged data extents can lead to false alerts
The problem is, with disk_bytenr/disk_num_bytes, if we're merging two extent maps like this:

Quoted 5 lines
    |<- data extent A -->|<-- data extent B -->|
              |<- em 1 ->|<- em 2 ->|

Let's assume em2 has orig_offset of 0 and start of 0, and obvisouly offset 0.

obvisouly -> obviously

I'm confused. How can em2 have a "start" of 0?
By "start" you mean file offset I suppose, since that's what it is before this patchset. That would mean em 1 would have a negative "start".

And by "offset" you mean extent offset?

Quoted 2 lines

But after merging, the merged em would have offset of em1, screwing up

But this suggests that by "offset" you mean file offset and not the offset within an extent's range.
So did you mean "start" here?

Quoted 21 lines

whatever the @orig_start cross check against @start.

The checks happens at the following timing:

- add_extent_mapping()
This is for newly added extent map

- replace_extent_mapping()
This is for btrfs_drop_extent_map_range() and split_extent_map()

- try_merge_map()

Since the check is way more strict than before, the following code has to be modified to pass the check:

- extent-map-tests
Previously the test case never populate ram_bytes, not to mention the newly introduced disk_bytenr/disk_num_bytes.
Populate the involved numbers mostly to follow the existing block_start/block_len values.

Quoted 8 lines

There are two special cases worth mentioning:
- test_case_3()
The test case is already way too invalid that tree-checker will reject almost all extents.

And there is a special unaligned regular extent which has mismatch disk_num_bytes (4096) and ram_bytes (4096 - 1).
Fix it by all assigned the disk_num_bytes and ram_bytes to 4096 - 1.

Looking at the diff for test_case_3(), I don't see any assignment of 4096 - 1 anywhere.
All the assignments I see have a value of SZ_4K or SZ_16K.

Quoted 10 lines

- test_case_7()
An extent is inserted with 16K length, but on-disk extent size is only 4K.
This means it must be a compressed extent, so set the compressed flag for it.

- setup_relocation_extent_mapping()
This is mostly utilized by relocation code to read the chunk like an inode.

Not "mostly" - it's exclusively used by the relocation code.

Quoted 1 line

So populate the extent map using a regular non-compressed extent.

Isn't that what we already do? I'm confused.
We are already creating a regular non-compressed extent, and all the diff does is to initialize some fields unrelated to that.

Quoted 4 lines

In fact, the new cross checks already exposed a bug in btrfs_drop_extent_map_range(), and caught tons of bugs in the new members assignment.

I remember one bug in btrfs_drop_extent_map_range(), which fortunately didn't have any consequences.
Those tons of bugs you mention are only in the code added by the patchset if I understand you correctly.

Quoted 16 lines
---
 fs/btrfs/extent_map.c             | 66 +++++++++++++++++++++++++++++++
 fs/btrfs/relocation.c             |  4 ++
 fs/btrfs/tests/extent-map-tests.c | 56 +++++++++++++++++++++++++-
 fs/btrfs/tests/inode-tests.c      |  2 +-
 4 files changed, 126 insertions(+), 2 deletions(-)
fs/btrfs/extent_map.c +0 -0
index 4d4ac9fc43e2..8d0e257fc113 100644
@@ -284,6 +284,66 @@ static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *nex
        next->offset = new_offset;

fs/btrfs/extent_map.c static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *nex · line 294

Quoted 10 lines
fs/btrfs/extent_map.c +8 -0
 }
 
+static void dump_extent_map(const char *prefix, struct extent_map *em)
+{
+       if (!IS_ENABLED(CONFIG_BTRFS_DEBUG))
+               return;
+       pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu orig_start=%llu block_start=%llu block_len=%llu flags=0x%x\n",
+               prefix, em->start, em->len, em->disk_bytenr, em->disk_num_bytes,
+               em->ram_bytes, em->offset, em->orig_start, em->block_start,
+               em->block_len, em->flags);

Instead of pr_crit() please use btrfs_crit(), in the call chain we have access to the fs_info at try_merge_map().

Quoted 25 lines
fs/btrfs/extent_map.c +25 -0
+       ASSERT(0);
+}
+
+/* Internal sanity checks for btrfs debug builds. */
+static void validate_extent_map(struct extent_map *em)
+{
+       if (!IS_ENABLED(CONFIG_BTRFS_DEBUG))
+               return;
+       if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE) {
+               if (em->disk_num_bytes == 0)
+                       dump_extent_map("zero disk_num_bytes", em);
+               if (em->offset + em->len > em->ram_bytes)
+                       dump_extent_map("ram_bytes too small", em);
+               if (em->offset + em->len > em->disk_num_bytes &&
+                   !extent_map_is_compressed(em))
+                       dump_extent_map("disk_num_bytes too small", em);
+
+               if (extent_map_is_compressed(em)) {
+                       if (em->block_start != em->disk_bytenr)
+                               dump_extent_map(
+                               "mismatch block_start/disk_bytenr/offset", em);
+                       if (em->disk_num_bytes != em->block_len)
+                               dump_extent_map(
+                               "mismatch disk_num_bytes/block_len", em);
+                       /*

fs/btrfs/extent_map.c static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *nex · line 329

Quoted 10 lines
fs/btrfs/extent_map.c +10 -0
+                        * Here we only check the start/orig_start/offset for
+                        * compressed extents.
+                        * This is because em::offset is always based on the
+                        * referred data extent, which can be merged.
+                        *
+                        * In that case, @offset would no longer match
+                        * em::start - em::orig_start, and cause false alert.
+                        *
+                        * Thankfully only compressed extent read/encoded write
+                        * really bothers @orig_start, so we can skip

really bothers -> care about

fs/btrfs/extent_map.c static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *nex · line 337

Quoted 8 lines
fs/btrfs/extent_map.c +8 -0
+                        * the check for non-compressed extents.
+                        */
+                       if (em->orig_start != em->start - em->offset)
+                               dump_extent_map(
+                               "mismatch orig_start/offset/start", em);
+
+               } else {
+                       if (em->block_start != em->disk_bytenr + em->offset)

You can combine this as:

} else if (em->block_start != em->disk_bytenr + em->offset) {

Which is clear and helps reduce indentation.

fs/btrfs/extent_map.c static void merge_ondisk_extents(struct extent_map *prev, struct extent_map *nex · line 342

Quoted 5 lines
fs/btrfs/extent_map.c +5 -0
+                               dump_extent_map(
+                               "mismatch block_start/disk_bytenr/offset", em);
+               }
+       } else {
+               if (em->offset)

Same here, can be combined.

Quoted 32 lines
fs/btrfs/extent_map.c +7 -0
+                       dump_extent_map("non-zero offset for hole/inline", em);
+       }
+}
+
 static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
 {
        struct extent_map_tree *tree = &inode->extent_tree;
@@ -320,6 +380,7 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                                merge_ondisk_extents(merge, em);
                        em->flags |= EXTENT_FLAG_MERGED;
 
+                       validate_extent_map(em);
                        rb_erase_cached(&merge->rb_node, &tree->map);
                        RB_CLEAR_NODE(&merge->rb_node);
                        free_extent_map(merge);
@@ -335,6 +396,7 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                em->block_len += merge->block_len;
                if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
                        merge_ondisk_extents(em, merge);
+               validate_extent_map(em);
                rb_erase_cached(&merge->rb_node, &tree->map);
                RB_CLEAR_NODE(&merge->rb_node);
                em->generation = max(em->generation, merge->generation);
@@ -446,6 +508,7 @@ static int add_extent_mapping(struct btrfs_inode *inode,
 
        lockdep_assert_held_write(&tree->lock);
 
+       validate_extent_map(em);
        ret = tree_insert(&tree->map, em);
        if (ret)
                return ret;

fs/btrfs/extent_map.c static void replace_extent_mapping(struct btrfs_inode *inode, · line 619

Quoted 4 lines
fs/btrfs/extent_map.c +1 -0
@@ -553,6 +616,9 @@ static void replace_extent_mapping(struct btrfs_inode *inode,
 
        lockdep_assert_held_write(&tree->lock);
 
+       validate_extent_map(cur);

Why validate the extent map we are throwing away?
We have already validated it when we inserted it or after we merged it.

Thanks.

Quoted 232 lines
fs/btrfs/extent_map.c +2 -0
+       validate_extent_map(new);
+
        WARN_ON(cur->flags & EXTENT_FLAG_PINNED);
        ASSERT(extent_map_in_tree(cur));
        if (!(cur->flags & EXTENT_FLAG_LOGGING))
fs/btrfs/relocation.c +4 -0
index 8b24bb5a0aa1..0eb737507d12 100644
@@ -2911,9 +2911,13 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inod
                return -ENOMEM;
 
        em->start = start;
+       em->orig_start = start;
        em->len = end + 1 - start;
        em->block_len = em->len;
        em->block_start = block_start;
+       em->disk_bytenr = block_start;
+       em->disk_num_bytes = em->len;
+       em->ram_bytes = em->len;
        em->flags |= EXTENT_FLAG_PINNED;
 
        lock_extent(&BTRFS_I(inode)->io_tree, start, end, &cached_state);
fs/btrfs/tests/extent-map-tests.c +55 -1
index ba36794ba2d5..8c683eed9f27 100644
@@ -78,6 +78,9 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->len = SZ_16K;
        em->block_start = 0;
        em->block_len = SZ_16K;
+       em->disk_bytenr = 0;
+       em->disk_num_bytes = SZ_16K;
+       em->ram_bytes = SZ_16K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -96,9 +99,13 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        }
 
        em->start = SZ_16K;
+       em->orig_start = SZ_16K;
        em->len = SZ_4K;
        em->block_start = SZ_32K; /* avoid merging */
        em->block_len = SZ_4K;
+       em->disk_bytenr = SZ_32K; /* avoid merging */
+       em->disk_num_bytes = SZ_4K;
+       em->ram_bytes = SZ_4K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -117,9 +124,13 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 
        /* Add [0, 8K), should return [0, 16K) instead. */
        em->start = start;
+       em->orig_start = start;
        em->len = len;
        em->block_start = start;
        em->block_len = len;
+       em->disk_bytenr = start;
+       em->disk_num_bytes = len;
+       em->ram_bytes = len;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -174,6 +185,9 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->len = SZ_1K;
        em->block_start = EXTENT_MAP_INLINE;
        em->block_len = (u64)-1;
+       em->disk_bytenr = EXTENT_MAP_INLINE;
+       em->disk_num_bytes = 0;
+       em->ram_bytes = SZ_1K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -192,9 +206,13 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        }
 
        em->start = SZ_4K;
+       em->orig_start = SZ_4K;
        em->len = SZ_4K;
        em->block_start = SZ_4K;
        em->block_len = SZ_4K;
+       em->disk_bytenr = SZ_4K;
+       em->disk_num_bytes = SZ_4K;
+       em->ram_bytes = SZ_4K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -216,6 +234,9 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->len = SZ_1K;
        em->block_start = EXTENT_MAP_INLINE;
        em->block_len = (u64)-1;
+       em->disk_bytenr = EXTENT_MAP_INLINE;
+       em->disk_num_bytes = 0;
+       em->ram_bytes = SZ_1K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -262,9 +283,13 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
 
        /* Add [4K, 8K) */
        em->start = SZ_4K;
+       em->orig_start = SZ_4K;
        em->len = SZ_4K;
        em->block_start = SZ_4K;
        em->block_len = SZ_4K;
+       em->disk_bytenr = SZ_4K;
+       em->disk_num_bytes = SZ_4K;
+       em->ram_bytes = SZ_4K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -286,6 +311,9 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
        em->len = SZ_16K;
        em->block_start = 0;
        em->block_len = SZ_16K;
+       em->disk_bytenr = 0;
+       em->disk_num_bytes = SZ_16K;
+       em->ram_bytes = SZ_16K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, start, len);
        write_unlock(&em_tree->lock);
@@ -372,6 +400,9 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        em->len = SZ_8K;
        em->block_start = 0;
        em->block_len = SZ_8K;
+       em->disk_bytenr = 0;
+       em->disk_num_bytes = SZ_8K;
+       em->ram_bytes = SZ_8K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -390,9 +421,13 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
 
        /* Add [8K, 32K) */
        em->start = SZ_8K;
+       em->orig_start = SZ_8K;
        em->len = 24 * SZ_1K;
        em->block_start = SZ_16K; /* avoid merging */
        em->block_len = 24 * SZ_1K;
+       em->disk_bytenr = SZ_16K; /* avoid merging */
+       em->disk_num_bytes = 24 * SZ_1K;
+       em->ram_bytes = 24 * SZ_1K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -410,9 +445,13 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        }
        /* Add [0K, 32K) */
        em->start = 0;
+       em->orig_start = 0;
        em->len = SZ_32K;
        em->block_start = 0;
        em->block_len = SZ_32K;
+       em->disk_bytenr = 0;
+       em->disk_num_bytes = SZ_32K;
+       em->ram_bytes = SZ_32K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, start, len);
        write_unlock(&em_tree->lock);
@@ -494,9 +533,13 @@ static int add_compressed_extent(struct btrfs_inode *inode,
        }
 
        em->start = start;
+       em->orig_start = start;
        em->len = len;
        em->block_start = block_start;
        em->block_len = SZ_4K;
+       em->disk_bytenr = block_start;
+       em->disk_num_bytes = SZ_4K;
+       em->ram_bytes = len;
        em->flags |= EXTENT_FLAG_COMPRESS_ZLIB;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
@@ -715,9 +758,13 @@ static int test_case_6(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        }
 
        em->start = SZ_4K;
+       em->orig_start = SZ_4K;
        em->len = SZ_4K;
        em->block_start = SZ_16K;
        em->block_len = SZ_16K;
+       em->disk_bytenr = SZ_16K;
+       em->disk_num_bytes = SZ_16K;
+       em->ram_bytes = SZ_16K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, 0, SZ_8K);
        write_unlock(&em_tree->lock);
@@ -771,7 +818,10 @@ static int test_case_7(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->len = SZ_16K;
        em->block_start = 0;
        em->block_len = SZ_4K;
-       em->flags |= EXTENT_FLAG_PINNED;
+       em->disk_bytenr = 0;
+       em->disk_num_bytes = SZ_4K;
+       em->ram_bytes = SZ_16K;
+       em->flags |= (EXTENT_FLAG_PINNED | EXTENT_FLAG_COMPRESS_ZLIB);
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
@@ -790,9 +840,13 @@ static int test_case_7(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 
        /* [32K, 48K), not pinned */
        em->start = SZ_32K;
+       em->orig_start = SZ_32K;
        em->len = SZ_16K;
        em->block_start = SZ_32K;
        em->block_len = SZ_16K;
+       em->disk_bytenr = SZ_32K;
+       em->disk_num_bytes = SZ_16K;
+       em->ram_bytes = SZ_16K;
        write_lock(&em_tree->lock);
        ret = btrfs_add_extent_mapping(inode, &em, em->start, em->len);
        write_unlock(&em_tree->lock);
fs/btrfs/tests/inode-tests.c +1 -2
index 99da9d34b77a..0895c6e06812 100644
@@ -117,7 +117,7 @@ static void setup_file_extents(struct btrfs_root *root, u32 sectorsize)
 
        /* Now for a regular extent */
        insert_extent(root, offset, sectorsize - 1, sectorsize - 1, 0,
-                     disk_bytenr, sectorsize, BTRFS_FILE_EXTENT_REG, 0, slot);
+                     disk_bytenr, sectorsize - 1, BTRFS_FILE_EXTENT_REG, 0, slot);
        slot++;
        disk_bytenr += sectorsize;
        offset += sectorsize - 1;
--

2.45.0

#16 lore raw

Re: [PATCH v2 04/11] btrfs: introduce extra sanity checks for extent maps

Quoted 26 lines · Filipe Manana
Quoted 19 lines · Qu Wenruo

Since extent_map structure has the all the needed members to represent a file extent directly, we can apply all the file extent sanity checks to an extent map.

The new sanity checks would cross check both the old members (block_start/block_len/orig_start) and the new members (disk_bytenr/disk_num_bytes/offset).

There is a special case for offset/orig_start/start cross check, we only do such sanity check for compressed extent:

- Only compressed read/encoded write really utilize orig_start This can be proved by the cleanup patch of orig_start.

- Merged data extents can lead to false alerts
The problem is, with disk_bytenr/disk_num_bytes, if we're merging two extent maps like this:

Quoted 5 lines
     |<- data extent A -->|<-- data extent B -->|
               |<- em 1 ->|<- em 2 ->|

Let's assume em2 has orig_offset of 0 and start of 0, and obvisouly offset 0.

Quoted 6 lines

obvisouly -> obviously

I'm confused. How can em2 have a "start" of 0?
By "start" you mean file offset I suppose, since that's what it is before this patchset. That would mean em 1 would have a negative "start".

My bad, I screwed up the example numbers.

For "offset" I mean the the offset inside the data extent, aka, "btrfs_file_extent_item::offset"
And for "start" I mean filepos, aka the key.offset.

And in above case, it's all dependent on the filepos of the em1/em2.

I would change the example to something like this:

em1: file pos 4k, len 4k, extent offset 4k, ram bytes 8k, disk_bytenr X, disk_num_bytes 8K, compress none.

Then it's orig_start should be 0.

em2: file pos 8k, len 4k, extent offset 0, ram_bytes 8K, disk_bytenr X + 8K, disk_num_bytes 8K, compress none.

And its orig_start should be 8K.

Quoted 8 lines

And by "offset" you mean extent offset?

Quoted 2 lines

But after merging, the merged em would have offset of em1, screwing up

But this suggests that by "offset" you mean file offset and not the offset within an extent's range.

No, I still mean "extent offset".

As the "merged" em would be:

em merged: filepos 4k, len 8K, extent offset 4K, ram_bytes 16K, disk_bytenr X, disk_num_bytes 16K, compress none.

Thus the orig_offset would be still be 0, the same as the em1 before merging.

Quoted 28 lines

So did you mean "start" here?

Quoted 26 lines

whatever the @orig_start cross check against @start.

The checks happens at the following timing:

- add_extent_mapping()
This is for newly added extent map

- replace_extent_mapping()
This is for btrfs_drop_extent_map_range() and split_extent_map()

- try_merge_map()

Since the check is way more strict than before, the following code has to be modified to pass the check:

- extent-map-tests
Previously the test case never populate ram_bytes, not to mention the newly introduced disk_bytenr/disk_num_bytes.
Populate the involved numbers mostly to follow the existing block_start/block_len values.

There are two special cases worth mentioning:
- test_case_3()
The test case is already way too invalid that tree-checker will reject almost all extents.

Quoted 7 lines
Quoted 3 lines

And there is a special unaligned regular extent which has mismatch disk_num_bytes (4096) and ram_bytes (4096 - 1).
Fix it by all assigned the disk_num_bytes and ram_bytes to 4096 - 1.

Looking at the diff for test_case_3(), I don't see any assignment of 4096 - 1 anywhere.
All the assignments I see have a value of SZ_4K or SZ_16K.

It's in fact setup_file_extents() from inode-tests.c, I got confused with other fixes.

Quoted 12 lines
Quoted 11 lines

- test_case_7()
An extent is inserted with 16K length, but on-disk extent size is only 4K.
This means it must be a compressed extent, so set the compressed flag for it.

- setup_relocation_extent_mapping()
This is mostly utilized by relocation code to read the chunk like an inode.

Quoted 7 lines

Not "mostly" - it's exclusively used by the relocation code.

Quoted 1 line

So populate the extent map using a regular non-compressed extent.

Isn't that what we already do? I'm confused.
We are already creating a regular non-compressed extent, and all the diff does is to initialize some fields unrelated to that.

It's more like initializing the remaining members, or those uninitialized members would not pass cross-check.

I guess I should put the missing initialization where I introduced them?

Quoted 10 lines
Quoted 5 lines

In fact, the new cross checks already exposed a bug in btrfs_drop_extent_map_range(), and caught tons of bugs in the new members assignment.

I remember one bug in btrfs_drop_extent_map_range(), which fortunately didn't have any consequences.
Those tons of bugs you mention are only in the code added by the patchset if I understand you correctly.

Isn't that why I mention is as "in the new members assignment"?

Or should I just drop the mention completely?

Thanks,
Qu

author
#17 lore raw

PATCH v2 5/11 btrfs: remove extent_map::orig_start member

In reply to Qu Wenruo

Since we have extent_map::offset, the old extent_map::orig_start is just extent_map::start - extent_map::offset for non-hole/inline extents.

And since the new extent_map::offset would be verified by validate_extent_map() already meanwhile the old orig_start is not, let's just remove the old member from all call sites.

---
 fs/btrfs/btrfs_inode.h            |  2 +-
 fs/btrfs/compression.c            |  2 +-
 fs/btrfs/defrag.c                 |  1 -
 fs/btrfs/extent_map.c             | 29 +----------
 fs/btrfs/extent_map.h             |  9 ----
 fs/btrfs/file-item.c              |  5 +-
 fs/btrfs/file.c                   |  3 +-
 fs/btrfs/inode.c                  | 37 +++++---------
 fs/btrfs/relocation.c             |  1 -
 fs/btrfs/tests/extent-map-tests.c |  9 ----
 fs/btrfs/tests/inode-tests.c      | 84 +++++++++++++------------------
 fs/btrfs/tree-log.c               |  2 +-
 include/trace/events/btrfs.h      | 14 ++----
 13 files changed, 60 insertions(+), 138 deletions(-)
fs/btrfs/btrfs_inode.h +1 -1
index 18678762615a..f30afce4f6ca 100644
@@ -461,7 +461,7 @@ struct btrfs_file_extent {
 };
 
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
-			      u64 *orig_start, u64 *orig_block_len,
+			      u64 *orig_block_len,
 			      u64 *ram_bytes, struct btrfs_file_extent *file_extent,
 			      bool nowait, bool strict);
 
fs/btrfs/compression.c +1 -1
index 6441e47d8a5e..a4cd0e743027 100644
@@ -590,7 +590,7 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio)
 	cb = alloc_compressed_bio(inode, file_offset, REQ_OP_READ,
 				  end_bbio_comprssed_read);
 
-	cb->start = em->orig_start;
+	cb->start = em->start - em->offset;
 	em_len = em->len;
 	em_start = em->start;
 
fs/btrfs/defrag.c +0 -1
index 242c5469f4ba..025e7f853a68 100644
@@ -707,7 +707,6 @@ static struct extent_map *defrag_get_extent(struct btrfs_inode *inode,
 		 */
 		if (key.offset > start) {
 			em->start = start;
-			em->orig_start = start;
 			em->block_start = EXTENT_MAP_HOLE;
 			em->disk_bytenr = EXTENT_MAP_HOLE;
 			em->disk_num_bytes = 0;
fs/btrfs/extent_map.c +2 -27
index 8d0e257fc113..dc73b8a81271 100644
@@ -288,9 +288,9 @@ static void dump_extent_map(const char *prefix, struct extent_map *em)
 {
 	if (!IS_ENABLED(CONFIG_BTRFS_DEBUG))
 		return;
-	pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu orig_start=%llu block_start=%llu block_len=%llu flags=0x%x\n",
+	pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu block_start=%llu block_len=%llu flags=0x%x\n",
 		prefix, em->start, em->len, em->disk_bytenr, em->disk_num_bytes,
-		em->ram_bytes, em->offset, em->orig_start, em->block_start,
+		em->ram_bytes, em->offset, em->block_start,
 		em->block_len, em->flags);
 	ASSERT(0);
 }
@@ -316,23 +316,6 @@ static void validate_extent_map(struct extent_map *em)
 			if (em->disk_num_bytes != em->block_len)
 				dump_extent_map(
 				"mismatch disk_num_bytes/block_len", em);
-			/*
-			 * Here we only check the start/orig_start/offset for
-			 * compressed extents.
-			 * This is because em::offset is always based on the
-			 * referred data extent, which can be merged.
-			 *
-			 * In that case, @offset would no longer match
-			 * em::start - em::orig_start, and cause false alert.
-			 *
-			 * Thankfully only compressed extent read/encoded write
-			 * really bothers @orig_start, so we can skip
-			 * the check for non-compressed extents.
-			 */
-			if (em->orig_start != em->start - em->offset)
-				dump_extent_map(
-				"mismatch orig_start/offset/start", em);
-
 		} else {
 			if (em->block_start != em->disk_bytenr + em->offset)
 				dump_extent_map(
@@ -370,7 +353,6 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
 			merge = rb_entry(rb, struct extent_map, rb_node);
 		if (rb && can_merge_extent_map(merge) && mergeable_maps(merge, em)) {
 			em->start = merge->start;
-			em->orig_start = merge->orig_start;
 			em->len += merge->len;
 			em->block_len += merge->block_len;
 			em->block_start = merge->block_start;
@@ -900,7 +882,6 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
 			split->len = start - em->start;
 
 			if (em->block_start < EXTENT_MAP_LAST_BYTE) {
-				split->orig_start = em->orig_start;
 				split->block_start = em->block_start;
 
 				if (compressed)
@@ -913,7 +894,6 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
 				split->offset = em->offset;
 				split->ram_bytes = em->ram_bytes;
 			} else {
-				split->orig_start = split->start;
 				split->block_len = 0;
 				split->block_start = em->block_start;
 				split->disk_bytenr = em->disk_bytenr;
@@ -950,19 +930,16 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
 				split->ram_bytes = em->ram_bytes;
 				if (compressed) {
 					split->block_len = em->block_len;
-					split->orig_start = em->orig_start;
 				} else {
 					const u64 diff = end - em->start;
 
 					split->block_len = split->len;
 					split->block_start += diff;
-					split->orig_start = em->orig_start;
 				}
 			} else {
 				split->disk_num_bytes = 0;
 				split->offset = 0;
 				split->ram_bytes = split->len;
-				split->orig_start = split->start;
 				split->block_len = 0;
 			}
 
@@ -1120,7 +1097,6 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
 	split_pre->disk_bytenr = new_logical;
 	split_pre->disk_num_bytes = split_pre->len;
 	split_pre->offset = 0;
-	split_pre->orig_start = split_pre->start;
 	split_pre->block_start = new_logical;
 	split_pre->block_len = split_pre->len;
 	split_pre->disk_num_bytes = split_pre->block_len;
@@ -1141,7 +1117,6 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
 	split_mid->disk_bytenr = em->block_start + pre;
 	split_mid->disk_num_bytes = split_mid->len;
 	split_mid->offset = 0;
-	split_mid->orig_start = split_mid->start;
 	split_mid->block_start = em->block_start + pre;
 	split_mid->block_len = split_mid->len;
 	split_mid->ram_bytes = split_mid->len;
fs/btrfs/extent_map.h +0 -9
index cc9c8092b704..454a4bb08d95 100644
@@ -61,15 +61,6 @@ struct extent_map {
 	 */
 	u64 len;
 
-	/*
-	 * The file offset of the original file extent before splitting.
-	 *
-	 * This is an in-memory only member, matching
-	 * extent_map::start - btrfs_file_extent_item::offset for
-	 * regular/preallocated extents. EXTENT_MAP_HOLE otherwise.
-	 */
-	u64 orig_start;
-
 	/*
 	 * The bytenr for of the full on-disk extent.
 	 *
fs/btrfs/file-item.c +1 -4
index 47bd4fe0a44b..08d608f0ae5d 100644
@@ -1292,8 +1292,6 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 	    type == BTRFS_FILE_EXTENT_PREALLOC) {
 		em->start = extent_start;
 		em->len = btrfs_file_extent_end(path) - extent_start;
-		em->orig_start = extent_start -
-			btrfs_file_extent_offset(leaf, fi);
 		bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
 		if (bytenr == 0) {
 			em->block_start = EXTENT_MAP_HOLE;
@@ -1326,10 +1324,9 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		em->len = fs_info->sectorsize;
 		em->offset = 0;
 		/*
-		 * Initialize orig_start and block_len with the same values
+		 * Initialize block_len with the same values
 		 * as in inode.c:btrfs_get_extent().
 		 */
-		em->orig_start = EXTENT_MAP_HOLE;
 		em->block_len = (u64)-1;
 		extent_map_set_compression(em, compress_type);
 	} else {
fs/btrfs/file.c +1 -2
index 8931eeee199d..be4e6acb08f3 100644
@@ -1104,7 +1104,7 @@ int btrfs_check_nocow_lock(struct btrfs_inode *inode, loff_t pos,
 						   &cached_state);
 	}
 	ret = can_nocow_extent(&inode->vfs_inode, lockstart, &num_bytes,
-			NULL, NULL, NULL, NULL, nowait, false);
+			NULL, NULL, NULL, nowait, false);
 	if (ret <= 0)
 		btrfs_drew_write_unlock(&root->snapshot_lock);
 	else
@@ -2334,7 +2334,6 @@ static int fill_holes(struct btrfs_trans_handle *trans,
 		hole_em->start = offset;
 		hole_em->len = end - offset;
 		hole_em->ram_bytes = hole_em->len;
-		hole_em->orig_start = offset;
 
 		hole_em->block_start = EXTENT_MAP_HOLE;
 		hole_em->disk_bytenr = EXTENT_MAP_HOLE;
fs/btrfs/inode.c +12 -25
index 42fea12d509f..d1c948ea1421 100644
@@ -138,7 +138,7 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
 				     u64 end, struct writeback_control *wbc,
 				     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-				       u64 len, u64 orig_start, u64 block_start,
+				       u64 len, u64 block_start,
 				       u64 block_len, u64 disk_num_bytes,
 				       u64 ram_bytes, int compress_type,
 				       struct btrfs_file_extent *file_extent,
@@ -1209,7 +1209,6 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
 
 	em = create_io_em(inode, start,
 			  async_extent->ram_size,	/* len */
-			  start,			/* orig_start */
 			  ins.objectid,			/* block_start */
 			  ins.offset,			/* block_len */
 			  ins.offset,			/* orig_block_len */
@@ -1453,7 +1452,6 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
 			    &cached);
 
 		em = create_io_em(inode, start, ins.offset, /* len */
-				  start, /* orig_start */
 				  ins.objectid, /* block_start */
 				  ins.offset, /* block_len */
 				  ins.offset, /* orig_block_len */
@@ -2192,11 +2190,9 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
 
 		is_prealloc = extent_type == BTRFS_FILE_EXTENT_PREALLOC;
 		if (is_prealloc) {
-			u64 orig_start = found_key.offset - nocow_args.extent_offset;
 			struct extent_map *em;
 
 			em = create_io_em(inode, cur_offset, nocow_args.num_bytes,
-					  orig_start,
 					  nocow_args.disk_bytenr, /* block_start */
 					  nocow_args.num_bytes, /* block_len */
 					  nocow_args.disk_num_bytes, /* orig_block_len */
@@ -4999,7 +4995,6 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
 			}
 			hole_em->start = cur_offset;
 			hole_em->len = hole_size;
-			hole_em->orig_start = cur_offset;
 
 			hole_em->block_start = EXTENT_MAP_HOLE;
 			hole_em->disk_bytenr = EXTENT_MAP_HOLE;
@@ -6862,7 +6857,6 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
 		goto out;
 	}
 	em->start = EXTENT_MAP_HOLE;
-	em->orig_start = EXTENT_MAP_HOLE;
 	em->disk_bytenr = EXTENT_MAP_HOLE;
 	em->len = (u64)-1;
 	em->block_len = (u64)-1;
@@ -6955,7 +6949,6 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
 
 		/* New extent overlaps with existing one */
 		em->start = start;
-		em->orig_start = start;
 		em->len = found_key.offset - start;
 		em->block_start = EXTENT_MAP_HOLE;
 		goto insert;
@@ -6991,7 +6984,6 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
 	}
 not_found:
 	em->start = start;
-	em->orig_start = start;
 	em->len = len;
 	em->block_start = EXTENT_MAP_HOLE;
 insert:
@@ -7024,7 +7016,6 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
 						  struct btrfs_dio_data *dio_data,
 						  const u64 start,
 						  const u64 len,
-						  const u64 orig_start,
 						  const u64 block_start,
 						  const u64 block_len,
 						  const u64 orig_block_len,
@@ -7036,7 +7027,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
 	struct btrfs_ordered_extent *ordered;
 
 	if (type != BTRFS_ORDERED_NOCOW) {
-		em = create_io_em(inode, start, len, orig_start, block_start,
+		em = create_io_em(inode, start, len, block_start,
 				  block_len, orig_block_len, ram_bytes,
 				  BTRFS_COMPRESS_NONE, /* compress_type */
 				  file_extent, type);
@@ -7095,7 +7086,7 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
 	file_extent.ram_bytes = ins.offset;
 	file_extent.offset = 0;
 	file_extent.compression = BTRFS_COMPRESS_NONE;
-	em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset, start,
+	em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset,
 				     ins.objectid, ins.offset, ins.offset,
 				     ins.offset, BTRFS_ORDERED_REGULAR,
 				     &file_extent);
@@ -7141,7 +7132,7 @@ static bool btrfs_extent_readonly(struct btrfs_fs_info *fs_info, u64 bytenr)
  *	 any ordered extents.
  */
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
-			      u64 *orig_start, u64 *orig_block_len,
+			      u64 *orig_block_len,
 			      u64 *ram_bytes, struct btrfs_file_extent *file_extent,
 			      bool nowait, bool strict)
 {
@@ -7228,8 +7219,6 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
 		}
 	}
 
-	if (orig_start)
-		*orig_start = key.offset - nocow_args.extent_offset;
 	if (orig_block_len)
 		*orig_block_len = nocow_args.disk_num_bytes;
 	if (file_extent)
@@ -7338,7 +7327,7 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
 
 /* The callers of this must take lock_extent() */
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-				       u64 len, u64 orig_start, u64 block_start,
+				       u64 len, u64 block_start,
 				       u64 block_len, u64 disk_num_bytes,
 				       u64 ram_bytes, int compress_type,
 				       struct btrfs_file_extent *file_extent,
@@ -7376,7 +7365,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 		ASSERT(ram_bytes == len);
 
 		/* Since it's a new extent, we should not have any offset. */
-		ASSERT(orig_start == start);
+		ASSERT(file_extent->offset == 0);
 		break;
 	case BTRFS_ORDERED_COMPRESSED:
 		/* Must be compressed. */
@@ -7395,7 +7384,6 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 		return ERR_PTR(-ENOMEM);
 
 	em->start = start;
-	em->orig_start = orig_start;
 	em->len = len;
 	em->block_len = block_len;
 	em->block_start = block_start;
@@ -7430,7 +7418,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 	struct btrfs_file_extent file_extent = { 0 };
 	struct extent_map *em = *map;
 	int type;
-	u64 block_start, orig_start, orig_block_len, ram_bytes;
+	u64 block_start, orig_block_len, ram_bytes;
 	struct btrfs_block_group *bg;
 	bool can_nocow = false;
 	bool space_reserved = false;
@@ -7457,7 +7445,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 		len = min(len, em->len - (start - em->start));
 		block_start = em->block_start + (start - em->start);
 
-		if (can_nocow_extent(inode, start, &len, &orig_start,
+		if (can_nocow_extent(inode, start, &len,
 				     &orig_block_len, &ram_bytes,
 				     &file_extent, false, false) == 1) {
 			bg = btrfs_inc_nocow_writers(fs_info, block_start);
@@ -7485,7 +7473,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 		space_reserved = true;
 
 		em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
-					      orig_start, block_start,
+					      block_start,
 					      len, orig_block_len,
 					      ram_bytes, type,
 					      &file_extent);
@@ -9636,7 +9624,6 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
 		}
 
 		em->start = cur_offset;
-		em->orig_start = cur_offset;
 		em->len = ins.offset;
 		em->block_start = ins.objectid;
 		em->disk_bytenr = ins.objectid;
@@ -10145,7 +10132,7 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter,
 		disk_io_size = em->block_len;
 		count = em->block_len;
 		encoded->unencoded_len = em->ram_bytes;
-		encoded->unencoded_offset = iocb->ki_pos - em->orig_start;
+		encoded->unencoded_offset = iocb->ki_pos - em->start + em->offset;
 		ret = btrfs_encoded_io_compression_from_extent(fs_info,
 							       extent_map_compression(em));
 		if (ret < 0)
@@ -10390,7 +10377,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
 	file_extent.offset = encoded->unencoded_offset;
 	file_extent.compression = compression;
 	em = create_io_em(inode, start, num_bytes,
-			  start - encoded->unencoded_offset, ins.objectid,
+			  ins.objectid,
 			  ins.offset, ins.offset, ram_bytes, compression,
 			  &file_extent, BTRFS_ORDERED_COMPRESSED);
 	if (IS_ERR(em)) {
@@ -10722,7 +10709,7 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file,
 		free_extent_map(em);
 		em = NULL;
 
-		ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, NULL, false, true);
+		ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, false, true);
 		if (ret < 0) {
 			goto out;
 		} else if (ret) {
fs/btrfs/relocation.c +0 -1
index 0eb737507d12..33662b3aad38 100644
@@ -2911,7 +2911,6 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inod
 		return -ENOMEM;
 
 	em->start = start;
-	em->orig_start = start;
 	em->len = end + 1 - start;
 	em->block_len = em->len;
 	em->block_start = block_start;
fs/btrfs/tests/extent-map-tests.c +0 -9
index 8c683eed9f27..bd56efe37f02 100644
@@ -99,7 +99,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 	}
 
 	em->start = SZ_16K;
-	em->orig_start = SZ_16K;
 	em->len = SZ_4K;
 	em->block_start = SZ_32K; /* avoid merging */
 	em->block_len = SZ_4K;
@@ -124,7 +123,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 
 	/* Add [0, 8K), should return [0, 16K) instead. */
 	em->start = start;
-	em->orig_start = start;
 	em->len = len;
 	em->block_start = start;
 	em->block_len = len;
@@ -206,7 +204,6 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 	}
 
 	em->start = SZ_4K;
-	em->orig_start = SZ_4K;
 	em->len = SZ_4K;
 	em->block_start = SZ_4K;
 	em->block_len = SZ_4K;
@@ -283,7 +280,6 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
 
 	/* Add [4K, 8K) */
 	em->start = SZ_4K;
-	em->orig_start = SZ_4K;
 	em->len = SZ_4K;
 	em->block_start = SZ_4K;
 	em->block_len = SZ_4K;
@@ -421,7 +417,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
 
 	/* Add [8K, 32K) */
 	em->start = SZ_8K;
-	em->orig_start = SZ_8K;
 	em->len = 24 * SZ_1K;
 	em->block_start = SZ_16K; /* avoid merging */
 	em->block_len = 24 * SZ_1K;
@@ -445,7 +440,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
 	}
 	/* Add [0K, 32K) */
 	em->start = 0;
-	em->orig_start = 0;
 	em->len = SZ_32K;
 	em->block_start = 0;
 	em->block_len = SZ_32K;
@@ -533,7 +527,6 @@ static int add_compressed_extent(struct btrfs_inode *inode,
 	}
 
 	em->start = start;
-	em->orig_start = start;
 	em->len = len;
 	em->block_start = block_start;
 	em->block_len = SZ_4K;
@@ -758,7 +751,6 @@ static int test_case_6(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 	}
 
 	em->start = SZ_4K;
-	em->orig_start = SZ_4K;
 	em->len = SZ_4K;
 	em->block_start = SZ_16K;
 	em->block_len = SZ_16K;
@@ -840,7 +832,6 @@ static int test_case_7(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 
 	/* [32K, 48K), not pinned */
 	em->start = SZ_32K;
-	em->orig_start = SZ_32K;
 	em->len = SZ_16K;
 	em->block_start = SZ_32K;
 	em->block_len = SZ_16K;
fs/btrfs/tests/inode-tests.c +36 -48

174 lines not on this page · raw message

fs/btrfs/tree-log.c +1 -1
index 83dff4b06c84..c9e8c5f96b1c 100644
@@ -4688,7 +4688,7 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
 	struct extent_buffer *leaf;
 	struct btrfs_key key;
 	enum btrfs_compression_type compress_type;
-	u64 extent_offset = em->start - em->orig_start;
+	u64 extent_offset = em->offset;
 	u64 block_len;
 	int ret;
 
include/trace/events/btrfs.h +5 -9
index d2d94d7c3fb5..6dacdc1fb63e 100644
@@ -291,7 +291,6 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
 		__field(	u64,  ino		)
 		__field(	u64,  start		)
 		__field(	u64,  len		)
-		__field(	u64,  orig_start	)
 		__field(	u64,  block_start	)
 		__field(	u64,  block_len		)
 		__field(	u32,  flags		)
@@ -303,7 +302,6 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
 		__entry->ino		= btrfs_ino(inode);
 		__entry->start		= map->start;
 		__entry->len		= map->len;
-		__entry->orig_start	= map->orig_start;
 		__entry->block_start	= map->block_start;
 		__entry->block_len	= map->block_len;
 		__entry->flags		= map->flags;
@@ -311,13 +309,11 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
 	),
 
 	TP_printk_btrfs("root=%llu(%s) ino=%llu start=%llu len=%llu "
-		  "orig_start=%llu block_start=%llu(%s) "
-		  "block_len=%llu flags=%s refs=%u",
+		  "block_start=%llu(%s) block_len=%llu flags=%s refs=%u",
 		  show_root_type(__entry->root_objectid),
 		  __entry->ino,
 		  __entry->start,
 		  __entry->len,
-		  __entry->orig_start,
 		  show_map_type(__entry->block_start),
 		  __entry->block_len,
 		  show_map_flags(__entry->flags),
@@ -861,7 +857,7 @@ TRACE_EVENT(btrfs_add_block_group,
 		{ BTRFS_DROP_DELAYED_REF,   "DROP_DELAYED_REF" },	\
 		{ BTRFS_ADD_DELAYED_EXTENT, "ADD_DELAYED_EXTENT" }, 	\
 		{ BTRFS_UPDATE_DELAYED_HEAD, "UPDATE_DELAYED_HEAD" })
-			
+
 
 DECLARE_EVENT_CLASS(btrfs_delayed_tree_ref,
 
@@ -873,7 +869,7 @@ DECLARE_EVENT_CLASS(btrfs_delayed_tree_ref,
 	TP_STRUCT__entry_btrfs(
 		__field(	u64,  bytenr		)
 		__field(	u64,  num_bytes		)
-		__field(	int,  action		) 
+		__field(	int,  action		)
 		__field(	u64,  parent		)
 		__field(	u64,  ref_root		)
 		__field(	int,  level		)
@@ -930,7 +926,7 @@ DECLARE_EVENT_CLASS(btrfs_delayed_data_ref,
 	TP_STRUCT__entry_btrfs(
 		__field(	u64,  bytenr		)
 		__field(	u64,  num_bytes		)
-		__field(	int,  action		) 
+		__field(	int,  action		)
 		__field(	u64,  parent		)
 		__field(	u64,  ref_root		)
 		__field(	u64,  owner		)
@@ -992,7 +988,7 @@ DECLARE_EVENT_CLASS(btrfs_delayed_ref_head,
 	TP_STRUCT__entry_btrfs(
 		__field(	u64,  bytenr		)
 		__field(	u64,  num_bytes		)
-		__field(	int,  action		) 
+		__field(	int,  action		)
 		__field(	int,  is_data		)
 	),
 
Signature · 1 line
2.45.0
#18 lore raw

Re: [PATCH v2 05/11] btrfs: remove extent_map::orig_start member

Quoted 5 lines · Qu Wenruo

Since we have extent_map::offset, the old extent_map::orig_start is just extent_map::start - extent_map::offset for non-hole/inline extents.

And since the new extent_map::offset would be verified by

would be -> is already

Quoted 404 lines

validate_extent_map() already meanwhile the old orig_start is not, let's just remove the old member from all call sites.

---
 fs/btrfs/btrfs_inode.h            |  2 +-
 fs/btrfs/compression.c            |  2 +-
 fs/btrfs/defrag.c                 |  1 -
 fs/btrfs/extent_map.c             | 29 +----------
 fs/btrfs/extent_map.h             |  9 ----
 fs/btrfs/file-item.c              |  5 +-
 fs/btrfs/file.c                   |  3 +-
 fs/btrfs/inode.c                  | 37 +++++---------
 fs/btrfs/relocation.c             |  1 -
 fs/btrfs/tests/extent-map-tests.c |  9 ----
 fs/btrfs/tests/inode-tests.c      | 84 +++++++++++++------------------
 fs/btrfs/tree-log.c               |  2 +-
 include/trace/events/btrfs.h      | 14 ++----
 13 files changed, 60 insertions(+), 138 deletions(-)
fs/btrfs/btrfs_inode.h +1 -1
index 18678762615a..f30afce4f6ca 100644
@@ -461,7 +461,7 @@ struct btrfs_file_extent {
 };
 
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
-                             u64 *orig_start, u64 *orig_block_len,
+                             u64 *orig_block_len,
                              u64 *ram_bytes, struct btrfs_file_extent *file_extent,
                              bool nowait, bool strict);
 
fs/btrfs/compression.c +1 -1
index 6441e47d8a5e..a4cd0e743027 100644
@@ -590,7 +590,7 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio)
        cb = alloc_compressed_bio(inode, file_offset, REQ_OP_READ,
                                  end_bbio_comprssed_read);
 
-       cb->start = em->orig_start;
+       cb->start = em->start - em->offset;
        em_len = em->len;
        em_start = em->start;
 
fs/btrfs/defrag.c +0 -1
index 242c5469f4ba..025e7f853a68 100644
@@ -707,7 +707,6 @@ static struct extent_map *defrag_get_extent(struct btrfs_inode *inode,
                 */
                if (key.offset > start) {
                        em->start = start;
-                       em->orig_start = start;
                        em->block_start = EXTENT_MAP_HOLE;
                        em->disk_bytenr = EXTENT_MAP_HOLE;
                        em->disk_num_bytes = 0;
fs/btrfs/extent_map.c +2 -27
index 8d0e257fc113..dc73b8a81271 100644
@@ -288,9 +288,9 @@ static void dump_extent_map(const char *prefix, struct extent_map *em)
 {
        if (!IS_ENABLED(CONFIG_BTRFS_DEBUG))
                return;
-       pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu orig_start=%llu block_start=%llu block_len=%llu flags=0x%x\n",
+       pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu block_start=%llu block_len=%llu flags=0x%x\n",
                prefix, em->start, em->len, em->disk_bytenr, em->disk_num_bytes,
-               em->ram_bytes, em->offset, em->orig_start, em->block_start,
+               em->ram_bytes, em->offset, em->block_start,
                em->block_len, em->flags);
        ASSERT(0);
 }
@@ -316,23 +316,6 @@ static void validate_extent_map(struct extent_map *em)
                        if (em->disk_num_bytes != em->block_len)
                                dump_extent_map(
                                "mismatch disk_num_bytes/block_len", em);
-                       /*
-                        * Here we only check the start/orig_start/offset for
-                        * compressed extents.
-                        * This is because em::offset is always based on the
-                        * referred data extent, which can be merged.
-                        *
-                        * In that case, @offset would no longer match
-                        * em::start - em::orig_start, and cause false alert.
-                        *
-                        * Thankfully only compressed extent read/encoded write
-                        * really bothers @orig_start, so we can skip
-                        * the check for non-compressed extents.
-                        */
-                       if (em->orig_start != em->start - em->offset)
-                               dump_extent_map(
-                               "mismatch orig_start/offset/start", em);
-
                } else {
                        if (em->block_start != em->disk_bytenr + em->offset)
                                dump_extent_map(
@@ -370,7 +353,6 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                        merge = rb_entry(rb, struct extent_map, rb_node);
                if (rb && can_merge_extent_map(merge) && mergeable_maps(merge, em)) {
                        em->start = merge->start;
-                       em->orig_start = merge->orig_start;
                        em->len += merge->len;
                        em->block_len += merge->block_len;
                        em->block_start = merge->block_start;
@@ -900,7 +882,6 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        split->len = start - em->start;
 
                        if (em->block_start < EXTENT_MAP_LAST_BYTE) {
-                               split->orig_start = em->orig_start;
                                split->block_start = em->block_start;
 
                                if (compressed)
@@ -913,7 +894,6 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                split->offset = em->offset;
                                split->ram_bytes = em->ram_bytes;
                        } else {
-                               split->orig_start = split->start;
                                split->block_len = 0;
                                split->block_start = em->block_start;
                                split->disk_bytenr = em->disk_bytenr;
@@ -950,19 +930,16 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                                split->ram_bytes = em->ram_bytes;
                                if (compressed) {
                                        split->block_len = em->block_len;
-                                       split->orig_start = em->orig_start;
                                } else {
                                        const u64 diff = end - em->start;
 
                                        split->block_len = split->len;
                                        split->block_start += diff;
-                                       split->orig_start = em->orig_start;
                                }
                        } else {
                                split->disk_num_bytes = 0;
                                split->offset = 0;
                                split->ram_bytes = split->len;
-                               split->orig_start = split->start;
                                split->block_len = 0;
                        }
 
@@ -1120,7 +1097,6 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        split_pre->disk_bytenr = new_logical;
        split_pre->disk_num_bytes = split_pre->len;
        split_pre->offset = 0;
-       split_pre->orig_start = split_pre->start;
        split_pre->block_start = new_logical;
        split_pre->block_len = split_pre->len;
        split_pre->disk_num_bytes = split_pre->block_len;
@@ -1141,7 +1117,6 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        split_mid->disk_bytenr = em->block_start + pre;
        split_mid->disk_num_bytes = split_mid->len;
        split_mid->offset = 0;
-       split_mid->orig_start = split_mid->start;
        split_mid->block_start = em->block_start + pre;
        split_mid->block_len = split_mid->len;
        split_mid->ram_bytes = split_mid->len;
fs/btrfs/extent_map.h +0 -9
index cc9c8092b704..454a4bb08d95 100644
@@ -61,15 +61,6 @@ struct extent_map {
         */
        u64 len;
 
-       /*
-        * The file offset of the original file extent before splitting.
-        *
-        * This is an in-memory only member, matching
-        * extent_map::start - btrfs_file_extent_item::offset for
-        * regular/preallocated extents. EXTENT_MAP_HOLE otherwise.
-        */
-       u64 orig_start;
-
        /*
         * The bytenr for of the full on-disk extent.
         *
fs/btrfs/file-item.c +1 -4
index 47bd4fe0a44b..08d608f0ae5d 100644
@@ -1292,8 +1292,6 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
            type == BTRFS_FILE_EXTENT_PREALLOC) {
                em->start = extent_start;
                em->len = btrfs_file_extent_end(path) - extent_start;
-               em->orig_start = extent_start -
-                       btrfs_file_extent_offset(leaf, fi);
                bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
                if (bytenr == 0) {
                        em->block_start = EXTENT_MAP_HOLE;
@@ -1326,10 +1324,9 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                em->len = fs_info->sectorsize;
                em->offset = 0;
                /*
-                * Initialize orig_start and block_len with the same values
+                * Initialize block_len with the same values
                 * as in inode.c:btrfs_get_extent().
                 */
-               em->orig_start = EXTENT_MAP_HOLE;
                em->block_len = (u64)-1;
                extent_map_set_compression(em, compress_type);
        } else {
fs/btrfs/file.c +1 -2
index 8931eeee199d..be4e6acb08f3 100644
@@ -1104,7 +1104,7 @@ int btrfs_check_nocow_lock(struct btrfs_inode *inode, loff_t pos,
                                                   &cached_state);
        }
        ret = can_nocow_extent(&inode->vfs_inode, lockstart, &num_bytes,
-                       NULL, NULL, NULL, NULL, nowait, false);
+                       NULL, NULL, NULL, nowait, false);
        if (ret <= 0)
                btrfs_drew_write_unlock(&root->snapshot_lock);
        else
@@ -2334,7 +2334,6 @@ static int fill_holes(struct btrfs_trans_handle *trans,
                hole_em->start = offset;
                hole_em->len = end - offset;
                hole_em->ram_bytes = hole_em->len;
-               hole_em->orig_start = offset;
 
                hole_em->block_start = EXTENT_MAP_HOLE;
                hole_em->disk_bytenr = EXTENT_MAP_HOLE;
fs/btrfs/inode.c +9 -22
index 42fea12d509f..d1c948ea1421 100644
@@ -138,7 +138,7 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
                                     u64 end, struct writeback_control *wbc,
                                     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-                                      u64 len, u64 orig_start, u64 block_start,
+                                      u64 len, u64 block_start,
                                       u64 block_len, u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
                                       struct btrfs_file_extent *file_extent,
@@ -1209,7 +1209,6 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
 
        em = create_io_em(inode, start,
                          async_extent->ram_size,       /* len */
-                         start,                        /* orig_start */
                          ins.objectid,                 /* block_start */
                          ins.offset,                   /* block_len */
                          ins.offset,                   /* orig_block_len */
@@ -1453,7 +1452,6 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                            &cached);
 
                em = create_io_em(inode, start, ins.offset, /* len */
-                                 start, /* orig_start */
                                  ins.objectid, /* block_start */
                                  ins.offset, /* block_len */
                                  ins.offset, /* orig_block_len */
@@ -2192,11 +2190,9 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
 
                is_prealloc = extent_type == BTRFS_FILE_EXTENT_PREALLOC;
                if (is_prealloc) {
-                       u64 orig_start = found_key.offset - nocow_args.extent_offset;
                        struct extent_map *em;
 
                        em = create_io_em(inode, cur_offset, nocow_args.num_bytes,
-                                         orig_start,
                                          nocow_args.disk_bytenr, /* block_start */
                                          nocow_args.num_bytes, /* block_len */
                                          nocow_args.disk_num_bytes, /* orig_block_len */
@@ -4999,7 +4995,6 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
                        }
                        hole_em->start = cur_offset;
                        hole_em->len = hole_size;
-                       hole_em->orig_start = cur_offset;
 
                        hole_em->block_start = EXTENT_MAP_HOLE;
                        hole_em->disk_bytenr = EXTENT_MAP_HOLE;
@@ -6862,7 +6857,6 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
                goto out;
        }
        em->start = EXTENT_MAP_HOLE;
-       em->orig_start = EXTENT_MAP_HOLE;
        em->disk_bytenr = EXTENT_MAP_HOLE;
        em->len = (u64)-1;
        em->block_len = (u64)-1;
@@ -6955,7 +6949,6 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
 
                /* New extent overlaps with existing one */
                em->start = start;
-               em->orig_start = start;
                em->len = found_key.offset - start;
                em->block_start = EXTENT_MAP_HOLE;
                goto insert;
@@ -6991,7 +6984,6 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
        }
 not_found:
        em->start = start;
-       em->orig_start = start;
        em->len = len;
        em->block_start = EXTENT_MAP_HOLE;
 insert:
@@ -7024,7 +7016,6 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                                                  struct btrfs_dio_data *dio_data,
                                                  const u64 start,
                                                  const u64 len,
-                                                 const u64 orig_start,
                                                  const u64 block_start,
                                                  const u64 block_len,
                                                  const u64 orig_block_len,
@@ -7036,7 +7027,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
        struct btrfs_ordered_extent *ordered;
 
        if (type != BTRFS_ORDERED_NOCOW) {
-               em = create_io_em(inode, start, len, orig_start, block_start,
+               em = create_io_em(inode, start, len, block_start,
                                  block_len, orig_block_len, ram_bytes,
                                  BTRFS_COMPRESS_NONE, /* compress_type */
                                  file_extent, type);
@@ -7095,7 +7086,7 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
        file_extent.ram_bytes = ins.offset;
        file_extent.offset = 0;
        file_extent.compression = BTRFS_COMPRESS_NONE;
-       em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset, start,
+       em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset,
                                     ins.objectid, ins.offset, ins.offset,
                                     ins.offset, BTRFS_ORDERED_REGULAR,
                                     &file_extent);
@@ -7141,7 +7132,7 @@ static bool btrfs_extent_readonly(struct btrfs_fs_info *fs_info, u64 bytenr)
  *      any ordered extents.
  */
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
-                             u64 *orig_start, u64 *orig_block_len,
+                             u64 *orig_block_len,
                              u64 *ram_bytes, struct btrfs_file_extent *file_extent,
                              bool nowait, bool strict)
 {
@@ -7228,8 +7219,6 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
                }
        }
 
-       if (orig_start)
-               *orig_start = key.offset - nocow_args.extent_offset;
        if (orig_block_len)
                *orig_block_len = nocow_args.disk_num_bytes;
        if (file_extent)
@@ -7338,7 +7327,7 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
 
 /* The callers of this must take lock_extent() */
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-                                      u64 len, u64 orig_start, u64 block_start,
+                                      u64 len, u64 block_start,
                                       u64 block_len, u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
                                       struct btrfs_file_extent *file_extent,
@@ -7376,7 +7365,7 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                ASSERT(ram_bytes == len);
 
                /* Since it's a new extent, we should not have any offset. */
-               ASSERT(orig_start == start);
+               ASSERT(file_extent->offset == 0);
                break;
        case BTRFS_ORDERED_COMPRESSED:
                /* Must be compressed. */
@@ -7395,7 +7384,6 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                return ERR_PTR(-ENOMEM);
 
        em->start = start;
-       em->orig_start = orig_start;
        em->len = len;
        em->block_len = block_len;
        em->block_start = block_start;
@@ -7430,7 +7418,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
        struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em = *map;
        int type;
-       u64 block_start, orig_start, orig_block_len, ram_bytes;
+       u64 block_start, orig_block_len, ram_bytes;
        struct btrfs_block_group *bg;
        bool can_nocow = false;
        bool space_reserved = false;
@@ -7457,7 +7445,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                len = min(len, em->len - (start - em->start));
                block_start = em->block_start + (start - em->start);
 
-               if (can_nocow_extent(inode, start, &len, &orig_start,
+               if (can_nocow_extent(inode, start, &len,
                                     &orig_block_len, &ram_bytes,
                                     &file_extent, false, false) == 1) {
                        bg = btrfs_inc_nocow_writers(fs_info, block_start);
@@ -7485,7 +7473,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                space_reserved = true;
 
                em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
-                                             orig_start, block_start,
+                                             block_start,
                                              len, orig_block_len,
                                              ram_bytes, type,
                                              &file_extent);
@@ -9636,7 +9624,6 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
                }
 
                em->start = cur_offset;
-               em->orig_start = cur_offset;
                em->len = ins.offset;
                em->block_start = ins.objectid;
                em->disk_bytenr = ins.objectid;

fs/btrfs/inode.c ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter, · line 10135

Quoted 5 lines
fs/btrfs/inode.c +1 -1
@@ -10145,7 +10132,7 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter,
                disk_io_size = em->block_len;
                count = em->block_len;
                encoded->unencoded_len = em->ram_bytes;
-               encoded->unencoded_offset = iocb->ki_pos - em->orig_start;
+               encoded->unencoded_offset = iocb->ki_pos - em->start + em->offset;

Probably simpler to reason doing instead:

->ki_pos - (em->start - em->offset)

As that's what is used everywhere.

Quoted 138 lines
fs/btrfs/inode.c +2 -2
                ret = btrfs_encoded_io_compression_from_extent(fs_info,
                                                               extent_map_compression(em));
                if (ret < 0)
@@ -10390,7 +10377,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
        file_extent.offset = encoded->unencoded_offset;
        file_extent.compression = compression;
        em = create_io_em(inode, start, num_bytes,
-                         start - encoded->unencoded_offset, ins.objectid,
+                         ins.objectid,
                          ins.offset, ins.offset, ram_bytes, compression,
                          &file_extent, BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
@@ -10722,7 +10709,7 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file,
                free_extent_map(em);
                em = NULL;
 
-               ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, NULL, false, true);
+               ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, false, true);
                if (ret < 0) {
                        goto out;
                } else if (ret) {
fs/btrfs/relocation.c +0 -1
index 0eb737507d12..33662b3aad38 100644
@@ -2911,7 +2911,6 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inod
                return -ENOMEM;
 
        em->start = start;
-       em->orig_start = start;
        em->len = end + 1 - start;
        em->block_len = em->len;
        em->block_start = block_start;
fs/btrfs/tests/extent-map-tests.c +0 -9
index 8c683eed9f27..bd56efe37f02 100644
@@ -99,7 +99,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        }
 
        em->start = SZ_16K;
-       em->orig_start = SZ_16K;
        em->len = SZ_4K;
        em->block_start = SZ_32K; /* avoid merging */
        em->block_len = SZ_4K;
@@ -124,7 +123,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 
        /* Add [0, 8K), should return [0, 16K) instead. */
        em->start = start;
-       em->orig_start = start;
        em->len = len;
        em->block_start = start;
        em->block_len = len;
@@ -206,7 +204,6 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        }
 
        em->start = SZ_4K;
-       em->orig_start = SZ_4K;
        em->len = SZ_4K;
        em->block_start = SZ_4K;
        em->block_len = SZ_4K;
@@ -283,7 +280,6 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
 
        /* Add [4K, 8K) */
        em->start = SZ_4K;
-       em->orig_start = SZ_4K;
        em->len = SZ_4K;
        em->block_start = SZ_4K;
        em->block_len = SZ_4K;
@@ -421,7 +417,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
 
        /* Add [8K, 32K) */
        em->start = SZ_8K;
-       em->orig_start = SZ_8K;
        em->len = 24 * SZ_1K;
        em->block_start = SZ_16K; /* avoid merging */
        em->block_len = 24 * SZ_1K;
@@ -445,7 +440,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        }
        /* Add [0K, 32K) */
        em->start = 0;
-       em->orig_start = 0;
        em->len = SZ_32K;
        em->block_start = 0;
        em->block_len = SZ_32K;
@@ -533,7 +527,6 @@ static int add_compressed_extent(struct btrfs_inode *inode,
        }
 
        em->start = start;
-       em->orig_start = start;
        em->len = len;
        em->block_start = block_start;
        em->block_len = SZ_4K;
@@ -758,7 +751,6 @@ static int test_case_6(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        }
 
        em->start = SZ_4K;
-       em->orig_start = SZ_4K;
        em->len = SZ_4K;
        em->block_start = SZ_16K;
        em->block_len = SZ_16K;
@@ -840,7 +832,6 @@ static int test_case_7(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 
        /* [32K, 48K), not pinned */
        em->start = SZ_32K;
-       em->orig_start = SZ_32K;
        em->len = SZ_16K;
        em->block_start = SZ_32K;
        em->block_len = SZ_16K;
fs/btrfs/tests/inode-tests.c +4 -6
index 0895c6e06812..1b8c39edfc18 100644
@@ -358,9 +358,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("unexpected flags set, want 0 have %u", em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu", em->start,
-                        em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
        offset = em->start + em->len;
@@ -386,9 +385,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("unexpected flags set, want 0 have %u", em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu", em->start,
-                        em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
        disk_bytenr = em->block_start;

fs/btrfs/tests/inode-tests.c static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize) · line 440

Quoted 9 lines
fs/btrfs/tests/inode-tests.c +3 -3
@@ -437,9 +435,9 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("unexpected flags set, want 0 have %u", em->flags);
                goto out;
        }
-       if (em->orig_start != orig_start) {
-               test_err("wrong orig offset, want %llu, have %llu",
-                        orig_start, em->orig_start);
+       if (em->start - em->offset != orig_start) {
+               test_err("wrong offset, want %llu, have %llu",
+                        em->start - orig_start, em->offset);

Should the first argument be:

orig_start

And the second one:

em->start - em->offset

Otherwise that is confusing.
Maybe print something like:

test_err("wrong offset, em->start=%llu em->offset=%llu orig_start=%llu", em->start, em->offset, orig_start);

Quoted 28 lines
fs/btrfs/tests/inode-tests.c +4 -6
                goto out;
        }
        disk_bytenr += (em->start - orig_start);
@@ -472,9 +470,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         prealloc_only, em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu", em->start,
-                        em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
        offset = em->start + em->len;
@@ -501,9 +498,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         prealloc_only, em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu", em->start,
-                        em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
        disk_bytenr = em->block_start;

fs/btrfs/tests/inode-tests.c static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize) · line 531

Quoted 9 lines
fs/btrfs/tests/inode-tests.c +3 -3
@@ -530,15 +526,14 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("unexpected flags set, want 0 have %u", em->flags);
                goto out;
        }
-       if (em->orig_start != orig_start) {
-               test_err("unexpected orig offset, wanted %llu, have %llu",
-                        orig_start, em->orig_start);
+       if (em->start - em->offset != orig_start) {
+               test_err("unexpected offset, wanted %llu, have %llu",
+                        em->start - orig_start, em->offset);

Same here.

Quoted 12 lines
fs/btrfs/tests/inode-tests.c +2 -3
                goto out;
        }
-       if (em->block_start != (disk_bytenr + (em->start - em->orig_start))) {
+       if (em->block_start != disk_bytenr + em->offset) {
                test_err("unexpected block start, wanted %llu, have %llu",
-                        disk_bytenr + (em->start - em->orig_start),
-                        em->block_start);
+                        disk_bytenr + em->offset, em->block_start);
                goto out;
        }
        offset = em->start + em->len;

fs/btrfs/tests/inode-tests.c static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize) · line 564

Quoted 9 lines
fs/btrfs/tests/inode-tests.c +3 -3
@@ -564,15 +559,14 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         prealloc_only, em->flags);
                goto out;
        }
-       if (em->orig_start != orig_start) {
-               test_err("wrong orig offset, want %llu, have %llu", orig_start,
-                        em->orig_start);
+       if (em->start - em->offset != orig_start) {
+               test_err("wrong offset, want %llu, have %llu",
+                        em->start - orig_start, em->offset);

Same.

Quoted 48 lines
fs/btrfs/tests/inode-tests.c +8 -12
                goto out;
        }
-       if (em->block_start != (disk_bytenr + (em->start - em->orig_start))) {
+       if (em->block_start != disk_bytenr + em->offset) {
                test_err("unexpected block start, wanted %llu, have %llu",
-                        disk_bytenr + (em->start - em->orig_start),
-                        em->block_start);
+                        disk_bytenr + em->offset, em->block_start);
                goto out;
        }
        offset = em->start + em->len;
@@ -599,9 +593,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         compressed_only, em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu",
-                        em->start, em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
        if (extent_map_compression(em) != BTRFS_COMPRESS_ZLIB) {
@@ -633,9 +626,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         compressed_only, em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu",
-                        em->start, em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
        if (extent_map_compression(em) != BTRFS_COMPRESS_ZLIB) {
@@ -667,9 +659,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("unexpected flags set, want 0 have %u", em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu", em->start,
-                        em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
        offset = em->start + em->len;

fs/btrfs/tests/inode-tests.c static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize) · line 692

Quoted 9 lines
fs/btrfs/tests/inode-tests.c +3 -3
@@ -696,9 +687,9 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         compressed_only, em->flags);
                goto out;
        }
-       if (em->orig_start != orig_start) {
-               test_err("wrong orig offset, want %llu, have %llu",
-                        em->start, orig_start);
+       if (em->start - em->offset != orig_start) {
+               test_err("wrong offset, want %llu, have %llu",
+                        em->start - orig_start, em->offset);

Same.

Quoted 88 lines
fs/btrfs/tests/inode-tests.c +6 -9
                goto out;
        }
        if (extent_map_compression(em) != BTRFS_COMPRESS_ZLIB) {
@@ -729,9 +720,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("unexpected flags set, want 0 have %u", em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu", em->start,
-                        em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
        offset = em->start + em->len;
@@ -762,9 +752,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         vacancy_only, em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu", em->start,
-                        em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
        offset = em->start + em->len;
@@ -789,9 +778,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("unexpected flags set, want 0 have %u", em->flags);
                goto out;
        }
-       if (em->orig_start != em->start) {
-               test_err("wrong orig offset, want %llu, have %llu", em->start,
-                        em->orig_start);
+       if (em->offset != 0) {
+               test_err("wrong orig offset, want 0, have %llu", em->offset);
                goto out;
        }
        ret = 0;
fs/btrfs/tree-log.c +1 -1
index 83dff4b06c84..c9e8c5f96b1c 100644
@@ -4688,7 +4688,7 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
        struct extent_buffer *leaf;
        struct btrfs_key key;
        enum btrfs_compression_type compress_type;
-       u64 extent_offset = em->start - em->orig_start;
+       u64 extent_offset = em->offset;
        u64 block_len;
        int ret;
 
include/trace/events/btrfs.h +1 -5
index d2d94d7c3fb5..6dacdc1fb63e 100644
@@ -291,7 +291,6 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
                __field(        u64,  ino               )
                __field(        u64,  start             )
                __field(        u64,  len               )
-               __field(        u64,  orig_start        )
                __field(        u64,  block_start       )
                __field(        u64,  block_len         )
                __field(        u32,  flags             )
@@ -303,7 +302,6 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
                __entry->ino            = btrfs_ino(inode);
                __entry->start          = map->start;
                __entry->len            = map->len;
-               __entry->orig_start     = map->orig_start;
                __entry->block_start    = map->block_start;
                __entry->block_len      = map->block_len;
                __entry->flags          = map->flags;
@@ -311,13 +309,11 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
        ),
 
        TP_printk_btrfs("root=%llu(%s) ino=%llu start=%llu len=%llu "
-                 "orig_start=%llu block_start=%llu(%s) "
-                 "block_len=%llu flags=%s refs=%u",
+                 "block_start=%llu(%s) block_len=%llu flags=%s refs=%u",
                  show_root_type(__entry->root_objectid),
                  __entry->ino,
                  __entry->start,
                  __entry->len,
-                 __entry->orig_start,
                  show_map_type(__entry->block_start),
                  __entry->block_len,
                  show_map_flags(__entry->flags),

include/trace/events/btrfs.h TRACE_EVENT(btrfs_add_block_group, · line 860

Quoted 5 lines
include/trace/events/btrfs.h +1 -1
@@ -861,7 +857,7 @@ TRACE_EVENT(btrfs_add_block_group,
                { BTRFS_DROP_DELAYED_REF,   "DROP_DELAYED_REF" },       \
                { BTRFS_ADD_DELAYED_EXTENT, "ADD_DELAYED_EXTENT" },     \
                { BTRFS_UPDATE_DELAYED_HEAD, "UPDATE_DELAYED_HEAD" })
-
+

Unrelated white space change. Please avoid that.

Thanks.

Quoted 34 lines
include/trace/events/btrfs.h +3 -4
 DECLARE_EVENT_CLASS(btrfs_delayed_tree_ref,
 
@@ -873,7 +869,7 @@ DECLARE_EVENT_CLASS(btrfs_delayed_tree_ref,
        TP_STRUCT__entry_btrfs(
                __field(        u64,  bytenr            )
                __field(        u64,  num_bytes         )
-               __field(        int,  action            )
+               __field(        int,  action            )
                __field(        u64,  parent            )
                __field(        u64,  ref_root          )
                __field(        int,  level             )
@@ -930,7 +926,7 @@ DECLARE_EVENT_CLASS(btrfs_delayed_data_ref,
        TP_STRUCT__entry_btrfs(
                __field(        u64,  bytenr            )
                __field(        u64,  num_bytes         )
-               __field(        int,  action            )
+               __field(        int,  action            )
                __field(        u64,  parent            )
                __field(        u64,  ref_root          )
                __field(        u64,  owner             )
@@ -992,7 +988,7 @@ DECLARE_EVENT_CLASS(btrfs_delayed_ref_head,
        TP_STRUCT__entry_btrfs(
                __field(        u64,  bytenr            )
                __field(        u64,  num_bytes         )
-               __field(        int,  action            )
+               __field(        int,  action            )
                __field(        int,  is_data           )
        ),
 
--

2.45.0

#19 lore raw

Re: [PATCH v2 05/11] btrfs: remove extent_map::orig_start member

在 2024/5/13 22:39, Filipe Manana 写道:
[...]

Quoted 8 lines

include/trace/events/btrfs.h TRACE_EVENT(btrfs_add_block_group, · line 860

Quoted 6 lines
include/trace/events/btrfs.h +1 -1
@@ -861,7 +857,7 @@ TRACE_EVENT(btrfs_add_block_group,
                 { BTRFS_DROP_DELAYED_REF,   "DROP_DELAYED_REF" },       \
                 { BTRFS_ADD_DELAYED_EXTENT, "ADD_DELAYED_EXTENT" },     \
                 { BTRFS_UPDATE_DELAYED_HEAD, "UPDATE_DELAYED_HEAD" })
-
+

Unrelated white space change. Please avoid that.

That seems to be caused by clangd lsp server.

I have no idea why it automatically cleaned up the whitespace problems every time I modified the header file.
And IIRC it's not the first time it's doing this.

I'd better find a way to disable the auto fix.

Thanks,
Qu

Quoted 38 lines

Thanks.

Quoted 34 lines
include/trace/events/btrfs.h +3 -4
  DECLARE_EVENT_CLASS(btrfs_delayed_tree_ref,
 
@@ -873,7 +869,7 @@ DECLARE_EVENT_CLASS(btrfs_delayed_tree_ref,
         TP_STRUCT__entry_btrfs(
                 __field(        u64,  bytenr            )
                 __field(        u64,  num_bytes         )
-               __field(        int,  action            )
+               __field(        int,  action            )
                 __field(        u64,  parent            )
                 __field(        u64,  ref_root          )
                 __field(        int,  level             )
@@ -930,7 +926,7 @@ DECLARE_EVENT_CLASS(btrfs_delayed_data_ref,
         TP_STRUCT__entry_btrfs(
                 __field(        u64,  bytenr            )
                 __field(        u64,  num_bytes         )
-               __field(        int,  action            )
+               __field(        int,  action            )
                 __field(        u64,  parent            )
                 __field(        u64,  ref_root          )
                 __field(        u64,  owner             )
@@ -992,7 +988,7 @@ DECLARE_EVENT_CLASS(btrfs_delayed_ref_head,
         TP_STRUCT__entry_btrfs(
                 __field(        u64,  bytenr            )
                 __field(        u64,  num_bytes         )
-               __field(        int,  action            )
+               __field(        int,  action            )
                 __field(        int,  is_data           )
         ),
 
--

2.45.0

author
#20 lore raw

PATCH v2 6/11 btrfs: remove extent_map::block_len member

In reply to Qu Wenruo

The extent_map::block_len is either extent_map::len (non-compressed extent) or extent_map::disk_num_bytes (compressed extent).

Since we already have sanity checks to do the cross-check between the new and old members, we can drop the old extent_map::block_len now.

For most call sites, they can manually select extent_map::len or extent_map::disk_num_bytes, since most if not all of them have checked if the extent is compressed.

---
 fs/btrfs/compression.c            |  2 +-
 fs/btrfs/extent_map.c             | 41 +++++++++++--------------------
 fs/btrfs/extent_map.h             |  9 -------
 fs/btrfs/file-item.c              |  7 ------
 fs/btrfs/file.c                   |  1 -
 fs/btrfs/inode.c                  | 36 +++++++++------------------
 fs/btrfs/relocation.c             |  1 -
 fs/btrfs/tests/extent-map-tests.c | 41 ++++++++++---------------------
 fs/btrfs/tree-log.c               |  4 +--
 include/trace/events/btrfs.h      |  5 +---
 10 files changed, 42 insertions(+), 105 deletions(-)
fs/btrfs/compression.c +1 -1
index a4cd0e743027..3af87911c83e 100644
@@ -585,7 +585,7 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio)
 	}
 
 	ASSERT(extent_map_is_compressed(em));
-	compressed_len = em->block_len;
+	compressed_len = em->disk_num_bytes;
 
 	cb = alloc_compressed_bio(inode, file_offset, REQ_OP_READ,
 				  end_bbio_comprssed_read);
fs/btrfs/extent_map.c +14 -27

120 lines not on this page · raw message

fs/btrfs/extent_map.h +0 -9
index 454a4bb08d95..aee721eaa7f3 100644
@@ -102,15 +102,6 @@ struct extent_map {
 	 */
 	u64 block_start;
 
-	/*
-	 * The on-disk length for the file extent.
-	 *
-	 * For compressed extents it matches btrfs_file_extent_item::disk_num_bytes.
-	 * For uncompressed extents it matches extent_map::len.
-	 * For holes and inline extents it's -1 and shouldn't be used.
-	 */
-	u64 block_len;
-
 	/*
 	 * Generation of the extent map, for merged em it's the highest
 	 * generation of all merged ems.
fs/btrfs/file-item.c +0 -7
index 08d608f0ae5d..95fb7c059a1a 100644
@@ -1306,11 +1306,9 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		if (compress_type != BTRFS_COMPRESS_NONE) {
 			extent_map_set_compression(em, compress_type);
 			em->block_start = bytenr;
-			em->block_len = em->disk_num_bytes;
 		} else {
 			bytenr += btrfs_file_extent_offset(leaf, fi);
 			em->block_start = bytenr;
-			em->block_len = em->len;
 			if (type == BTRFS_FILE_EXTENT_PREALLOC)
 				em->flags |= EXTENT_FLAG_PREALLOC;
 		}
@@ -1323,11 +1321,6 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		em->start = 0;
 		em->len = fs_info->sectorsize;
 		em->offset = 0;
-		/*
-		 * Initialize block_len with the same values
-		 * as in inode.c:btrfs_get_extent().
-		 */
-		em->block_len = (u64)-1;
 		extent_map_set_compression(em, compress_type);
 	} else {
 		btrfs_err(fs_info,
fs/btrfs/file.c +0 -1
index be4e6acb08f3..05c7b5429b85 100644
@@ -2337,7 +2337,6 @@ static int fill_holes(struct btrfs_trans_handle *trans,
 
 		hole_em->block_start = EXTENT_MAP_HOLE;
 		hole_em->disk_bytenr = EXTENT_MAP_HOLE;
-		hole_em->block_len = 0;
 		hole_em->disk_num_bytes = 0;
 		hole_em->generation = trans->transid;
 
fs/btrfs/inode.c +11 -25
index d1c948ea1421..7df295e0046d 100644
@@ -139,7 +139,7 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
 				     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 				       u64 len, u64 block_start,
-				       u64 block_len, u64 disk_num_bytes,
+				       u64 disk_num_bytes,
 				       u64 ram_bytes, int compress_type,
 				       struct btrfs_file_extent *file_extent,
 				       int type);
@@ -1210,7 +1210,6 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
 	em = create_io_em(inode, start,
 			  async_extent->ram_size,	/* len */
 			  ins.objectid,			/* block_start */
-			  ins.offset,			/* block_len */
 			  ins.offset,			/* orig_block_len */
 			  async_extent->ram_size,	/* ram_bytes */
 			  async_extent->compress_type,
@@ -1453,7 +1452,6 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
 
 		em = create_io_em(inode, start, ins.offset, /* len */
 				  ins.objectid, /* block_start */
-				  ins.offset, /* block_len */
 				  ins.offset, /* orig_block_len */
 				  ram_size, /* ram_bytes */
 				  BTRFS_COMPRESS_NONE, /* compress_type */
@@ -2194,7 +2192,6 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
 
 			em = create_io_em(inode, cur_offset, nocow_args.num_bytes,
 					  nocow_args.disk_bytenr, /* block_start */
-					  nocow_args.num_bytes, /* block_len */
 					  nocow_args.disk_num_bytes, /* orig_block_len */
 					  ram_bytes, BTRFS_COMPRESS_NONE,
 					  &nocow_args.file_extent,
@@ -4998,7 +4995,6 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
 
 			hole_em->block_start = EXTENT_MAP_HOLE;
 			hole_em->disk_bytenr = EXTENT_MAP_HOLE;
-			hole_em->block_len = 0;
 			hole_em->disk_num_bytes = 0;
 			hole_em->ram_bytes = hole_size;
 			hole_em->generation = btrfs_get_fs_generation(fs_info);
@@ -6859,7 +6855,6 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
 	em->start = EXTENT_MAP_HOLE;
 	em->disk_bytenr = EXTENT_MAP_HOLE;
 	em->len = (u64)-1;
-	em->block_len = (u64)-1;
 
 	path = btrfs_alloc_path();
 	if (!path) {
@@ -7017,7 +7012,6 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
 						  const u64 start,
 						  const u64 len,
 						  const u64 block_start,
-						  const u64 block_len,
 						  const u64 orig_block_len,
 						  const u64 ram_bytes,
 						  const int type,
@@ -7028,14 +7022,14 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
 
 	if (type != BTRFS_ORDERED_NOCOW) {
 		em = create_io_em(inode, start, len, block_start,
-				  block_len, orig_block_len, ram_bytes,
+				  orig_block_len, ram_bytes,
 				  BTRFS_COMPRESS_NONE, /* compress_type */
 				  file_extent, type);
 		if (IS_ERR(em))
 			goto out;
 	}
 	ordered = btrfs_alloc_ordered_extent(inode, start, len, len,
-					     block_start, block_len, 0,
+					     block_start, len, 0,
 					     (1 << type) |
 					     (1 << BTRFS_ORDERED_DIRECT),
 					     BTRFS_COMPRESS_NONE);
@@ -7087,7 +7081,7 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
 	file_extent.offset = 0;
 	file_extent.compression = BTRFS_COMPRESS_NONE;
 	em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset,
-				     ins.objectid, ins.offset, ins.offset,
+				     ins.objectid, ins.offset,
 				     ins.offset, BTRFS_ORDERED_REGULAR,
 				     &file_extent);
 	btrfs_dec_block_group_reservations(fs_info, ins.objectid);
@@ -7328,7 +7322,7 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
 /* The callers of this must take lock_extent() */
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 				       u64 len, u64 block_start,
-				       u64 block_len, u64 disk_num_bytes,
+				       u64 disk_num_bytes,
 				       u64 ram_bytes, int compress_type,
 				       struct btrfs_file_extent *file_extent,
 				       int type)
@@ -7350,16 +7344,10 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 
 	switch (type) {
 	case BTRFS_ORDERED_PREALLOC:
-		/* Uncompressed extents. */
-		ASSERT(block_len == len);
-
 		/* We're only referring part of a larger preallocated extent. */
-		ASSERT(block_len <= ram_bytes);
+		ASSERT(len <= ram_bytes);
 		break;
 	case BTRFS_ORDERED_REGULAR:
-		/* Uncompressed extents. */
-		ASSERT(block_len == len);
-
 		/* COW results a new extent matching our file extent size. */
 		ASSERT(disk_num_bytes == len);
 		ASSERT(ram_bytes == len);
@@ -7385,7 +7373,6 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 
 	em->start = start;
 	em->len = len;
-	em->block_len = block_len;
 	em->block_start = block_start;
 	em->disk_bytenr = file_extent->disk_bytenr;
 	em->disk_num_bytes = disk_num_bytes;
@@ -7474,7 +7461,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 
 		em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
 					      block_start,
-					      len, orig_block_len,
+					      orig_block_len,
 					      ram_bytes, type,
 					      &file_extent);
 		btrfs_dec_nocow_writers(bg);
@@ -9628,7 +9615,6 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
 		em->block_start = ins.objectid;
 		em->disk_bytenr = ins.objectid;
 		em->offset = 0;
-		em->block_len = ins.offset;
 		em->disk_num_bytes = ins.offset;
 		em->ram_bytes = ins.offset;
 		em->flags |= EXTENT_FLAG_PREALLOC;
@@ -10125,12 +10111,12 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter,
 		 * Bail if the buffer isn't large enough to return the whole
 		 * compressed extent.
 		 */
-		if (em->block_len > count) {
+		if (em->disk_num_bytes > count) {
 			ret = -ENOBUFS;
 			goto out_em;
 		}
-		disk_io_size = em->block_len;
-		count = em->block_len;
+		disk_io_size = em->disk_num_bytes;
+		count = em->disk_num_bytes;
 		encoded->unencoded_len = em->ram_bytes;
 		encoded->unencoded_offset = iocb->ki_pos - em->start + em->offset;
 		ret = btrfs_encoded_io_compression_from_extent(fs_info,
@@ -10378,7 +10364,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
 	file_extent.compression = compression;
 	em = create_io_em(inode, start, num_bytes,
 			  ins.objectid,
-			  ins.offset, ins.offset, ram_bytes, compression,
+			  ins.offset, ram_bytes, compression,
 			  &file_extent, BTRFS_ORDERED_COMPRESSED);
 	if (IS_ERR(em)) {
 		ret = PTR_ERR(em);
fs/btrfs/relocation.c +0 -1
index 33662b3aad38..a66d9b921f84 100644
@@ -2912,7 +2912,6 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inod
 
 	em->start = start;
 	em->len = end + 1 - start;
-	em->block_len = em->len;
 	em->block_start = block_start;
 	em->disk_bytenr = block_start;
 	em->disk_num_bytes = em->len;
fs/btrfs/tests/extent-map-tests.c +13 -28

169 lines not on this page · raw message

fs/btrfs/tree-log.c +2 -2
index c9e8c5f96b1c..13f35180e3a0 100644
@@ -4648,7 +4648,7 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
 	/* If we're compressed we have to save the entire range of csums. */
 	if (extent_map_is_compressed(em)) {
 		csum_offset = 0;
-		csum_len = max(em->block_len, em->disk_num_bytes);
+		csum_len = em->disk_num_bytes;
 	} else {
 		csum_offset = mod_start - em->start;
 		csum_len = mod_len;
@@ -4698,7 +4698,7 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
 	else
 		btrfs_set_stack_file_extent_type(&fi, BTRFS_FILE_EXTENT_REG);
 
-	block_len = max(em->block_len, em->disk_num_bytes);
+	block_len = em->disk_num_bytes;
 	compress_type = extent_map_compression(em);
 	if (compress_type != BTRFS_COMPRESS_NONE) {
 		btrfs_set_stack_file_extent_disk_bytenr(&fi, em->block_start);
include/trace/events/btrfs.h +1 -4
index 6dacdc1fb63e..3743719d13f2 100644
@@ -292,7 +292,6 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
 		__field(	u64,  start		)
 		__field(	u64,  len		)
 		__field(	u64,  block_start	)
-		__field(	u64,  block_len		)
 		__field(	u32,  flags		)
 		__field(	int,  refs		)
 	),
@@ -303,19 +302,17 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
 		__entry->start		= map->start;
 		__entry->len		= map->len;
 		__entry->block_start	= map->block_start;
-		__entry->block_len	= map->block_len;
 		__entry->flags		= map->flags;
 		__entry->refs		= refcount_read(&map->refs);
 	),
 
 	TP_printk_btrfs("root=%llu(%s) ino=%llu start=%llu len=%llu "
-		  "block_start=%llu(%s) block_len=%llu flags=%s refs=%u",
+		  "block_start=%llu(%s) flags=%s refs=%u",
 		  show_root_type(__entry->root_objectid),
 		  __entry->ino,
 		  __entry->start,
 		  __entry->len,
 		  show_map_type(__entry->block_start),
-		  __entry->block_len,
 		  show_map_flags(__entry->flags),
 		  __entry->refs)
 );
Signature · 1 line
2.45.0
#21 lore raw

Re: [PATCH v2 06/11] btrfs: remove extent_map::block_len member

Quoted 157 lines · Qu Wenruo

The extent_map::block_len is either extent_map::len (non-compressed extent) or extent_map::disk_num_bytes (compressed extent).

Since we already have sanity checks to do the cross-check between the new and old members, we can drop the old extent_map::block_len now.

For most call sites, they can manually select extent_map::len or extent_map::disk_num_bytes, since most if not all of them have checked if the extent is compressed.

---
 fs/btrfs/compression.c            |  2 +-
 fs/btrfs/extent_map.c             | 41 +++++++++++--------------------
 fs/btrfs/extent_map.h             |  9 -------
 fs/btrfs/file-item.c              |  7 ------
 fs/btrfs/file.c                   |  1 -
 fs/btrfs/inode.c                  | 36 +++++++++------------------
 fs/btrfs/relocation.c             |  1 -
 fs/btrfs/tests/extent-map-tests.c | 41 ++++++++++---------------------
 fs/btrfs/tree-log.c               |  4 +--
 include/trace/events/btrfs.h      |  5 +---
 10 files changed, 42 insertions(+), 105 deletions(-)
fs/btrfs/compression.c +1 -1
index a4cd0e743027..3af87911c83e 100644
@@ -585,7 +585,7 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio)
        }
 
        ASSERT(extent_map_is_compressed(em));
-       compressed_len = em->block_len;
+       compressed_len = em->disk_num_bytes;
 
        cb = alloc_compressed_bio(inode, file_offset, REQ_OP_READ,
                                  end_bbio_comprssed_read);
fs/btrfs/extent_map.c +14 -24
index dc73b8a81271..dcd191c2c4b3 100644
@@ -186,11 +186,18 @@ static struct rb_node *__tree_search(struct rb_root *root, u64 offset,
        return NULL;
 }
 
+static inline u64 extent_map_block_len(const struct extent_map *em)
+{
+       if (extent_map_is_compressed(em))
+               return em->disk_num_bytes;
+       return em->len;
+}
+
 static inline u64 extent_map_block_end(const struct extent_map *em)
 {
-       if (em->block_start + em->block_len < em->block_start)
+       if (em->block_start + extent_map_block_len(em) < em->block_start)
                return (u64)-1;
-       return em->block_start + em->block_len;
+       return em->block_start + extent_map_block_len(em);
 }
 
 static bool can_merge_extent_map(const struct extent_map *em)
@@ -288,10 +295,10 @@ static void dump_extent_map(const char *prefix, struct extent_map *em)
 {
        if (!IS_ENABLED(CONFIG_BTRFS_DEBUG))
                return;
-       pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu block_start=%llu block_len=%llu flags=0x%x\n",
+       pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu block_start=%llu flags=0x%x\n",
                prefix, em->start, em->len, em->disk_bytenr, em->disk_num_bytes,
                em->ram_bytes, em->offset, em->block_start,
-               em->block_len, em->flags);
+               em->flags);
        ASSERT(0);
 }
 
@@ -313,9 +320,6 @@ static void validate_extent_map(struct extent_map *em)
                        if (em->block_start != em->disk_bytenr)
                                dump_extent_map(
                                "mismatch block_start/disk_bytenr/offset", em);
-                       if (em->disk_num_bytes != em->block_len)
-                               dump_extent_map(
-                               "mismatch disk_num_bytes/block_len", em);
                } else {
                        if (em->block_start != em->disk_bytenr + em->offset)
                                dump_extent_map(
@@ -354,7 +358,6 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                if (rb && can_merge_extent_map(merge) && mergeable_maps(merge, em)) {
                        em->start = merge->start;
                        em->len += merge->len;
-                       em->block_len += merge->block_len;
                        em->block_start = merge->block_start;
                        em->generation = max(em->generation, merge->generation);
 
@@ -375,7 +378,6 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                merge = rb_entry(rb, struct extent_map, rb_node);
        if (rb && can_merge_extent_map(merge) && mergeable_maps(em, merge)) {
                em->len += merge->len;
-               em->block_len += merge->block_len;
                if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
                        merge_ondisk_extents(em, merge);
                validate_extent_map(em);
@@ -669,7 +671,6 @@ static noinline int merge_extent_mapping(struct btrfs_inode *inode,
        if (em->block_start < EXTENT_MAP_LAST_BYTE &&
            !extent_map_is_compressed(em)) {
                em->block_start += start_diff;
-               em->block_len = em->len;
                em->offset += start_diff;
        }
        return add_extent_mapping(inode, em, 0);
@@ -884,17 +885,11 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        if (em->block_start < EXTENT_MAP_LAST_BYTE) {
                                split->block_start = em->block_start;
 
-                               if (compressed)
-                                       split->block_len = em->block_len;
-                               else
-                                       split->block_len = split->len;
                                split->disk_bytenr = em->disk_bytenr;
-                               split->disk_num_bytes = max(split->block_len,
-                                                           em->disk_num_bytes);
+                               split->disk_num_bytes = em->disk_num_bytes;
                                split->offset = em->offset;
                                split->ram_bytes = em->ram_bytes;
                        } else {
-                               split->block_len = 0;
                                split->block_start = em->block_start;
                                split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = 0;
@@ -924,23 +919,18 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        split->generation = gen;
 
                        if (em->block_start < EXTENT_MAP_LAST_BYTE) {
-                               split->disk_num_bytes = max(em->block_len,
-                                                           em->disk_num_bytes);
+                               split->disk_num_bytes = em->disk_num_bytes;
                                split->offset = em->offset + end - em->start;
                                split->ram_bytes = em->ram_bytes;
-                               if (compressed) {
-                                       split->block_len = em->block_len;
-                               } else {
+                               if (!compressed) {
                                        const u64 diff = end - em->start;
 
-                                       split->block_len = split->len;
                                        split->block_start += diff;
                                }
                        } else {
                                split->disk_num_bytes = 0;
                                split->offset = 0;
                                split->ram_bytes = split->len;
-                               split->block_len = 0;
                        }
 
                        if (extent_map_in_tree(em)) {

fs/btrfs/extent_map.c int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre, · line 1102

Quoted 5 lines
fs/btrfs/extent_map.c +0 -2
@@ -1098,8 +1088,6 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        split_pre->disk_num_bytes = split_pre->len;
        split_pre->offset = 0;
        split_pre->block_start = new_logical;
-       split_pre->block_len = split_pre->len;
-       split_pre->disk_num_bytes = split_pre->block_len;

Ok so this relates to my comment on patch 3, which added the first ->disk_num_bytes assignment, a few lines above and visible in this diff, which was useless because of this one.

This is hard to follow. Some patches add code just to be removed a few patches later.

Quoted 499 lines
fs/btrfs/extent_map.c +0 -1
        split_pre->ram_bytes = split_pre->len;
        split_pre->flags = flags;
        split_pre->generation = em->generation;
@@ -1118,7 +1106,6 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        split_mid->disk_num_bytes = split_mid->len;
        split_mid->offset = 0;
        split_mid->block_start = em->block_start + pre;
-       split_mid->block_len = split_mid->len;
        split_mid->ram_bytes = split_mid->len;
        split_mid->flags = flags;
        split_mid->generation = em->generation;
fs/btrfs/extent_map.h +0 -9
index 454a4bb08d95..aee721eaa7f3 100644
@@ -102,15 +102,6 @@ struct extent_map {
         */
        u64 block_start;
 
-       /*
-        * The on-disk length for the file extent.
-        *
-        * For compressed extents it matches btrfs_file_extent_item::disk_num_bytes.
-        * For uncompressed extents it matches extent_map::len.
-        * For holes and inline extents it's -1 and shouldn't be used.
-        */
-       u64 block_len;
-
        /*
         * Generation of the extent map, for merged em it's the highest
         * generation of all merged ems.
fs/btrfs/file-item.c +0 -7
index 08d608f0ae5d..95fb7c059a1a 100644
@@ -1306,11 +1306,9 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                if (compress_type != BTRFS_COMPRESS_NONE) {
                        extent_map_set_compression(em, compress_type);
                        em->block_start = bytenr;
-                       em->block_len = em->disk_num_bytes;
                } else {
                        bytenr += btrfs_file_extent_offset(leaf, fi);
                        em->block_start = bytenr;
-                       em->block_len = em->len;
                        if (type == BTRFS_FILE_EXTENT_PREALLOC)
                                em->flags |= EXTENT_FLAG_PREALLOC;
                }
@@ -1323,11 +1321,6 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                em->start = 0;
                em->len = fs_info->sectorsize;
                em->offset = 0;
-               /*
-                * Initialize block_len with the same values
-                * as in inode.c:btrfs_get_extent().
-                */
-               em->block_len = (u64)-1;
                extent_map_set_compression(em, compress_type);
        } else {
                btrfs_err(fs_info,
fs/btrfs/file.c +0 -1
index be4e6acb08f3..05c7b5429b85 100644
@@ -2337,7 +2337,6 @@ static int fill_holes(struct btrfs_trans_handle *trans,
 
                hole_em->block_start = EXTENT_MAP_HOLE;
                hole_em->disk_bytenr = EXTENT_MAP_HOLE;
-               hole_em->block_len = 0;
                hole_em->disk_num_bytes = 0;
                hole_em->generation = trans->transid;
 
fs/btrfs/inode.c +11 -25
index d1c948ea1421..7df295e0046d 100644
@@ -139,7 +139,7 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
                                     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 block_start,
-                                      u64 block_len, u64 disk_num_bytes,
+                                      u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
                                       struct btrfs_file_extent *file_extent,
                                       int type);
@@ -1210,7 +1210,6 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        em = create_io_em(inode, start,
                          async_extent->ram_size,       /* len */
                          ins.objectid,                 /* block_start */
-                         ins.offset,                   /* block_len */
                          ins.offset,                   /* orig_block_len */
                          async_extent->ram_size,       /* ram_bytes */
                          async_extent->compress_type,
@@ -1453,7 +1452,6 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
 
                em = create_io_em(inode, start, ins.offset, /* len */
                                  ins.objectid, /* block_start */
-                                 ins.offset, /* block_len */
                                  ins.offset, /* orig_block_len */
                                  ram_size, /* ram_bytes */
                                  BTRFS_COMPRESS_NONE, /* compress_type */
@@ -2194,7 +2192,6 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
 
                        em = create_io_em(inode, cur_offset, nocow_args.num_bytes,
                                          nocow_args.disk_bytenr, /* block_start */
-                                         nocow_args.num_bytes, /* block_len */
                                          nocow_args.disk_num_bytes, /* orig_block_len */
                                          ram_bytes, BTRFS_COMPRESS_NONE,
                                          &nocow_args.file_extent,
@@ -4998,7 +4995,6 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
 
                        hole_em->block_start = EXTENT_MAP_HOLE;
                        hole_em->disk_bytenr = EXTENT_MAP_HOLE;
-                       hole_em->block_len = 0;
                        hole_em->disk_num_bytes = 0;
                        hole_em->ram_bytes = hole_size;
                        hole_em->generation = btrfs_get_fs_generation(fs_info);
@@ -6859,7 +6855,6 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
        em->start = EXTENT_MAP_HOLE;
        em->disk_bytenr = EXTENT_MAP_HOLE;
        em->len = (u64)-1;
-       em->block_len = (u64)-1;
 
        path = btrfs_alloc_path();
        if (!path) {
@@ -7017,7 +7012,6 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                                                  const u64 start,
                                                  const u64 len,
                                                  const u64 block_start,
-                                                 const u64 block_len,
                                                  const u64 orig_block_len,
                                                  const u64 ram_bytes,
                                                  const int type,
@@ -7028,14 +7022,14 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
 
        if (type != BTRFS_ORDERED_NOCOW) {
                em = create_io_em(inode, start, len, block_start,
-                                 block_len, orig_block_len, ram_bytes,
+                                 orig_block_len, ram_bytes,
                                  BTRFS_COMPRESS_NONE, /* compress_type */
                                  file_extent, type);
                if (IS_ERR(em))
                        goto out;
        }
        ordered = btrfs_alloc_ordered_extent(inode, start, len, len,
-                                            block_start, block_len, 0,
+                                            block_start, len, 0,
                                             (1 << type) |
                                             (1 << BTRFS_ORDERED_DIRECT),
                                             BTRFS_COMPRESS_NONE);
@@ -7087,7 +7081,7 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
        file_extent.offset = 0;
        file_extent.compression = BTRFS_COMPRESS_NONE;
        em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset,
-                                    ins.objectid, ins.offset, ins.offset,
+                                    ins.objectid, ins.offset,
                                     ins.offset, BTRFS_ORDERED_REGULAR,
                                     &file_extent);
        btrfs_dec_block_group_reservations(fs_info, ins.objectid);
@@ -7328,7 +7322,7 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
 /* The callers of this must take lock_extent() */
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                                       u64 len, u64 block_start,
-                                      u64 block_len, u64 disk_num_bytes,
+                                      u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
                                       struct btrfs_file_extent *file_extent,
                                       int type)
@@ -7350,16 +7344,10 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 
        switch (type) {
        case BTRFS_ORDERED_PREALLOC:
-               /* Uncompressed extents. */
-               ASSERT(block_len == len);
-
                /* We're only referring part of a larger preallocated extent. */
-               ASSERT(block_len <= ram_bytes);
+               ASSERT(len <= ram_bytes);
                break;
        case BTRFS_ORDERED_REGULAR:
-               /* Uncompressed extents. */
-               ASSERT(block_len == len);
-
                /* COW results a new extent matching our file extent size. */
                ASSERT(disk_num_bytes == len);
                ASSERT(ram_bytes == len);
@@ -7385,7 +7373,6 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 
        em->start = start;
        em->len = len;
-       em->block_len = block_len;
        em->block_start = block_start;
        em->disk_bytenr = file_extent->disk_bytenr;
        em->disk_num_bytes = disk_num_bytes;
@@ -7474,7 +7461,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 
                em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
                                              block_start,
-                                             len, orig_block_len,
+                                             orig_block_len,
                                              ram_bytes, type,
                                              &file_extent);
                btrfs_dec_nocow_writers(bg);
@@ -9628,7 +9615,6 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
                em->block_start = ins.objectid;
                em->disk_bytenr = ins.objectid;
                em->offset = 0;
-               em->block_len = ins.offset;
                em->disk_num_bytes = ins.offset;
                em->ram_bytes = ins.offset;
                em->flags |= EXTENT_FLAG_PREALLOC;
@@ -10125,12 +10111,12 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter,
                 * Bail if the buffer isn't large enough to return the whole
                 * compressed extent.
                 */
-               if (em->block_len > count) {
+               if (em->disk_num_bytes > count) {
                        ret = -ENOBUFS;
                        goto out_em;
                }
-               disk_io_size = em->block_len;
-               count = em->block_len;
+               disk_io_size = em->disk_num_bytes;
+               count = em->disk_num_bytes;
                encoded->unencoded_len = em->ram_bytes;
                encoded->unencoded_offset = iocb->ki_pos - em->start + em->offset;
                ret = btrfs_encoded_io_compression_from_extent(fs_info,
@@ -10378,7 +10364,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
        file_extent.compression = compression;
        em = create_io_em(inode, start, num_bytes,
                          ins.objectid,
-                         ins.offset, ins.offset, ram_bytes, compression,
+                         ins.offset, ram_bytes, compression,
                          &file_extent, BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
fs/btrfs/relocation.c +0 -1
index 33662b3aad38..a66d9b921f84 100644
@@ -2912,7 +2912,6 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inod
 
        em->start = start;
        em->len = end + 1 - start;
-       em->block_len = em->len;
        em->block_start = block_start;
        em->disk_bytenr = block_start;
        em->disk_num_bytes = em->len;
fs/btrfs/tests/extent-map-tests.c +13 -28
index bd56efe37f02..ffdaa6a682af 100644
@@ -28,9 +28,10 @@ static int free_extent_map_tree(struct btrfs_inode *inode)
                if (refcount_read(&em->refs) != 1) {
                        ret = -EINVAL;
                        test_err(
-"em leak: em (start %llu len %llu block_start %llu block_len %llu) refs %d",
+"em leak: em (start %llu len %llu block_start %llu disk_num_bytes %llu offset %llu) refs %d",
                                 em->start, em->len, em->block_start,
-                                em->block_len, refcount_read(&em->refs));
+                                em->disk_num_bytes, em->offset,
+                                refcount_read(&em->refs));
 
                        refcount_set(&em->refs, 1);
                }
@@ -77,7 +78,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->start = 0;
        em->len = SZ_16K;
        em->block_start = 0;
-       em->block_len = SZ_16K;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_16K;
        em->ram_bytes = SZ_16K;
@@ -101,7 +101,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->start = SZ_16K;
        em->len = SZ_4K;
        em->block_start = SZ_32K; /* avoid merging */
-       em->block_len = SZ_4K;
        em->disk_bytenr = SZ_32K; /* avoid merging */
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = SZ_4K;
@@ -125,7 +124,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->start = start;
        em->len = len;
        em->block_start = start;
-       em->block_len = len;
        em->disk_bytenr = start;
        em->disk_num_bytes = len;
        em->ram_bytes = len;
@@ -143,11 +141,11 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
                goto out;
        }
        if (em->start != 0 || extent_map_end(em) != SZ_16K ||
-           em->block_start != 0 || em->block_len != SZ_16K) {
+           em->block_start != 0 || em->disk_num_bytes != SZ_16K) {
                test_err(
-"case1 [%llu %llu]: ret %d return a wrong em (start %llu len %llu block_start %llu block_len %llu",
+"case1 [%llu %llu]: ret %d return a wrong em (start %llu len %llu block_start %llu disk_num_bytes %llu",
                         start, start + len, ret, em->start, em->len,
-                        em->block_start, em->block_len);
+                        em->block_start, em->disk_num_bytes);
                ret = -EINVAL;
        }
        free_extent_map(em);
@@ -182,7 +180,6 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->start = 0;
        em->len = SZ_1K;
        em->block_start = EXTENT_MAP_INLINE;
-       em->block_len = (u64)-1;
        em->disk_bytenr = EXTENT_MAP_INLINE;
        em->disk_num_bytes = 0;
        em->ram_bytes = SZ_1K;
@@ -206,7 +203,6 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->start = SZ_4K;
        em->len = SZ_4K;
        em->block_start = SZ_4K;
-       em->block_len = SZ_4K;
        em->disk_bytenr = SZ_4K;
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = SZ_4K;
@@ -230,7 +226,6 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->start = 0;
        em->len = SZ_1K;
        em->block_start = EXTENT_MAP_INLINE;
-       em->block_len = (u64)-1;
        em->disk_bytenr = EXTENT_MAP_INLINE;
        em->disk_num_bytes = 0;
        em->ram_bytes = SZ_1K;
@@ -247,11 +242,10 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
                goto out;
        }
        if (em->start != 0 || extent_map_end(em) != SZ_1K ||
-           em->block_start != EXTENT_MAP_INLINE || em->block_len != (u64)-1) {
+           em->block_start != EXTENT_MAP_INLINE) {
                test_err(
-"case2 [0 1K]: ret %d return a wrong em (start %llu len %llu block_start %llu block_len %llu",
-                        ret, em->start, em->len, em->block_start,
-                        em->block_len);
+"case2 [0 1K]: ret %d return a wrong em (start %llu len %llu block_start %llu",
+                        ret, em->start, em->len, em->block_start);
                ret = -EINVAL;
        }
        free_extent_map(em);
@@ -282,7 +276,6 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
        em->start = SZ_4K;
        em->len = SZ_4K;
        em->block_start = SZ_4K;
-       em->block_len = SZ_4K;
        em->disk_bytenr = SZ_4K;
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = SZ_4K;
@@ -306,7 +299,6 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
        em->start = 0;
        em->len = SZ_16K;
        em->block_start = 0;
-       em->block_len = SZ_16K;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_16K;
        em->ram_bytes = SZ_16K;
@@ -329,11 +321,11 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
         * em->start.
         */
        if (start < em->start || start + len > extent_map_end(em) ||
-           em->start != em->block_start || em->len != em->block_len) {
+           em->start != em->block_start) {
                test_err(
 "case3 [%llu %llu): ret %d em (start %llu len %llu block_start %llu block_len %llu)",
                         start, start + len, ret, em->start, em->len,
-                        em->block_start, em->block_len);
+                        em->block_start, em->disk_num_bytes);
                ret = -EINVAL;
        }
        free_extent_map(em);
@@ -395,7 +387,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        em->start = 0;
        em->len = SZ_8K;
        em->block_start = 0;
-       em->block_len = SZ_8K;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_8K;
        em->ram_bytes = SZ_8K;
@@ -419,7 +410,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        em->start = SZ_8K;
        em->len = 24 * SZ_1K;
        em->block_start = SZ_16K; /* avoid merging */
-       em->block_len = 24 * SZ_1K;
        em->disk_bytenr = SZ_16K; /* avoid merging */
        em->disk_num_bytes = 24 * SZ_1K;
        em->ram_bytes = 24 * SZ_1K;
@@ -442,7 +432,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        em->start = 0;
        em->len = SZ_32K;
        em->block_start = 0;
-       em->block_len = SZ_32K;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_32K;
        em->ram_bytes = SZ_32K;
@@ -462,9 +451,9 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        }
        if (start < em->start || start + len > extent_map_end(em)) {
                test_err(
-"case4 [%llu %llu): ret %d, added wrong em (start %llu len %llu block_start %llu block_len %llu)",
+"case4 [%llu %llu): ret %d, added wrong em (start %llu len %llu block_start %llu disk_num_bytes %llu)",
                         start, start + len, ret, em->start, em->len, em->block_start,
-                        em->block_len);
+                        em->disk_num_bytes);
                ret = -EINVAL;
        }
        free_extent_map(em);
@@ -529,7 +518,6 @@ static int add_compressed_extent(struct btrfs_inode *inode,
        em->start = start;
        em->len = len;
        em->block_start = block_start;
-       em->block_len = SZ_4K;
        em->disk_bytenr = block_start;
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = len;
@@ -753,7 +741,6 @@ static int test_case_6(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->start = SZ_4K;
        em->len = SZ_4K;
        em->block_start = SZ_16K;
-       em->block_len = SZ_16K;
        em->disk_bytenr = SZ_16K;
        em->disk_num_bytes = SZ_16K;
        em->ram_bytes = SZ_16K;
@@ -809,7 +796,6 @@ static int test_case_7(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->start = 0;
        em->len = SZ_16K;
        em->block_start = 0;
-       em->block_len = SZ_4K;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = SZ_16K;
@@ -834,7 +820,6 @@ static int test_case_7(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        em->start = SZ_32K;
        em->len = SZ_16K;
        em->block_start = SZ_32K;
-       em->block_len = SZ_16K;
        em->disk_bytenr = SZ_32K;
        em->disk_num_bytes = SZ_16K;
        em->ram_bytes = SZ_16K;
fs/btrfs/tree-log.c +2 -2
index c9e8c5f96b1c..13f35180e3a0 100644
@@ -4648,7 +4648,7 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
        /* If we're compressed we have to save the entire range of csums. */
        if (extent_map_is_compressed(em)) {
                csum_offset = 0;
-               csum_len = max(em->block_len, em->disk_num_bytes);
+               csum_len = em->disk_num_bytes;
        } else {
                csum_offset = mod_start - em->start;
                csum_len = mod_len;
@@ -4698,7 +4698,7 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
        else
                btrfs_set_stack_file_extent_type(&fi, BTRFS_FILE_EXTENT_REG);
 
-       block_len = max(em->block_len, em->disk_num_bytes);
+       block_len = em->disk_num_bytes;
        compress_type = extent_map_compression(em);
        if (compress_type != BTRFS_COMPRESS_NONE) {
                btrfs_set_stack_file_extent_disk_bytenr(&fi, em->block_start);
include/trace/events/btrfs.h +1 -5
index 6dacdc1fb63e..3743719d13f2 100644
@@ -292,7 +292,6 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
                __field(        u64,  start             )
                __field(        u64,  len               )
                __field(        u64,  block_start       )
-               __field(        u64,  block_len         )
                __field(        u32,  flags             )
                __field(        int,  refs              )
        ),
@@ -303,19 +302,17 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
                __entry->start          = map->start;
                __entry->len            = map->len;
                __entry->block_start    = map->block_start;
-               __entry->block_len      = map->block_len;
                __entry->flags          = map->flags;
                __entry->refs           = refcount_read(&map->refs);
        ),
 
        TP_printk_btrfs("root=%llu(%s) ino=%llu start=%llu len=%llu "
-                 "block_start=%llu(%s) block_len=%llu flags=%s refs=%u",
+                 "block_start=%llu(%s) flags=%s refs=%u",
                  show_root_type(__entry->root_objectid),
                  __entry->ino,
                  __entry->start,
                  __entry->len,
                  show_map_type(__entry->block_start),
-                 __entry->block_len,
                  show_map_flags(__entry->flags),
                  __entry->refs)
 );
--

2.45.0

#22 lore raw

Re: [PATCH v2 06/11] btrfs: remove extent_map::block_len member

在 2024/5/14 03:14, Filipe Manana 写道:
[...]

Quoted 11 lines

fs/btrfs/extent_map.c int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre, · line 1102

Quoted 6 lines
fs/btrfs/extent_map.c +0 -2
@@ -1098,8 +1088,6 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
         split_pre->disk_num_bytes = split_pre->len;
         split_pre->offset = 0;
         split_pre->block_start = new_logical;
-       split_pre->block_len = split_pre->len;
-       split_pre->disk_num_bytes = split_pre->block_len;

Ok so this relates to my comment on patch 3, which added the first ->disk_num_bytes assignment, a few lines above and visible in this diff, which was useless because of this one.

Quoted 3 lines

This is hard to follow. Some patches add code just to be removed a few patches later.

Thankfully after addressing the comments in patch 3, this one is gone.

But indeed this is a big problem, I'll re-check the patches again manually to ensure the deletion is only there when really needed.

Thanks,
Qu

author
#23 lore raw

PATCH v2 7/11 btrfs: remove extent_map::block_start member

In reply to Qu Wenruo

The member extent_map::block_start can be calculated from extent_map::disk_bytenr + extent_map::offset for regular extents. And otherwise just extent_map::disk_bytenr.

And this is already validated by the validate_extent_map(). Now we can remove the member.

However there is a special case in btrfs_alloc_ordered_extent(), where we intentionally pass a pseudo ordered extent, exploiting the fact that for NOCOW/PREALLOC writes we do not rely on ordered extent members to update the file extent items.

And that's the only way to pass btrfs_extract_ordered_extent(), as it doesn't accept any ordered extent with an offset.

For now we will keep the old pseudo ordered extent members, and leave the cleanup of it for the future.

---
 fs/btrfs/compression.c            |  3 +-
 fs/btrfs/defrag.c                 |  9 ++-
 fs/btrfs/extent_io.c              | 10 ++--
 fs/btrfs/extent_map.c             | 56 +++++------------
 fs/btrfs/extent_map.h             | 22 ++++---
 fs/btrfs/file-item.c              |  4 --
 fs/btrfs/file.c                   | 11 ++--
 fs/btrfs/inode.c                  | 84 +++++++++++++++-----------
 fs/btrfs/relocation.c             |  1 -
 fs/btrfs/tests/extent-map-tests.c | 48 ++++++---------
 fs/btrfs/tests/inode-tests.c      | 99 ++++++++++++++++---------------
 fs/btrfs/tree-log.c               | 17 +++---
 fs/btrfs/zoned.c                  |  4 +-
 include/trace/events/btrfs.h      | 11 +---
 14 files changed, 172 insertions(+), 207 deletions(-)
fs/btrfs/compression.c +2 -1
index 3af87911c83e..414adbba07b8 100644
@@ -507,7 +507,8 @@ static noinline int add_ra_bio_pages(struct inode *inode,
 		 */
 		if (!em || cur < em->start ||
 		    (cur + fs_info->sectorsize > extent_map_end(em)) ||
-		    (em->block_start >> SECTOR_SHIFT) != orig_bio->bi_iter.bi_sector) {
+		    (extent_map_block_start(em) >> SECTOR_SHIFT) !=
+		    orig_bio->bi_iter.bi_sector) {
 			free_extent_map(em);
 			unlock_extent(tree, cur, page_end, NULL);
 			unlock_page(page);
fs/btrfs/defrag.c +4 -5
index 025e7f853a68..6fb94e897fc5 100644
@@ -707,7 +707,6 @@ static struct extent_map *defrag_get_extent(struct btrfs_inode *inode,
 		 */
 		if (key.offset > start) {
 			em->start = start;
-			em->block_start = EXTENT_MAP_HOLE;
 			em->disk_bytenr = EXTENT_MAP_HOLE;
 			em->disk_num_bytes = 0;
 			em->ram_bytes = 0;
@@ -828,7 +827,7 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
 	 */
 	next = defrag_lookup_extent(inode, em->start + em->len, newer_than, locked);
 	/* No more em or hole */
-	if (!next || next->block_start >= EXTENT_MAP_LAST_BYTE)
+	if (!next || next->disk_bytenr >= EXTENT_MAP_LAST_BYTE)
 		goto out;
 	if (next->flags & EXTENT_FLAG_PREALLOC)
 		goto out;
@@ -995,12 +994,12 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
 		 * This is for users who want to convert inline extents to
 		 * regular ones through max_inline= mount option.
 		 */
-		if (em->block_start == EXTENT_MAP_INLINE &&
+		if (em->disk_bytenr == EXTENT_MAP_INLINE &&
 		    em->len <= inode->root->fs_info->max_inline)
 			goto next;
 
 		/* Skip holes and preallocated extents. */
-		if (em->block_start == EXTENT_MAP_HOLE ||
+		if (em->disk_bytenr == EXTENT_MAP_HOLE ||
 		    (em->flags & EXTENT_FLAG_PREALLOC))
 			goto next;
 
@@ -1065,7 +1064,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
 		 * So if an inline extent passed all above checks, just add it
 		 * for defrag, and be converted to regular extents.
 		 */
-		if (em->block_start == EXTENT_MAP_INLINE)
+		if (em->disk_bytenr == EXTENT_MAP_INLINE)
 			goto add;
 
 		next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
fs/btrfs/extent_io.c +5 -5
index 597387e9f040..ef9928cc6467 100644
@@ -1083,10 +1083,10 @@ static int btrfs_do_readpage(struct page *page, struct extent_map **em_cached,
 		iosize = min(extent_map_end(em) - cur, end - cur + 1);
 		iosize = ALIGN(iosize, blocksize);
 		if (compress_type != BTRFS_COMPRESS_NONE)
-			disk_bytenr = em->block_start;
+			disk_bytenr = em->disk_bytenr;
 		else
-			disk_bytenr = em->block_start + extent_offset;
-		block_start = em->block_start;
+			disk_bytenr = extent_map_block_start(em) + extent_offset;
+		block_start = extent_map_block_start(em);
 		if (em->flags & EXTENT_FLAG_PREALLOC)
 			block_start = EXTENT_MAP_HOLE;
 
@@ -1405,8 +1405,8 @@ static noinline_for_stack int __extent_writepage_io(struct btrfs_inode *inode,
 		ASSERT(IS_ALIGNED(em->start, fs_info->sectorsize));
 		ASSERT(IS_ALIGNED(em->len, fs_info->sectorsize));
 
-		block_start = em->block_start;
-		disk_bytenr = em->block_start + extent_offset;
+		block_start = extent_map_block_start(em);
+		disk_bytenr = extent_map_block_start(em) + extent_offset;
 
 		ASSERT(!extent_map_is_compressed(em));
 		ASSERT(block_start != EXTENT_MAP_HOLE);
fs/btrfs/extent_map.c +15 -41

159 lines not on this page · raw message

fs/btrfs/extent_map.h +10 -12
index aee721eaa7f3..a0e7f49ec88f 100644
@@ -90,18 +90,6 @@ struct extent_map {
 	 */
 	u64 ram_bytes;
 
-	/*
-	 * The on-disk logical bytenr for the file extent.
-	 *
-	 * For compressed extents it matches btrfs_file_extent_item::disk_bytenr.
-	 * For uncompressed extents it matches
-	 * btrfs_file_extent_item::disk_bytenr + btrfs_file_extent_item::offset
-	 *
-	 * For holes it is EXTENT_MAP_HOLE and for inline extents it is
-	 * EXTENT_MAP_INLINE.
-	 */
-	u64 block_start;
-
 	/*
 	 * Generation of the extent map, for merged em it's the highest
 	 * generation of all merged ems.
@@ -162,6 +150,16 @@ static inline int extent_map_in_tree(const struct extent_map *em)
 	return !RB_EMPTY_NODE(&em->rb_node);
 }
 
+static inline u64 extent_map_block_start(const struct extent_map *em)
+{
+	if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE) {
+		if (extent_map_is_compressed(em))
+			return em->disk_bytenr;
+		return em->disk_bytenr + em->offset;
+	}
+	return em->disk_bytenr;
+}
+
 static inline u64 extent_map_end(const struct extent_map *em)
 {
 	if (em->start + em->len < em->start)
fs/btrfs/file-item.c +0 -4
index 95fb7c059a1a..b8e213a1a8ee 100644
@@ -1294,7 +1294,6 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		em->len = btrfs_file_extent_end(path) - extent_start;
 		bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
 		if (bytenr == 0) {
-			em->block_start = EXTENT_MAP_HOLE;
 			em->disk_bytenr = EXTENT_MAP_HOLE;
 			em->disk_num_bytes = 0;
 			em->offset = 0;
@@ -1305,10 +1304,8 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		em->offset = btrfs_file_extent_offset(leaf, fi);
 		if (compress_type != BTRFS_COMPRESS_NONE) {
 			extent_map_set_compression(em, compress_type);
-			em->block_start = bytenr;
 		} else {
 			bytenr += btrfs_file_extent_offset(leaf, fi);
-			em->block_start = bytenr;
 			if (type == BTRFS_FILE_EXTENT_PREALLOC)
 				em->flags |= EXTENT_FLAG_PREALLOC;
 		}
@@ -1316,7 +1313,6 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
 		/* Tree-checker has ensured this. */
 		ASSERT(extent_start == 0);
 
-		em->block_start = EXTENT_MAP_INLINE;
 		em->disk_bytenr = EXTENT_MAP_INLINE;
 		em->start = 0;
 		em->len = fs_info->sectorsize;
fs/btrfs/file.c +5 -6
index 05c7b5429b85..102b5c17ece1 100644
@@ -2335,7 +2335,6 @@ static int fill_holes(struct btrfs_trans_handle *trans,
 		hole_em->len = end - offset;
 		hole_em->ram_bytes = hole_em->len;
 
-		hole_em->block_start = EXTENT_MAP_HOLE;
 		hole_em->disk_bytenr = EXTENT_MAP_HOLE;
 		hole_em->disk_num_bytes = 0;
 		hole_em->generation = trans->transid;
@@ -2368,7 +2367,7 @@ static int find_first_non_hole(struct btrfs_inode *inode, u64 *start, u64 *len)
 		return PTR_ERR(em);
 
 	/* Hole or vacuum extent(only exists in no-hole mode) */
-	if (em->block_start == EXTENT_MAP_HOLE) {
+	if (em->disk_bytenr == EXTENT_MAP_HOLE) {
 		ret = 1;
 		*len = em->start + em->len > *start + *len ?
 		       0 : *start + *len - em->start - em->len;
@@ -3025,7 +3024,7 @@ static int btrfs_zero_range_check_range_boundary(struct btrfs_inode *inode,
 	if (IS_ERR(em))
 		return PTR_ERR(em);
 
-	if (em->block_start == EXTENT_MAP_HOLE)
+	if (em->disk_bytenr == EXTENT_MAP_HOLE)
 		ret = RANGE_BOUNDARY_HOLE;
 	else if (em->flags & EXTENT_FLAG_PREALLOC)
 		ret = RANGE_BOUNDARY_PREALLOC_EXTENT;
@@ -3089,7 +3088,7 @@ static int btrfs_zero_range(struct inode *inode,
 		ASSERT(IS_ALIGNED(alloc_start, sectorsize));
 		len = offset + len - alloc_start;
 		offset = alloc_start;
-		alloc_hint = em->block_start + em->len;
+		alloc_hint = extent_map_block_start(em) + em->len;
 	}
 	free_extent_map(em);
 
@@ -3107,7 +3106,7 @@ static int btrfs_zero_range(struct inode *inode,
 							   mode);
 			goto out;
 		}
-		if (len < sectorsize && em->block_start != EXTENT_MAP_HOLE) {
+		if (len < sectorsize && em->disk_bytenr != EXTENT_MAP_HOLE) {
 			free_extent_map(em);
 			ret = btrfs_truncate_block(BTRFS_I(inode), offset, len,
 						   0);
@@ -3320,7 +3319,7 @@ static long btrfs_fallocate(struct file *file, int mode,
 		last_byte = min(extent_map_end(em), alloc_end);
 		actual_end = min_t(u64, extent_map_end(em), offset + len);
 		last_byte = ALIGN(last_byte, blocksize);
-		if (em->block_start == EXTENT_MAP_HOLE ||
+		if (em->disk_bytenr == EXTENT_MAP_HOLE ||
 		    (cur_offset >= inode->i_size &&
 		     !(em->flags & EXTENT_FLAG_PREALLOC))) {
 			const u64 range_len = last_byte - cur_offset;
fs/btrfs/inode.c +49 -35

277 lines not on this page · raw message

fs/btrfs/relocation.c +0 -1
index a66d9b921f84..0d462c2a9c89 100644
@@ -2912,7 +2912,6 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inod
 
 	em->start = start;
 	em->len = end + 1 - start;
-	em->block_start = block_start;
 	em->disk_bytenr = block_start;
 	em->disk_num_bytes = em->len;
 	em->ram_bytes = em->len;
fs/btrfs/tests/extent-map-tests.c +17 -31

179 lines not on this page · raw message

fs/btrfs/tests/inode-tests.c +50 -49

259 lines not on this page · raw message

fs/btrfs/tree-log.c +10 -7
index 13f35180e3a0..c4b67bf993d6 100644
@@ -4575,6 +4575,7 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
 {
 	struct btrfs_ordered_extent *ordered;
 	struct btrfs_root *csum_root;
+	u64 block_start;
 	u64 csum_offset;
 	u64 csum_len;
 	u64 mod_start = em->start;
@@ -4584,7 +4585,7 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
 
 	if (inode->flags & BTRFS_INODE_NODATASUM ||
 	    (em->flags & EXTENT_FLAG_PREALLOC) ||
-	    em->block_start == EXTENT_MAP_HOLE)
+	    em->disk_bytenr == EXTENT_MAP_HOLE)
 		return 0;
 
 	list_for_each_entry(ordered, &ctx->ordered_extents, log_list) {
@@ -4655,9 +4656,10 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
 	}
 
 	/* block start is already adjusted for the file extent offset. */
-	csum_root = btrfs_csum_root(trans->fs_info, em->block_start);
-	ret = btrfs_lookup_csums_list(csum_root, em->block_start + csum_offset,
-				      em->block_start + csum_offset +
+	block_start = extent_map_block_start(em);
+	csum_root = btrfs_csum_root(trans->fs_info, block_start);
+	ret = btrfs_lookup_csums_list(csum_root, block_start + csum_offset,
+				      block_start + csum_offset +
 				      csum_len - 1, &ordered_sums, false);
 	if (ret < 0)
 		return ret;
@@ -4689,6 +4691,7 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
 	struct btrfs_key key;
 	enum btrfs_compression_type compress_type;
 	u64 extent_offset = em->offset;
+	u64 block_start = extent_map_block_start(em);
 	u64 block_len;
 	int ret;
 
@@ -4701,10 +4704,10 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
 	block_len = em->disk_num_bytes;
 	compress_type = extent_map_compression(em);
 	if (compress_type != BTRFS_COMPRESS_NONE) {
-		btrfs_set_stack_file_extent_disk_bytenr(&fi, em->block_start);
+		btrfs_set_stack_file_extent_disk_bytenr(&fi, block_start);
 		btrfs_set_stack_file_extent_disk_num_bytes(&fi, block_len);
-	} else if (em->block_start < EXTENT_MAP_LAST_BYTE) {
-		btrfs_set_stack_file_extent_disk_bytenr(&fi, em->block_start -
+	} else if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE) {
+		btrfs_set_stack_file_extent_disk_bytenr(&fi, block_start -
 							extent_offset);
 		btrfs_set_stack_file_extent_disk_num_bytes(&fi, block_len);
 	}
fs/btrfs/zoned.c +3 -1
index 4b52a8916dbb..6d0f7fc9aaf2 100644
@@ -1778,7 +1778,9 @@ static void btrfs_rewrite_logical_zoned(struct btrfs_ordered_extent *ordered,
 	write_lock(&em_tree->lock);
 	em = search_extent_mapping(em_tree, ordered->file_offset,
 				   ordered->num_bytes);
-	em->block_start = logical;
+	/* The em should be a new COW extent, thus it should not has offset. */
+	ASSERT(em->offset == 0);
+	em->disk_bytenr = logical;
 	free_extent_map(em);
 	write_unlock(&em_tree->lock);
 }
include/trace/events/btrfs.h +2 -9
index 3743719d13f2..89b2b66e5bc0 100644
@@ -291,7 +291,6 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
 		__field(	u64,  ino		)
 		__field(	u64,  start		)
 		__field(	u64,  len		)
-		__field(	u64,  block_start	)
 		__field(	u32,  flags		)
 		__field(	int,  refs		)
 	),
@@ -301,18 +300,16 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
 		__entry->ino		= btrfs_ino(inode);
 		__entry->start		= map->start;
 		__entry->len		= map->len;
-		__entry->block_start	= map->block_start;
 		__entry->flags		= map->flags;
 		__entry->refs		= refcount_read(&map->refs);
 	),
 
 	TP_printk_btrfs("root=%llu(%s) ino=%llu start=%llu len=%llu "
-		  "block_start=%llu(%s) flags=%s refs=%u",
+		  "flags=%s refs=%u",
 		  show_root_type(__entry->root_objectid),
 		  __entry->ino,
 		  __entry->start,
 		  __entry->len,
-		  show_map_type(__entry->block_start),
 		  show_map_flags(__entry->flags),
 		  __entry->refs)
 );
@@ -2608,7 +2605,6 @@ TRACE_EVENT(btrfs_extent_map_shrinker_remove_em,
 		__field(	u64,	root_id		)
 		__field(	u64,	start		)
 		__field(	u64,	len		)
-		__field(	u64,	block_start	)
 		__field(	u32,	flags		)
 	),
 
@@ -2617,15 +2613,12 @@ TRACE_EVENT(btrfs_extent_map_shrinker_remove_em,
 		__entry->root_id	= inode->root->root_key.objectid;
 		__entry->start		= em->start;
 		__entry->len		= em->len;
-		__entry->block_start	= em->block_start;
 		__entry->flags		= em->flags;
 	),
 
-	TP_printk_btrfs(
-"ino=%llu root=%llu(%s) start=%llu len=%llu block_start=%llu(%s) flags=%s",
+	TP_printk_btrfs("ino=%llu root=%llu(%s) start=%llu len=%llu flags=%s",
 			__entry->ino, show_root_type(__entry->root_id),
 			__entry->start, __entry->len,
-			show_map_type(__entry->block_start),
 			show_map_flags(__entry->flags))
 );
 
Signature · 1 line
2.45.0
#24 lore raw

Re: [PATCH v2 07/11] btrfs: remove extent_map::block_start member

Quoted 13 lines · Qu Wenruo

The member extent_map::block_start can be calculated from extent_map::disk_bytenr + extent_map::offset for regular extents. And otherwise just extent_map::disk_bytenr.

And this is already validated by the validate_extent_map(). Now we can remove the member.

However there is a special case in btrfs_alloc_ordered_extent(), where we intentionally pass a pseudo ordered extent, exploiting the fact that for NOCOW/PREALLOC writes we do not rely on ordered extent members to update the file extent items.

Quoted 5 lines

And that's the only way to pass btrfs_extract_ordered_extent(), as it doesn't accept any ordered extent with an offset.

For now we will keep the old pseudo ordered extent members, and leave the cleanup of it for the future.

These last 3 paragraphs, the rant about NOCOW writes and ordered extents, seem unnecessary to me.

This is just how things work, and for me it makes total sense that an ordered extent for a NOCOW write does not represent the entirety of an existing file extent item.
NOCOW writes don't create new file extent items, so they can refer only to a section of an existing extent (or the whole extent).

I don't see how this rant is relevant to have here, as the patchset is about extent maps and not ordered extents.
I would leave it out, as it's a separate thing and doesn't affect anything here.

More comments inlined below.

Quoted 537 lines
---
 fs/btrfs/compression.c            |  3 +-
 fs/btrfs/defrag.c                 |  9 ++-
 fs/btrfs/extent_io.c              | 10 ++--
 fs/btrfs/extent_map.c             | 56 +++++------------
 fs/btrfs/extent_map.h             | 22 ++++---
 fs/btrfs/file-item.c              |  4 --
 fs/btrfs/file.c                   | 11 ++--
 fs/btrfs/inode.c                  | 84 +++++++++++++++-----------
 fs/btrfs/relocation.c             |  1 -
 fs/btrfs/tests/extent-map-tests.c | 48 ++++++---------
 fs/btrfs/tests/inode-tests.c      | 99 ++++++++++++++++---------------
 fs/btrfs/tree-log.c               | 17 +++---
 fs/btrfs/zoned.c                  |  4 +-
 include/trace/events/btrfs.h      | 11 +---
 14 files changed, 172 insertions(+), 207 deletions(-)
fs/btrfs/compression.c +2 -1
index 3af87911c83e..414adbba07b8 100644
@@ -507,7 +507,8 @@ static noinline int add_ra_bio_pages(struct inode *inode,
                 */
                if (!em || cur < em->start ||
                    (cur + fs_info->sectorsize > extent_map_end(em)) ||
-                   (em->block_start >> SECTOR_SHIFT) != orig_bio->bi_iter.bi_sector) {
+                   (extent_map_block_start(em) >> SECTOR_SHIFT) !=
+                   orig_bio->bi_iter.bi_sector) {
                        free_extent_map(em);
                        unlock_extent(tree, cur, page_end, NULL);
                        unlock_page(page);
fs/btrfs/defrag.c +4 -5
index 025e7f853a68..6fb94e897fc5 100644
@@ -707,7 +707,6 @@ static struct extent_map *defrag_get_extent(struct btrfs_inode *inode,
                 */
                if (key.offset > start) {
                        em->start = start;
-                       em->block_start = EXTENT_MAP_HOLE;
                        em->disk_bytenr = EXTENT_MAP_HOLE;
                        em->disk_num_bytes = 0;
                        em->ram_bytes = 0;
@@ -828,7 +827,7 @@ static bool defrag_check_next_extent(struct inode *inode, struct extent_map *em,
         */
        next = defrag_lookup_extent(inode, em->start + em->len, newer_than, locked);
        /* No more em or hole */
-       if (!next || next->block_start >= EXTENT_MAP_LAST_BYTE)
+       if (!next || next->disk_bytenr >= EXTENT_MAP_LAST_BYTE)
                goto out;
        if (next->flags & EXTENT_FLAG_PREALLOC)
                goto out;
@@ -995,12 +994,12 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
                 * This is for users who want to convert inline extents to
                 * regular ones through max_inline= mount option.
                 */
-               if (em->block_start == EXTENT_MAP_INLINE &&
+               if (em->disk_bytenr == EXTENT_MAP_INLINE &&
                    em->len <= inode->root->fs_info->max_inline)
                        goto next;
 
                /* Skip holes and preallocated extents. */
-               if (em->block_start == EXTENT_MAP_HOLE ||
+               if (em->disk_bytenr == EXTENT_MAP_HOLE ||
                    (em->flags & EXTENT_FLAG_PREALLOC))
                        goto next;
 
@@ -1065,7 +1064,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode,
                 * So if an inline extent passed all above checks, just add it
                 * for defrag, and be converted to regular extents.
                 */
-               if (em->block_start == EXTENT_MAP_INLINE)
+               if (em->disk_bytenr == EXTENT_MAP_INLINE)
                        goto add;
 
                next_mergeable = defrag_check_next_extent(&inode->vfs_inode, em,
fs/btrfs/extent_io.c +5 -5
index 597387e9f040..ef9928cc6467 100644
@@ -1083,10 +1083,10 @@ static int btrfs_do_readpage(struct page *page, struct extent_map **em_cached,
                iosize = min(extent_map_end(em) - cur, end - cur + 1);
                iosize = ALIGN(iosize, blocksize);
                if (compress_type != BTRFS_COMPRESS_NONE)
-                       disk_bytenr = em->block_start;
+                       disk_bytenr = em->disk_bytenr;
                else
-                       disk_bytenr = em->block_start + extent_offset;
-               block_start = em->block_start;
+                       disk_bytenr = extent_map_block_start(em) + extent_offset;
+               block_start = extent_map_block_start(em);
                if (em->flags & EXTENT_FLAG_PREALLOC)
                        block_start = EXTENT_MAP_HOLE;
 
@@ -1405,8 +1405,8 @@ static noinline_for_stack int __extent_writepage_io(struct btrfs_inode *inode,
                ASSERT(IS_ALIGNED(em->start, fs_info->sectorsize));
                ASSERT(IS_ALIGNED(em->len, fs_info->sectorsize));
 
-               block_start = em->block_start;
-               disk_bytenr = em->block_start + extent_offset;
+               block_start = extent_map_block_start(em);
+               disk_bytenr = extent_map_block_start(em) + extent_offset;
 
                ASSERT(!extent_map_is_compressed(em));
                ASSERT(block_start != EXTENT_MAP_HOLE);
fs/btrfs/extent_map.c +15 -41
index dcd191c2c4b3..9ae86125599a 100644
@@ -195,9 +195,10 @@ static inline u64 extent_map_block_len(const struct extent_map *em)
 
 static inline u64 extent_map_block_end(const struct extent_map *em)
 {
-       if (em->block_start + extent_map_block_len(em) < em->block_start)
+       if (extent_map_block_start(em) + extent_map_block_len(em) <
+           extent_map_block_start(em))
                return (u64)-1;
-       return em->block_start + extent_map_block_len(em);
+       return extent_map_block_start(em) + extent_map_block_len(em);
 }
 
 static bool can_merge_extent_map(const struct extent_map *em)
@@ -232,11 +233,11 @@ static bool mergeable_maps(const struct extent_map *prev, const struct extent_ma
        if (prev->flags != next->flags)
                return false;
 
-       if (next->block_start < EXTENT_MAP_LAST_BYTE - 1)
-               return next->block_start == extent_map_block_end(prev);
+       if (next->disk_bytenr < EXTENT_MAP_LAST_BYTE - 1)
+               return extent_map_block_start(next) == extent_map_block_end(prev);
 
        /* HOLES and INLINE extents. */
-       return next->block_start == prev->block_start;
+       return next->disk_bytenr == prev->disk_bytenr;
 }
 
 /*
@@ -295,10 +296,9 @@ static void dump_extent_map(const char *prefix, struct extent_map *em)
 {
        if (!IS_ENABLED(CONFIG_BTRFS_DEBUG))
                return;
-       pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu block_start=%llu flags=0x%x\n",
+       pr_crit("%s, start=%llu len=%llu disk_bytenr=%llu disk_num_bytes=%llu ram_bytes=%llu offset=%llu flags=0x%x\n",
                prefix, em->start, em->len, em->disk_bytenr, em->disk_num_bytes,
-               em->ram_bytes, em->offset, em->block_start,
-               em->flags);
+               em->ram_bytes, em->offset, em->flags);
        ASSERT(0);
 }
 
@@ -315,16 +315,6 @@ static void validate_extent_map(struct extent_map *em)
                if (em->offset + em->len > em->disk_num_bytes &&
                    !extent_map_is_compressed(em))
                        dump_extent_map("disk_num_bytes too small", em);
-
-               if (extent_map_is_compressed(em)) {
-                       if (em->block_start != em->disk_bytenr)
-                               dump_extent_map(
-                               "mismatch block_start/disk_bytenr/offset", em);
-               } else {
-                       if (em->block_start != em->disk_bytenr + em->offset)
-                               dump_extent_map(
-                               "mismatch block_start/disk_bytenr/offset", em);
-               }
        } else {
                if (em->offset)
                        dump_extent_map("non-zero offset for hole/inline", em);
@@ -358,7 +348,6 @@ static void try_merge_map(struct btrfs_inode *inode, struct extent_map *em)
                if (rb && can_merge_extent_map(merge) && mergeable_maps(merge, em)) {
                        em->start = merge->start;
                        em->len += merge->len;
-                       em->block_start = merge->block_start;
                        em->generation = max(em->generation, merge->generation);
 
                        if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
@@ -668,11 +657,9 @@ static noinline int merge_extent_mapping(struct btrfs_inode *inode,
        start_diff = start - em->start;
        em->start = start;
        em->len = end - start;
-       if (em->block_start < EXTENT_MAP_LAST_BYTE &&
-           !extent_map_is_compressed(em)) {
-               em->block_start += start_diff;
+       if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE &&
+           !extent_map_is_compressed(em))
                em->offset += start_diff;
-       }
        return add_extent_mapping(inode, em, 0);
 }
 
@@ -707,7 +694,7 @@ int btrfs_add_extent_mapping(struct btrfs_inode *inode,
         * Tree-checker should have rejected any inline extent with non-zero
         * file offset. Here just do a sanity check.
         */
-       if (em->block_start == EXTENT_MAP_INLINE)
+       if (em->disk_bytenr == EXTENT_MAP_INLINE)
                ASSERT(em->start == 0);
 
        ret = add_extent_mapping(inode, em, 0);
@@ -836,7 +823,6 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                u64 gen;
                unsigned long flags;
                bool modified;
-               bool compressed;
 
                if (em_end < end) {
                        next_em = next_extent_map(em);
@@ -870,7 +856,6 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        goto remove_em;
 
                gen = em->generation;
-               compressed = extent_map_is_compressed(em);
 
                if (em->start < start) {
                        if (!split) {
@@ -882,15 +867,12 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        split->start = em->start;
                        split->len = start - em->start;
 
-                       if (em->block_start < EXTENT_MAP_LAST_BYTE) {
-                               split->block_start = em->block_start;
-
+                       if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE) {
                                split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = em->disk_num_bytes;
                                split->offset = em->offset;
                                split->ram_bytes = em->ram_bytes;
                        } else {
-                               split->block_start = em->block_start;
                                split->disk_bytenr = em->disk_bytenr;
                                split->disk_num_bytes = 0;
                                split->offset = 0;
@@ -913,20 +895,14 @@ void btrfs_drop_extent_map_range(struct btrfs_inode *inode, u64 start, u64 end,
                        }
                        split->start = end;
                        split->len = em_end - end;
-                       split->block_start = em->block_start;
                        split->disk_bytenr = em->disk_bytenr;
                        split->flags = flags;
                        split->generation = gen;
 
-                       if (em->block_start < EXTENT_MAP_LAST_BYTE) {
+                       if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE) {
                                split->disk_num_bytes = em->disk_num_bytes;
                                split->offset = em->offset + end - em->start;
                                split->ram_bytes = em->ram_bytes;
-                               if (!compressed) {
-                                       const u64 diff = end - em->start;
-
-                                       split->block_start += diff;
-                               }
                        } else {
                                split->disk_num_bytes = 0;
                                split->offset = 0;
@@ -1073,7 +1049,7 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
 
        ASSERT(em->len == len);
        ASSERT(!extent_map_is_compressed(em));
-       ASSERT(em->block_start < EXTENT_MAP_LAST_BYTE);
+       ASSERT(em->disk_bytenr < EXTENT_MAP_LAST_BYTE);
        ASSERT(em->flags & EXTENT_FLAG_PINNED);
        ASSERT(!(em->flags & EXTENT_FLAG_LOGGING));
        ASSERT(!list_empty(&em->list));
@@ -1087,7 +1063,6 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        split_pre->disk_bytenr = new_logical;
        split_pre->disk_num_bytes = split_pre->len;
        split_pre->offset = 0;
-       split_pre->block_start = new_logical;
        split_pre->ram_bytes = split_pre->len;
        split_pre->flags = flags;
        split_pre->generation = em->generation;
@@ -1102,10 +1077,9 @@ int split_extent_map(struct btrfs_inode *inode, u64 start, u64 len, u64 pre,
        /* Insert the middle extent_map. */
        split_mid->start = em->start + pre;
        split_mid->len = em->len - pre;
-       split_mid->disk_bytenr = em->block_start + pre;
+       split_mid->disk_bytenr = extent_map_block_start(em) + pre;
        split_mid->disk_num_bytes = split_mid->len;
        split_mid->offset = 0;
-       split_mid->block_start = em->block_start + pre;
        split_mid->ram_bytes = split_mid->len;
        split_mid->flags = flags;
        split_mid->generation = em->generation;
fs/btrfs/extent_map.h +10 -12
index aee721eaa7f3..a0e7f49ec88f 100644
@@ -90,18 +90,6 @@ struct extent_map {
         */
        u64 ram_bytes;
 
-       /*
-        * The on-disk logical bytenr for the file extent.
-        *
-        * For compressed extents it matches btrfs_file_extent_item::disk_bytenr.
-        * For uncompressed extents it matches
-        * btrfs_file_extent_item::disk_bytenr + btrfs_file_extent_item::offset
-        *
-        * For holes it is EXTENT_MAP_HOLE and for inline extents it is
-        * EXTENT_MAP_INLINE.
-        */
-       u64 block_start;
-
        /*
         * Generation of the extent map, for merged em it's the highest
         * generation of all merged ems.
@@ -162,6 +150,16 @@ static inline int extent_map_in_tree(const struct extent_map *em)
        return !RB_EMPTY_NODE(&em->rb_node);
 }
 
+static inline u64 extent_map_block_start(const struct extent_map *em)
+{
+       if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE) {
+               if (extent_map_is_compressed(em))
+                       return em->disk_bytenr;
+               return em->disk_bytenr + em->offset;
+       }
+       return em->disk_bytenr;
+}
+
 static inline u64 extent_map_end(const struct extent_map *em)
 {
        if (em->start + em->len < em->start)
fs/btrfs/file-item.c +0 -4
index 95fb7c059a1a..b8e213a1a8ee 100644
@@ -1294,7 +1294,6 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                em->len = btrfs_file_extent_end(path) - extent_start;
                bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
                if (bytenr == 0) {
-                       em->block_start = EXTENT_MAP_HOLE;
                        em->disk_bytenr = EXTENT_MAP_HOLE;
                        em->disk_num_bytes = 0;
                        em->offset = 0;
@@ -1305,10 +1304,8 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                em->offset = btrfs_file_extent_offset(leaf, fi);
                if (compress_type != BTRFS_COMPRESS_NONE) {
                        extent_map_set_compression(em, compress_type);
-                       em->block_start = bytenr;
                } else {
                        bytenr += btrfs_file_extent_offset(leaf, fi);
-                       em->block_start = bytenr;
                        if (type == BTRFS_FILE_EXTENT_PREALLOC)
                                em->flags |= EXTENT_FLAG_PREALLOC;
                }
@@ -1316,7 +1313,6 @@ void btrfs_extent_item_to_extent_map(struct btrfs_inode *inode,
                /* Tree-checker has ensured this. */
                ASSERT(extent_start == 0);
 
-               em->block_start = EXTENT_MAP_INLINE;
                em->disk_bytenr = EXTENT_MAP_INLINE;
                em->start = 0;
                em->len = fs_info->sectorsize;
fs/btrfs/file.c +5 -6
index 05c7b5429b85..102b5c17ece1 100644
@@ -2335,7 +2335,6 @@ static int fill_holes(struct btrfs_trans_handle *trans,
                hole_em->len = end - offset;
                hole_em->ram_bytes = hole_em->len;
 
-               hole_em->block_start = EXTENT_MAP_HOLE;
                hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                hole_em->disk_num_bytes = 0;
                hole_em->generation = trans->transid;
@@ -2368,7 +2367,7 @@ static int find_first_non_hole(struct btrfs_inode *inode, u64 *start, u64 *len)
                return PTR_ERR(em);
 
        /* Hole or vacuum extent(only exists in no-hole mode) */
-       if (em->block_start == EXTENT_MAP_HOLE) {
+       if (em->disk_bytenr == EXTENT_MAP_HOLE) {
                ret = 1;
                *len = em->start + em->len > *start + *len ?
                       0 : *start + *len - em->start - em->len;
@@ -3025,7 +3024,7 @@ static int btrfs_zero_range_check_range_boundary(struct btrfs_inode *inode,
        if (IS_ERR(em))
                return PTR_ERR(em);
 
-       if (em->block_start == EXTENT_MAP_HOLE)
+       if (em->disk_bytenr == EXTENT_MAP_HOLE)
                ret = RANGE_BOUNDARY_HOLE;
        else if (em->flags & EXTENT_FLAG_PREALLOC)
                ret = RANGE_BOUNDARY_PREALLOC_EXTENT;
@@ -3089,7 +3088,7 @@ static int btrfs_zero_range(struct inode *inode,
                ASSERT(IS_ALIGNED(alloc_start, sectorsize));
                len = offset + len - alloc_start;
                offset = alloc_start;
-               alloc_hint = em->block_start + em->len;
+               alloc_hint = extent_map_block_start(em) + em->len;
        }
        free_extent_map(em);
 
@@ -3107,7 +3106,7 @@ static int btrfs_zero_range(struct inode *inode,
                                                           mode);
                        goto out;
                }
-               if (len < sectorsize && em->block_start != EXTENT_MAP_HOLE) {
+               if (len < sectorsize && em->disk_bytenr != EXTENT_MAP_HOLE) {
                        free_extent_map(em);
                        ret = btrfs_truncate_block(BTRFS_I(inode), offset, len,
                                                   0);
@@ -3320,7 +3319,7 @@ static long btrfs_fallocate(struct file *file, int mode,
                last_byte = min(extent_map_end(em), alloc_end);
                actual_end = min_t(u64, extent_map_end(em), offset + len);
                last_byte = ALIGN(last_byte, blocksize);
-               if (em->block_start == EXTENT_MAP_HOLE ||
+               if (em->disk_bytenr == EXTENT_MAP_HOLE ||
                    (cur_offset >= inode->i_size &&
                     !(em->flags & EXTENT_FLAG_PREALLOC))) {
                        const u64 range_len = last_byte - cur_offset;
fs/btrfs/inode.c +11 -16
index 7df295e0046d..8bc1f165193a 100644
@@ -138,7 +138,7 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
                                     u64 end, struct writeback_control *wbc,
                                     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-                                      u64 len, u64 block_start,
+                                      u64 len,
                                       u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
                                       struct btrfs_file_extent *file_extent,
@@ -1209,7 +1209,6 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
 
        em = create_io_em(inode, start,
                          async_extent->ram_size,       /* len */
-                         ins.objectid,                 /* block_start */
                          ins.offset,                   /* orig_block_len */
                          async_extent->ram_size,       /* ram_bytes */
                          async_extent->compress_type,
@@ -1287,15 +1286,15 @@ static u64 get_extent_allocation_hint(struct btrfs_inode *inode, u64 start,
                 * first block in this inode and use that as a hint.  If that
                 * block is also bogus then just don't worry about it.
                 */
-               if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
+               if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
                        free_extent_map(em);
                        em = search_extent_mapping(em_tree, 0, 0);
-                       if (em && em->block_start < EXTENT_MAP_LAST_BYTE)
-                               alloc_hint = em->block_start;
+                       if (em && em->disk_bytenr < EXTENT_MAP_LAST_BYTE)
+                               alloc_hint = extent_map_block_start(em);
                        if (em)
                                free_extent_map(em);
                } else {
-                       alloc_hint = em->block_start;
+                       alloc_hint = extent_map_block_start(em);
                        free_extent_map(em);
                }
        }
@@ -1451,7 +1450,6 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                            &cached);
 
                em = create_io_em(inode, start, ins.offset, /* len */
-                                 ins.objectid, /* block_start */
                                  ins.offset, /* orig_block_len */
                                  ram_size, /* ram_bytes */
                                  BTRFS_COMPRESS_NONE, /* compress_type */
@@ -2191,7 +2189,6 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                        struct extent_map *em;
 
                        em = create_io_em(inode, cur_offset, nocow_args.num_bytes,
-                                         nocow_args.disk_bytenr, /* block_start */
                                          nocow_args.disk_num_bytes, /* orig_block_len */
                                          ram_bytes, BTRFS_COMPRESS_NONE,
                                          &nocow_args.file_extent,
@@ -2706,7 +2703,7 @@ static int btrfs_find_new_delalloc_bytes(struct btrfs_inode *inode,
                if (IS_ERR(em))
                        return PTR_ERR(em);
 
-               if (em->block_start != EXTENT_MAP_HOLE)
+               if (extent_map_block_start(em) != EXTENT_MAP_HOLE)
                        goto next;
 
                em_len = em->len;
@@ -4993,7 +4990,6 @@ int btrfs_cont_expand(struct btrfs_inode *inode, loff_t oldsize, loff_t size)
                        hole_em->start = cur_offset;
                        hole_em->len = hole_size;
 
-                       hole_em->block_start = EXTENT_MAP_HOLE;
                        hole_em->disk_bytenr = EXTENT_MAP_HOLE;
                        hole_em->disk_num_bytes = 0;
                        hole_em->ram_bytes = hole_size;
@@ -6842,7 +6838,7 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
        if (em) {
                if (em->start > start || em->start + em->len <= start)
                        free_extent_map(em);
-               else if (em->block_start == EXTENT_MAP_INLINE && page)
+               else if (em->disk_bytenr == EXTENT_MAP_INLINE && page)
                        free_extent_map(em);
                else
                        goto out;
@@ -6945,7 +6941,7 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
                /* New extent overlaps with existing one */
                em->start = start;
                em->len = found_key.offset - start;
-               em->block_start = EXTENT_MAP_HOLE;
+               em->disk_bytenr = EXTENT_MAP_HOLE;
                goto insert;
        }
 
@@ -6969,7 +6965,7 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
                 *
                 * Other members are not utilized for inline extents.
                 */
-               ASSERT(em->block_start == EXTENT_MAP_INLINE);
+               ASSERT(em->disk_bytenr == EXTENT_MAP_INLINE);
                ASSERT(em->len == fs_info->sectorsize);
 
                ret = read_inline_extent(inode, path, page);
@@ -6980,7 +6976,7 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
 not_found:
        em->start = start;
        em->len = len;
-       em->block_start = EXTENT_MAP_HOLE;
+       em->disk_bytenr = EXTENT_MAP_HOLE;
 insert:
        ret = 0;
        btrfs_release_path(path);
@@ -7011,7 +7007,6 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                                                  struct btrfs_dio_data *dio_data,
                                                  const u64 start,
                                                  const u64 len,
-                                                 const u64 block_start,
                                                  const u64 orig_block_len,
                                                  const u64 ram_bytes,
                                                  const int type,
@@ -7021,15 +7016,38 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
        struct btrfs_ordered_extent *ordered;
 
        if (type != BTRFS_ORDERED_NOCOW) {
-               em = create_io_em(inode, start, len, block_start,
+               em = create_io_em(inode, start, len,

fs/btrfs/inode.c static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode, · line 7029

Quoted 10 lines
fs/btrfs/inode.c +4 -0
                                  orig_block_len, ram_bytes,
                                  BTRFS_COMPRESS_NONE, /* compress_type */
                                  file_extent, type);
                if (IS_ERR(em))
                        goto out;
        }
+
+       /*
+        * NOTE: I know the numbers are totally wrong for NOCOW/PREALLOC,
+        * but it doesn't cause problem at least for now.

They are not wrong, they are what they must be.

This is all about passing the right values to btrfs_alloc_ordered_extent(), which has nothing to do with extent maps.

fs/btrfs/inode.c static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode, · line 7036

Quoted 7 lines
fs/btrfs/inode.c +7 -0
+        *
+        * For regular writes, we would have file_extent->offset as 0,
+        * thus we really only need disk_bytenr, every other length
+        * (disk_num_bytes/ram_bytes) would match @len and fe->num_bytes.
+        * The current numbers are totally fine.
+        *
+        * For NOCOW, we don't really care about the numbers except @file_pos

I don't see any variable named @file_pos anywhere in this function.

fs/btrfs/inode.c static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode, · line 7037

Quoted 1 line
fs/btrfs/inode.c +1 -0
+        * and @num_bytes, as we won't insert a file extent item at all.

There's no @num_bytes either, there's a @len however.

fs/btrfs/inode.c static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode, · line 7039

Quoted 2 lines
fs/btrfs/inode.c +2 -0
+        *
+        * For PREALLOC, we do not use ordered extent's member, but

ordered extent's member -> ordered extent members

fs/btrfs/inode.c static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode, · line 7040

Quoted 1 line
fs/btrfs/inode.c +1 -0
+        * btrfs_mark_extent_written() would handle everything.

would handle -> handles

fs/btrfs/inode.c static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode, · line 7046

Quoted 6 lines
fs/btrfs/inode.c +6 -0
+        *
+        * So here we intentionally go with pseudo numbers for the NOCOW/PREALLOC
+        * OEs, or btrfs_extract_ordered_extent() would need a completely new
+        * routine to handle NOCOW/PREALLOC splits, meanwhile result nothing
+        * different.
+        */

I would just leave the entire comment out.

Quoted 724 lines
fs/btrfs/inode.c +17 -19
        ordered = btrfs_alloc_ordered_extent(inode, start, len, len,
-                                            block_start, len, 0,
+                                            file_extent->disk_bytenr +
+                                            file_extent->offset,
+                                            len, 0,
                                             (1 << type) |
                                             (1 << BTRFS_ORDERED_DIRECT),
                                             BTRFS_COMPRESS_NONE);
@@ -7081,7 +7099,7 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
        file_extent.offset = 0;
        file_extent.compression = BTRFS_COMPRESS_NONE;
        em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset,
-                                    ins.objectid, ins.offset,
+                                    ins.offset,
                                     ins.offset, BTRFS_ORDERED_REGULAR,
                                     &file_extent);
        btrfs_dec_block_group_reservations(fs_info, ins.objectid);
@@ -7321,7 +7339,7 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
 
 /* The callers of this must take lock_extent() */
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-                                      u64 len, u64 block_start,
+                                      u64 len,
                                       u64 disk_num_bytes,
                                       u64 ram_bytes, int compress_type,
                                       struct btrfs_file_extent *file_extent,
@@ -7373,7 +7391,6 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 
        em->start = start;
        em->len = len;
-       em->block_start = block_start;
        em->disk_bytenr = file_extent->disk_bytenr;
        em->disk_num_bytes = disk_num_bytes;
        em->ram_bytes = ram_bytes;
@@ -7424,13 +7441,13 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
         */
        if ((em->flags & EXTENT_FLAG_PREALLOC) ||
            ((BTRFS_I(inode)->flags & BTRFS_INODE_NODATACOW) &&
-            em->block_start != EXTENT_MAP_HOLE)) {
+            em->disk_bytenr != EXTENT_MAP_HOLE)) {
                if (em->flags & EXTENT_FLAG_PREALLOC)
                        type = BTRFS_ORDERED_PREALLOC;
                else
                        type = BTRFS_ORDERED_NOCOW;
                len = min(len, em->len - (start - em->start));
-               block_start = em->block_start + (start - em->start);
+               block_start = extent_map_block_start(em) + (start - em->start);
 
                if (can_nocow_extent(inode, start, &len,
                                     &orig_block_len, &ram_bytes,
@@ -7460,7 +7477,6 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                space_reserved = true;
 
                em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
-                                             block_start,
                                              orig_block_len,
                                              ram_bytes, type,
                                              &file_extent);
@@ -7663,7 +7679,7 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start,
         * the generic code.
         */
        if (extent_map_is_compressed(em) ||
-           em->block_start == EXTENT_MAP_INLINE) {
+           em->disk_bytenr == EXTENT_MAP_INLINE) {
                free_extent_map(em);
                /*
                 * If we are in a NOWAIT context, return -EAGAIN in order to
@@ -7757,12 +7773,12 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start,
         * We trim the extents (and move the addr) even though iomap code does
         * that, since we have locked only the parts we are performing I/O in.
         */
-       if ((em->block_start == EXTENT_MAP_HOLE) ||
+       if ((em->disk_bytenr == EXTENT_MAP_HOLE) ||
            ((em->flags & EXTENT_FLAG_PREALLOC) && !write)) {
                iomap->addr = IOMAP_NULL_ADDR;
                iomap->type = IOMAP_HOLE;
        } else {
-               iomap->addr = em->block_start + (start - em->start);
+               iomap->addr = extent_map_block_start(em) + (start - em->start);
                iomap->type = IOMAP_MAPPED;
        }
        iomap->offset = start;
@@ -9612,7 +9628,6 @@ static int __btrfs_prealloc_file_range(struct inode *inode, int mode,
 
                em->start = cur_offset;
                em->len = ins.offset;
-               em->block_start = ins.objectid;
                em->disk_bytenr = ins.objectid;
                em->offset = 0;
                em->disk_num_bytes = ins.offset;
@@ -10078,7 +10093,7 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter,
                goto out_unlock_extent;
        }
 
-       if (em->block_start == EXTENT_MAP_INLINE) {
+       if (em->disk_bytenr == EXTENT_MAP_INLINE) {
                u64 extent_start = em->start;
 
                /*
@@ -10099,14 +10114,14 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter,
         */
        encoded->len = min_t(u64, extent_map_end(em),
                             inode->vfs_inode.i_size) - iocb->ki_pos;
-       if (em->block_start == EXTENT_MAP_HOLE ||
+       if (em->disk_bytenr == EXTENT_MAP_HOLE ||
            (em->flags & EXTENT_FLAG_PREALLOC)) {
                disk_bytenr = EXTENT_MAP_HOLE;
                count = min_t(u64, count, encoded->len);
                encoded->len = count;
                encoded->unencoded_len = count;
        } else if (extent_map_is_compressed(em)) {
-               disk_bytenr = em->block_start;
+               disk_bytenr = em->disk_bytenr;
                /*
                 * Bail if the buffer isn't large enough to return the whole
                 * compressed extent.
@@ -10125,7 +10140,7 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter,
                        goto out_em;
                encoded->compression = ret;
        } else {
-               disk_bytenr = em->block_start + (start - em->start);
+               disk_bytenr = extent_map_block_start(em) + (start - em->start);
                if (encoded->len > count)
                        encoded->len = count;
                /*
@@ -10363,7 +10378,6 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
        file_extent.offset = encoded->unencoded_offset;
        file_extent.compression = compression;
        em = create_io_em(inode, start, num_bytes,
-                         ins.objectid,
                          ins.offset, ram_bytes, compression,
                          &file_extent, BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
@@ -10667,12 +10681,12 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file,
                        goto out;
                }
 
-               if (em->block_start == EXTENT_MAP_HOLE) {
+               if (em->disk_bytenr == EXTENT_MAP_HOLE) {
                        btrfs_warn(fs_info, "swapfile must not have holes");
                        ret = -EINVAL;
                        goto out;
                }
-               if (em->block_start == EXTENT_MAP_INLINE) {
+               if (em->disk_bytenr == EXTENT_MAP_INLINE) {
                        /*
                         * It's unlikely we'll ever actually find ourselves
                         * here, as a file small enough to fit inline won't be
@@ -10690,7 +10704,7 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file,
                        goto out;
                }
 
-               logical_block_start = em->block_start + (start - em->start);
+               logical_block_start = extent_map_block_start(em) + (start - em->start);
                len = min(len, em->len - (start - em->start));
                free_extent_map(em);
                em = NULL;
fs/btrfs/relocation.c +0 -1
index a66d9b921f84..0d462c2a9c89 100644
@@ -2912,7 +2912,6 @@ static noinline_for_stack int setup_relocation_extent_mapping(struct inode *inod
 
        em->start = start;
        em->len = end + 1 - start;
-       em->block_start = block_start;
        em->disk_bytenr = block_start;
        em->disk_num_bytes = em->len;
        em->ram_bytes = em->len;
fs/btrfs/tests/extent-map-tests.c +17 -31
index ffdaa6a682af..a145214be14b 100644
@@ -28,8 +28,8 @@ static int free_extent_map_tree(struct btrfs_inode *inode)
                if (refcount_read(&em->refs) != 1) {
                        ret = -EINVAL;
                        test_err(
-"em leak: em (start %llu len %llu block_start %llu disk_num_bytes %llu offset %llu) refs %d",
-                                em->start, em->len, em->block_start,
+"em leak: em (start %llu len %llu disk_bytenr %llu disk_num_bytes %llu offset %llu) refs %d",
+                                em->start, em->len, em->disk_bytenr,
                                 em->disk_num_bytes, em->offset,
                                 refcount_read(&em->refs));
 
@@ -77,7 +77,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        /* Add [0, 16K) */
        em->start = 0;
        em->len = SZ_16K;
-       em->block_start = 0;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_16K;
        em->ram_bytes = SZ_16K;
@@ -100,7 +99,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 
        em->start = SZ_16K;
        em->len = SZ_4K;
-       em->block_start = SZ_32K; /* avoid merging */
        em->disk_bytenr = SZ_32K; /* avoid merging */
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = SZ_4K;
@@ -123,7 +121,6 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        /* Add [0, 8K), should return [0, 16K) instead. */
        em->start = start;
        em->len = len;
-       em->block_start = start;
        em->disk_bytenr = start;
        em->disk_num_bytes = len;
        em->ram_bytes = len;
@@ -141,11 +138,11 @@ static int test_case_1(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
                goto out;
        }
        if (em->start != 0 || extent_map_end(em) != SZ_16K ||
-           em->block_start != 0 || em->disk_num_bytes != SZ_16K) {
+           em->disk_bytenr != 0 || em->disk_num_bytes != SZ_16K) {
                test_err(
-"case1 [%llu %llu]: ret %d return a wrong em (start %llu len %llu block_start %llu disk_num_bytes %llu",
+"case1 [%llu %llu]: ret %d return a wrong em (start %llu len %llu disk_bytenr %llu disk_num_bytes %llu",
                         start, start + len, ret, em->start, em->len,
-                        em->block_start, em->disk_num_bytes);
+                        em->disk_bytenr, em->disk_num_bytes);
                ret = -EINVAL;
        }
        free_extent_map(em);
@@ -179,7 +176,6 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        /* Add [0, 1K) */
        em->start = 0;
        em->len = SZ_1K;
-       em->block_start = EXTENT_MAP_INLINE;
        em->disk_bytenr = EXTENT_MAP_INLINE;
        em->disk_num_bytes = 0;
        em->ram_bytes = SZ_1K;
@@ -202,7 +198,6 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 
        em->start = SZ_4K;
        em->len = SZ_4K;
-       em->block_start = SZ_4K;
        em->disk_bytenr = SZ_4K;
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = SZ_4K;
@@ -225,7 +220,6 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        /* Add [0, 1K) */
        em->start = 0;
        em->len = SZ_1K;
-       em->block_start = EXTENT_MAP_INLINE;
        em->disk_bytenr = EXTENT_MAP_INLINE;
        em->disk_num_bytes = 0;
        em->ram_bytes = SZ_1K;
@@ -242,10 +236,10 @@ static int test_case_2(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
                goto out;
        }
        if (em->start != 0 || extent_map_end(em) != SZ_1K ||
-           em->block_start != EXTENT_MAP_INLINE) {
+           em->disk_bytenr != EXTENT_MAP_INLINE) {
                test_err(
-"case2 [0 1K]: ret %d return a wrong em (start %llu len %llu block_start %llu",
-                        ret, em->start, em->len, em->block_start);
+"case2 [0 1K]: ret %d return a wrong em (start %llu len %llu disk_bytenr %llu",
+                        ret, em->start, em->len, em->disk_bytenr);
                ret = -EINVAL;
        }
        free_extent_map(em);
@@ -275,7 +269,6 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
        /* Add [4K, 8K) */
        em->start = SZ_4K;
        em->len = SZ_4K;
-       em->block_start = SZ_4K;
        em->disk_bytenr = SZ_4K;
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = SZ_4K;
@@ -298,7 +291,6 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
        /* Add [0, 16K) */
        em->start = 0;
        em->len = SZ_16K;
-       em->block_start = 0;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_16K;
        em->ram_bytes = SZ_16K;
@@ -321,11 +313,11 @@ static int __test_case_3(struct btrfs_fs_info *fs_info,
         * em->start.
         */
        if (start < em->start || start + len > extent_map_end(em) ||
-           em->start != em->block_start) {
+           em->start != extent_map_block_start(em)) {
                test_err(
-"case3 [%llu %llu): ret %d em (start %llu len %llu block_start %llu block_len %llu)",
+"case3 [%llu %llu): ret %d em (start %llu len %llu disk_bytenr %llu block_len %llu)",
                         start, start + len, ret, em->start, em->len,
-                        em->block_start, em->disk_num_bytes);
+                        em->disk_bytenr, em->disk_num_bytes);
                ret = -EINVAL;
        }
        free_extent_map(em);
@@ -386,7 +378,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        /* Add [0K, 8K) */
        em->start = 0;
        em->len = SZ_8K;
-       em->block_start = 0;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_8K;
        em->ram_bytes = SZ_8K;
@@ -409,7 +400,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        /* Add [8K, 32K) */
        em->start = SZ_8K;
        em->len = 24 * SZ_1K;
-       em->block_start = SZ_16K; /* avoid merging */
        em->disk_bytenr = SZ_16K; /* avoid merging */
        em->disk_num_bytes = 24 * SZ_1K;
        em->ram_bytes = 24 * SZ_1K;
@@ -431,7 +421,6 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        /* Add [0K, 32K) */
        em->start = 0;
        em->len = SZ_32K;
-       em->block_start = 0;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_32K;
        em->ram_bytes = SZ_32K;
@@ -451,9 +440,9 @@ static int __test_case_4(struct btrfs_fs_info *fs_info,
        }
        if (start < em->start || start + len > extent_map_end(em)) {
                test_err(
-"case4 [%llu %llu): ret %d, added wrong em (start %llu len %llu block_start %llu disk_num_bytes %llu)",
-                        start, start + len, ret, em->start, em->len, em->block_start,
-                        em->disk_num_bytes);
+"case4 [%llu %llu): ret %d, added wrong em (start %llu len %llu disk_bytenr %llu disk_num_bytes %llu)",
+                        start, start + len, ret, em->start, em->len,
+                        em->disk_bytenr, em->disk_num_bytes);
                ret = -EINVAL;
        }
        free_extent_map(em);
@@ -517,7 +506,6 @@ static int add_compressed_extent(struct btrfs_inode *inode,
 
        em->start = start;
        em->len = len;
-       em->block_start = block_start;
        em->disk_bytenr = block_start;
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = len;
@@ -740,7 +728,6 @@ static int test_case_6(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
 
        em->start = SZ_4K;
        em->len = SZ_4K;
-       em->block_start = SZ_16K;
        em->disk_bytenr = SZ_16K;
        em->disk_num_bytes = SZ_16K;
        em->ram_bytes = SZ_16K;
@@ -795,7 +782,6 @@ static int test_case_7(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        /* [0, 16K), pinned */
        em->start = 0;
        em->len = SZ_16K;
-       em->block_start = 0;
        em->disk_bytenr = 0;
        em->disk_num_bytes = SZ_4K;
        em->ram_bytes = SZ_16K;
@@ -819,7 +805,6 @@ static int test_case_7(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
        /* [32K, 48K), not pinned */
        em->start = SZ_32K;
        em->len = SZ_16K;
-       em->block_start = SZ_32K;
        em->disk_bytenr = SZ_32K;
        em->disk_num_bytes = SZ_16K;
        em->ram_bytes = SZ_16K;
@@ -885,8 +870,9 @@ static int test_case_7(struct btrfs_fs_info *fs_info, struct btrfs_inode *inode)
                goto out;
        }
 
-       if (em->block_start != SZ_32K + SZ_4K) {
-               test_err("em->block_start is %llu, expected 36K", em->block_start);
+       if (extent_map_block_start(em) != SZ_32K + SZ_4K) {
+               test_err("em->block_start is %llu, expected 36K",
+                               extent_map_block_start(em));
                goto out;
        }
 
fs/btrfs/tests/inode-tests.c +50 -49
index 1b8c39edfc18..d6fd1978934a 100644
@@ -264,8 +264,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start != EXTENT_MAP_HOLE) {
-               test_err("expected a hole, got %llu", em->block_start);
+       if (em->disk_bytenr != EXTENT_MAP_HOLE) {
+               test_err("expected a hole, got %llu", em->disk_bytenr);
                goto out;
        }
        free_extent_map(em);
@@ -283,8 +283,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start != EXTENT_MAP_INLINE) {
-               test_err("expected an inline, got %llu", em->block_start);
+       if (em->disk_bytenr != EXTENT_MAP_INLINE) {
+               test_err("expected an inline, got %llu", em->disk_bytenr);
                goto out;
        }
 
@@ -321,8 +321,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start != EXTENT_MAP_HOLE) {
-               test_err("expected a hole, got %llu", em->block_start);
+       if (em->disk_bytenr != EXTENT_MAP_HOLE) {
+               test_err("expected a hole, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != 4) {
@@ -344,8 +344,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize - 1) {
@@ -371,8 +371,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize) {
@@ -389,7 +389,7 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
-       disk_bytenr = em->block_start;
+       disk_bytenr = extent_map_block_start(em);
        orig_start = em->start;
        offset = em->start + em->len;
        free_extent_map(em);
@@ -399,8 +399,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start != EXTENT_MAP_HOLE) {
-               test_err("expected a hole, got %llu", em->block_start);
+       if (em->disk_bytenr != EXTENT_MAP_HOLE) {
+               test_err("expected a hole, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize) {
@@ -421,8 +421,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != 2 * sectorsize) {
@@ -441,9 +441,9 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                goto out;
        }
        disk_bytenr += (em->start - orig_start);
-       if (em->block_start != disk_bytenr) {
+       if (extent_map_block_start(em) != disk_bytenr) {
                test_err("wrong block start, want %llu, have %llu",
-                        disk_bytenr, em->block_start);
+                        disk_bytenr, extent_map_block_start(em));
                goto out;
        }
        offset = em->start + em->len;
@@ -455,8 +455,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize) {
@@ -483,8 +483,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize) {
@@ -502,7 +502,7 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("wrong offset, want 0, have %llu", em->offset);
                goto out;
        }
-       disk_bytenr = em->block_start;
+       disk_bytenr = extent_map_block_start(em);
        orig_start = em->start;
        offset = em->start + em->len;
        free_extent_map(em);
@@ -512,8 +512,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_HOLE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_HOLE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize) {
@@ -531,9 +531,9 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         em->start - orig_start, em->offset);
                goto out;
        }
-       if (em->block_start != disk_bytenr + em->offset) {
+       if (extent_map_block_start(em) != disk_bytenr + em->offset) {
                test_err("unexpected block start, wanted %llu, have %llu",
-                        disk_bytenr + em->offset, em->block_start);
+                        disk_bytenr + em->offset, extent_map_block_start(em));
                goto out;
        }
        offset = em->start + em->len;
@@ -544,8 +544,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != 2 * sectorsize) {
@@ -564,9 +564,9 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         em->start - orig_start, em->offset);
                goto out;
        }
-       if (em->block_start != disk_bytenr + em->offset) {
+       if (extent_map_block_start(em) != disk_bytenr + em->offset) {
                test_err("unexpected block start, wanted %llu, have %llu",
-                        disk_bytenr + em->offset, em->block_start);
+                        disk_bytenr + em->offset, extent_map_block_start(em));
                goto out;
        }
        offset = em->start + em->len;
@@ -578,8 +578,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != 2 * sectorsize) {
@@ -611,8 +611,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize) {
@@ -635,7 +635,7 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                         BTRFS_COMPRESS_ZLIB, extent_map_compression(em));
                goto out;
        }
-       disk_bytenr = em->block_start;
+       disk_bytenr = extent_map_block_start(em);
        orig_start = em->start;
        offset = em->start + em->len;
        free_extent_map(em);
@@ -645,8 +645,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize) {
@@ -671,9 +671,9 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start != disk_bytenr) {
+       if (extent_map_block_start(em) != disk_bytenr) {
                test_err("block start does not match, want %llu got %llu",
-                        disk_bytenr, em->block_start);
+                        disk_bytenr, extent_map_block_start(em));
                goto out;
        }
        if (em->start != offset || em->len != 2 * sectorsize) {
@@ -706,8 +706,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize) {
@@ -732,8 +732,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start != EXTENT_MAP_HOLE) {
-               test_err("expected a hole extent, got %llu", em->block_start);
+       if (em->disk_bytenr != EXTENT_MAP_HOLE) {
+               test_err("expected a hole extent, got %llu", em->disk_bytenr);
                goto out;
        }
        /*
@@ -764,8 +764,8 @@ static noinline int test_btrfs_get_extent(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start >= EXTENT_MAP_LAST_BYTE) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (em->disk_bytenr >= EXTENT_MAP_LAST_BYTE) {
+               test_err("expected a real extent, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != offset || em->len != sectorsize) {
@@ -843,8 +843,8 @@ static int test_hole_first(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start != EXTENT_MAP_HOLE) {
-               test_err("expected a hole, got %llu", em->block_start);
+       if (em->disk_bytenr != EXTENT_MAP_HOLE) {
+               test_err("expected a hole, got %llu", em->disk_bytenr);
                goto out;
        }
        if (em->start != 0 || em->len != sectorsize) {
@@ -865,8 +865,9 @@ static int test_hole_first(u32 sectorsize, u32 nodesize)
                test_err("got an error when we shouldn't have");
                goto out;
        }
-       if (em->block_start != sectorsize) {
-               test_err("expected a real extent, got %llu", em->block_start);
+       if (extent_map_block_start(em) != sectorsize) {
+               test_err("expected a real extent, got %llu",
+                        extent_map_block_start(em));
                goto out;
        }
        if (em->start != sectorsize || em->len != sectorsize) {
fs/btrfs/tree-log.c +10 -7
index 13f35180e3a0..c4b67bf993d6 100644
@@ -4575,6 +4575,7 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
 {
        struct btrfs_ordered_extent *ordered;
        struct btrfs_root *csum_root;
+       u64 block_start;
        u64 csum_offset;
        u64 csum_len;
        u64 mod_start = em->start;
@@ -4584,7 +4585,7 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
 
        if (inode->flags & BTRFS_INODE_NODATASUM ||
            (em->flags & EXTENT_FLAG_PREALLOC) ||
-           em->block_start == EXTENT_MAP_HOLE)
+           em->disk_bytenr == EXTENT_MAP_HOLE)
                return 0;
 
        list_for_each_entry(ordered, &ctx->ordered_extents, log_list) {
@@ -4655,9 +4656,10 @@ static int log_extent_csums(struct btrfs_trans_handle *trans,
        }
 
        /* block start is already adjusted for the file extent offset. */
-       csum_root = btrfs_csum_root(trans->fs_info, em->block_start);
-       ret = btrfs_lookup_csums_list(csum_root, em->block_start + csum_offset,
-                                     em->block_start + csum_offset +
+       block_start = extent_map_block_start(em);
+       csum_root = btrfs_csum_root(trans->fs_info, block_start);
+       ret = btrfs_lookup_csums_list(csum_root, block_start + csum_offset,
+                                     block_start + csum_offset +
                                      csum_len - 1, &ordered_sums, false);
        if (ret < 0)
                return ret;
@@ -4689,6 +4691,7 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
        struct btrfs_key key;
        enum btrfs_compression_type compress_type;
        u64 extent_offset = em->offset;
+       u64 block_start = extent_map_block_start(em);
        u64 block_len;
        int ret;
 
@@ -4701,10 +4704,10 @@ static int log_one_extent(struct btrfs_trans_handle *trans,
        block_len = em->disk_num_bytes;
        compress_type = extent_map_compression(em);
        if (compress_type != BTRFS_COMPRESS_NONE) {
-               btrfs_set_stack_file_extent_disk_bytenr(&fi, em->block_start);
+               btrfs_set_stack_file_extent_disk_bytenr(&fi, block_start);
                btrfs_set_stack_file_extent_disk_num_bytes(&fi, block_len);
-       } else if (em->block_start < EXTENT_MAP_LAST_BYTE) {
-               btrfs_set_stack_file_extent_disk_bytenr(&fi, em->block_start -
+       } else if (em->disk_bytenr < EXTENT_MAP_LAST_BYTE) {
+               btrfs_set_stack_file_extent_disk_bytenr(&fi, block_start -
                                                        extent_offset);
                btrfs_set_stack_file_extent_disk_num_bytes(&fi, block_len);
        }
fs/btrfs/zoned.c +0 -0
index 4b52a8916dbb..6d0f7fc9aaf2 100644

fs/btrfs/zoned.c static void btrfs_rewrite_logical_zoned(struct btrfs_ordered_extent *ordered, · line 1781

Quoted 5 lines
fs/btrfs/zoned.c +1 -1
@@ -1778,7 +1778,9 @@ static void btrfs_rewrite_logical_zoned(struct btrfs_ordered_extent *ordered,
        write_lock(&em_tree->lock);
        em = search_extent_mapping(em_tree, ordered->file_offset,
                                   ordered->num_bytes);
-       em->block_start = logical;
+       /* The em should be a new COW extent, thus it should not has offset. */

not has offset -> not have an offset

Otherwise it seems fine, but I still want to go over the rest of the patchset. I'm going slowly over it, and after commenting on each inidividual patch, I'll comment on the cover letter.

Thanks.

Quoted 66 lines
fs/btrfs/zoned.c +2 -0
+       ASSERT(em->offset == 0);
+       em->disk_bytenr = logical;
        free_extent_map(em);
        write_unlock(&em_tree->lock);
 }
include/trace/events/btrfs.h +2 -10
index 3743719d13f2..89b2b66e5bc0 100644
@@ -291,7 +291,6 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
                __field(        u64,  ino               )
                __field(        u64,  start             )
                __field(        u64,  len               )
-               __field(        u64,  block_start       )
                __field(        u32,  flags             )
                __field(        int,  refs              )
        ),
@@ -301,18 +300,16 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
                __entry->ino            = btrfs_ino(inode);
                __entry->start          = map->start;
                __entry->len            = map->len;
-               __entry->block_start    = map->block_start;
                __entry->flags          = map->flags;
                __entry->refs           = refcount_read(&map->refs);
        ),
 
        TP_printk_btrfs("root=%llu(%s) ino=%llu start=%llu len=%llu "
-                 "block_start=%llu(%s) flags=%s refs=%u",
+                 "flags=%s refs=%u",
                  show_root_type(__entry->root_objectid),
                  __entry->ino,
                  __entry->start,
                  __entry->len,
-                 show_map_type(__entry->block_start),
                  show_map_flags(__entry->flags),
                  __entry->refs)
 );
@@ -2608,7 +2605,6 @@ TRACE_EVENT(btrfs_extent_map_shrinker_remove_em,
                __field(        u64,    root_id         )
                __field(        u64,    start           )
                __field(        u64,    len             )
-               __field(        u64,    block_start     )
                __field(        u32,    flags           )
        ),
 
@@ -2617,15 +2613,12 @@ TRACE_EVENT(btrfs_extent_map_shrinker_remove_em,
                __entry->root_id        = inode->root->root_key.objectid;
                __entry->start          = em->start;
                __entry->len            = em->len;
-               __entry->block_start    = em->block_start;
                __entry->flags          = em->flags;
        ),
 
-       TP_printk_btrfs(
-"ino=%llu root=%llu(%s) start=%llu len=%llu block_start=%llu(%s) flags=%s",
+       TP_printk_btrfs("ino=%llu root=%llu(%s) start=%llu len=%llu flags=%s",
                        __entry->ino, show_root_type(__entry->root_id),
                        __entry->start, __entry->len,
-                       show_map_type(__entry->block_start),
                        show_map_flags(__entry->flags))
 );
 
--

2.45.0

#25 lore raw

Re: [PATCH v2 07/11] btrfs: remove extent_map::block_start member

Quoted 20 lines · Filipe Manana
Quoted 13 lines · Qu Wenruo

The member extent_map::block_start can be calculated from extent_map::disk_bytenr + extent_map::offset for regular extents. And otherwise just extent_map::disk_bytenr.

And this is already validated by the validate_extent_map(). Now we can remove the member.

However there is a special case in btrfs_alloc_ordered_extent(), where we intentionally pass a pseudo ordered extent, exploiting the fact that for NOCOW/PREALLOC writes we do not rely on ordered extent members to update the file extent items.

Quoted 5 lines

And that's the only way to pass btrfs_extract_ordered_extent(), as it doesn't accept any ordered extent with an offset.

For now we will keep the old pseudo ordered extent members, and leave the cleanup of it for the future.

Quoted 8 lines

These last 3 paragraphs, the rant about NOCOW writes and ordered extents, seem unnecessary to me.

This is just how things work, and for me it makes total sense that an ordered extent for a NOCOW write does not represent the entirety of an existing file extent item.
NOCOW writes don't create new file extent items, so they can refer only to a section of an existing extent (or the whole extent).

To me, NOCOW/PREALLOC is really the exception, not following all the existing definition for all the OE/extent map definitions, even it's only for transient NOCOW/PREALLOC OE.

And I really hope to remove the exception in the near future, even it may mean some more complex OE spliting etc.

Quoted 4 lines

I don't see how this rant is relevant to have here, as the patchset is about extent maps and not ordered extents.
I would leave it out, as it's a separate thing and doesn't affect anything here.

Sure, I can leave it out for now.

Quoted 4 lines

More comments inlined below.

Quoted 1 line

[...]

Quoted 15 lines
Quoted 4 lines
         if (type != BTRFS_ORDERED_NOCOW) {
fs/btrfs/inode.c +1 -1
-               em = create_io_em(inode, start, len, block_start,
+               em = create_io_em(inode, start, len,

fs/btrfs/inode.c static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode, · line 7029

Quoted 10 lines
fs/btrfs/inode.c +4 -0
                                   orig_block_len, ram_bytes,
                                   BTRFS_COMPRESS_NONE, /* compress_type */
                                   file_extent, type);
                 if (IS_ERR(em))
                         goto out;
         }
+
+       /*
+        * NOTE: I know the numbers are totally wrong for NOCOW/PREALLOC,
+        * but it doesn't cause problem at least for now.
Quoted 5 lines

They are not wrong, they are what they must be.

This is all about passing the right values to btrfs_alloc_ordered_extent(), which has nothing to do with extent maps.

I think it's pretty obvious that, only for NOCOW/PREALLOC that OE members differs from their corresponding extent map.

Sure, it is not causing anything wrong (in fact, anything other than the current behavior is going to cause a lot of problems).

But just for the sake of consistency, even it doesn't affect any on-disk change, I still want to modify the call sites so that we can directly pass the btrfs_file_extent item to OE allocation.
Even if it means other modification mostly inside OE splitting part.

For now I can remove/modify the comments, but I do not think this is consistent.

Quoted 35 lines

fs/btrfs/inode.c static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode, · line 7036

Quoted 8 lines
fs/btrfs/inode.c +7 -0
+        *
+        * For regular writes, we would have file_extent->offset as 0,
+        * thus we really only need disk_bytenr, every other length
+        * (disk_num_bytes/ram_bytes) would match @len and fe->num_bytes.
+        * The current numbers are totally fine.
+        *
+        * For NOCOW, we don't really care about the numbers except @file_pos

I don't see any variable named @file_pos anywhere in this function.

Quoted 1 line

+ * and @num_bytes, as we won't insert a file extent item at all.

There's no @num_bytes either, there's a @len however.

Review

Quoted 2 lines
(quoted diff) +2 -0
+        *
+        * For PREALLOC, we do not use ordered extent's member, but

ordered extent's member -> ordered extent members

Quoted 1 line

+ * btrfs_mark_extent_written() would handle everything.

would handle -> handles

fs/btrfs/inode.c static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode, · line 7046

Quoted 6 lines
fs/btrfs/inode.c +6 -0
+        *
+        * So here we intentionally go with pseudo numbers for the NOCOW/PREALLOC
+        * OEs, or btrfs_extract_ordered_extent() would need a completely new
+        * routine to handle NOCOW/PREALLOC splits, meanwhile result nothing
+        * different.
+        */

I would just leave the entire comment out.

Quoted 2 lines

ordered = btrfs_alloc_ordered_extent(inode, start, len, len,
- block_start, len, 0,

[...]

Quoted 9 lines

fs/btrfs/zoned.c static void btrfs_rewrite_logical_zoned(struct btrfs_ordered_extent *ordered, · line 1781

Quoted 6 lines
fs/btrfs/zoned.c +1 -1
@@ -1778,7 +1778,9 @@ static void btrfs_rewrite_logical_zoned(struct btrfs_ordered_extent *ordered,
         write_lock(&em_tree->lock);
         em = search_extent_mapping(em_tree, ordered->file_offset,
                                    ordered->num_bytes);
-       em->block_start = logical;
+       /* The em should be a new COW extent, thus it should not has offset. */

not has offset -> not have an offset

Quoted 3 lines

Otherwise it seems fine, but I still want to go over the rest of the patchset. I'm going slowly over it, and after commenting on each inidividual patch, I'll comment on the cover letter.

Really appreciate the review so far.
And considering the patchset is not urgent, just take your time so that I can also address the comments at the same time.

Thanks,
Qu

Quoted 70 lines

Thanks.

Quoted 66 lines
fs/btrfs/zoned.c +2 -0
+       ASSERT(em->offset == 0);
+       em->disk_bytenr = logical;
         free_extent_map(em);
         write_unlock(&em_tree->lock);
  }
include/trace/events/btrfs.h +2 -10
index 3743719d13f2..89b2b66e5bc0 100644
@@ -291,7 +291,6 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
                 __field(        u64,  ino               )
                 __field(        u64,  start             )
                 __field(        u64,  len               )
-               __field(        u64,  block_start       )
                 __field(        u32,  flags             )
                 __field(        int,  refs              )
         ),
@@ -301,18 +300,16 @@ TRACE_EVENT_CONDITION(btrfs_get_extent,
                 __entry->ino            = btrfs_ino(inode);
                 __entry->start          = map->start;
                 __entry->len            = map->len;
-               __entry->block_start    = map->block_start;
                 __entry->flags          = map->flags;
                 __entry->refs           = refcount_read(&map->refs);
         ),
 
         TP_printk_btrfs("root=%llu(%s) ino=%llu start=%llu len=%llu "
-                 "block_start=%llu(%s) flags=%s refs=%u",
+                 "flags=%s refs=%u",
                   show_root_type(__entry->root_objectid),
                   __entry->ino,
                   __entry->start,
                   __entry->len,
-                 show_map_type(__entry->block_start),
                   show_map_flags(__entry->flags),
                   __entry->refs)
  );
@@ -2608,7 +2605,6 @@ TRACE_EVENT(btrfs_extent_map_shrinker_remove_em,
                 __field(        u64,    root_id         )
                 __field(        u64,    start           )
                 __field(        u64,    len             )
-               __field(        u64,    block_start     )
                 __field(        u32,    flags           )
         ),
 
@@ -2617,15 +2613,12 @@ TRACE_EVENT(btrfs_extent_map_shrinker_remove_em,
                 __entry->root_id        = inode->root->root_key.objectid;
                 __entry->start          = em->start;
                 __entry->len            = em->len;
-               __entry->block_start    = em->block_start;
                 __entry->flags          = em->flags;
         ),
 
-       TP_printk_btrfs(
-"ino=%llu root=%llu(%s) start=%llu len=%llu block_start=%llu(%s) flags=%s",
+       TP_printk_btrfs("ino=%llu root=%llu(%s) start=%llu len=%llu flags=%s",
                         __entry->ino, show_root_type(__entry->root_id),
                         __entry->start, __entry->len,
-                       show_map_type(__entry->block_start),
                         show_map_flags(__entry->flags))
  );
 
--

2.45.0

author
#26 lore raw

PATCH v2 8/11 btrfs: cleanup duplicated parameters related to can_nocow_file_extent_args

In reply to Qu Wenruo

The following functions and structures can be simplified using the btrfs_file_extent structure:

- can_nocow_extent()
No need to return ram_bytes/orig_block_len through the parameter list, the @file_extent parameter contains all needed info.

- can_nocow_file_extent_args
The following members are no longer needed:

* disk_bytenr
This one is confusing as it's not really the btrfs_file_extent_item::disk_bytenr, but where the IO would be, thus it's file_extent::disk_bytenr + file_extent::offset now.

* num_bytes
Now file_extent::num_bytes.

* extent_offset
Now file_extent::offset.

* disk_num_bytes
Now file_extent::disk_num_bytes.

---
 fs/btrfs/btrfs_inode.h |  3 +-
 fs/btrfs/file.c        |  2 +-
 fs/btrfs/inode.c       | 89 +++++++++++++++++++-----------------------
 3 files changed, 42 insertions(+), 52 deletions(-)
fs/btrfs/btrfs_inode.h +1 -2
index f30afce4f6ca..bea343615ad1 100644
@@ -461,8 +461,7 @@ struct btrfs_file_extent {
 };
 
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
-			      u64 *orig_block_len,
-			      u64 *ram_bytes, struct btrfs_file_extent *file_extent,
+			      struct btrfs_file_extent *file_extent,
 			      bool nowait, bool strict);
 
 void btrfs_del_delalloc_inode(struct btrfs_inode *inode);
fs/btrfs/file.c +1 -1
index 102b5c17ece1..6aaeb9ee048d 100644
@@ -1104,7 +1104,7 @@ int btrfs_check_nocow_lock(struct btrfs_inode *inode, loff_t pos,
 						   &cached_state);
 	}
 	ret = can_nocow_extent(&inode->vfs_inode, lockstart, &num_bytes,
-			NULL, NULL, NULL, nowait, false);
+			       NULL, nowait, false);
 	if (ret <= 0)
 		btrfs_drew_write_unlock(&root->snapshot_lock);
 	else
fs/btrfs/inode.c +40 -49

227 lines not on this page · raw message

Signature · 1 line
2.45.0
#27 lore raw

Re: [PATCH v2 08/11] btrfs: cleanup duplicated parameters related to can_nocow_file_extent_args

Quoted 105 lines · Qu Wenruo

The following functions and structures can be simplified using the btrfs_file_extent structure:

- can_nocow_extent()
No need to return ram_bytes/orig_block_len through the parameter list, the @file_extent parameter contains all needed info.

- can_nocow_file_extent_args
The following members are no longer needed:

* disk_bytenr
This one is confusing as it's not really the btrfs_file_extent_item::disk_bytenr, but where the IO would be, thus it's file_extent::disk_bytenr + file_extent::offset now.

* num_bytes
Now file_extent::num_bytes.

* extent_offset
Now file_extent::offset.

* disk_num_bytes
Now file_extent::disk_num_bytes.

---
 fs/btrfs/btrfs_inode.h |  3 +-
 fs/btrfs/file.c        |  2 +-
 fs/btrfs/inode.c       | 89 +++++++++++++++++++-----------------------
 3 files changed, 42 insertions(+), 52 deletions(-)
fs/btrfs/btrfs_inode.h +1 -2
index f30afce4f6ca..bea343615ad1 100644
@@ -461,8 +461,7 @@ struct btrfs_file_extent {
 };
 
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
-                             u64 *orig_block_len,
-                             u64 *ram_bytes, struct btrfs_file_extent *file_extent,
+                             struct btrfs_file_extent *file_extent,
                              bool nowait, bool strict);
 
 void btrfs_del_delalloc_inode(struct btrfs_inode *inode);
fs/btrfs/file.c +1 -1
index 102b5c17ece1..6aaeb9ee048d 100644
@@ -1104,7 +1104,7 @@ int btrfs_check_nocow_lock(struct btrfs_inode *inode, loff_t pos,
                                                   &cached_state);
        }
        ret = can_nocow_extent(&inode->vfs_inode, lockstart, &num_bytes,
-                       NULL, NULL, NULL, nowait, false);
+                              NULL, nowait, false);
        if (ret <= 0)
                btrfs_drew_write_unlock(&root->snapshot_lock);
        else
fs/btrfs/inode.c +5 -15
index 8bc1f165193a..89f284ae26a4 100644
@@ -1862,16 +1862,10 @@ struct can_nocow_file_extent_args {
         */
        bool free_path;
 
-       /* Output fields. Only set when can_nocow_file_extent() returns 1. */
-
-       u64 disk_bytenr;
-       u64 disk_num_bytes;
-       u64 extent_offset;
-
-       /* Number of bytes that can be written to in NOCOW mode. */
-       u64 num_bytes;
-
-       /* The expected file extent for the NOCOW write. */
+       /*
+        * Output fields. Only set when can_nocow_file_extent() returns 1.
+        * The expected file extent for the NOCOW write.
+        */
        struct btrfs_file_extent file_extent;
 };
 
@@ -1894,6 +1888,7 @@ static int can_nocow_file_extent(struct btrfs_path *path,
        struct btrfs_root *root = inode->root;
        struct btrfs_file_extent_item *fi;
        struct btrfs_root *csum_root;
+       u64 io_start;
        u64 extent_end;
        u8 extent_type;
        int can_nocow = 0;
@@ -1906,11 +1901,6 @@ static int can_nocow_file_extent(struct btrfs_path *path,
        if (extent_type == BTRFS_FILE_EXTENT_INLINE)
                goto out;
 
-       /* Can't access these fields unless we know it's not an inline extent. */
-       args->disk_bytenr = btrfs_file_extent_disk_bytenr(leaf, fi);
-       args->disk_num_bytes = btrfs_file_extent_disk_num_bytes(leaf, fi);
-       args->extent_offset = btrfs_file_extent_offset(leaf, fi);
-
        if (!(inode->flags & BTRFS_INODE_NODATACOW) &&
            extent_type == BTRFS_FILE_EXTENT_REG)
                goto out;

fs/btrfs/inode.c static int can_nocow_file_extent(struct btrfs_path *path, · line 1919

Quoted 5 lines
fs/btrfs/inode.c +1 -1
@@ -1926,7 +1916,7 @@ static int can_nocow_file_extent(struct btrfs_path *path,
                goto out;
 
        /* An explicit hole, must COW. */
-       if (args->disk_bytenr == 0)
+       if (btrfs_file_extent_disk_num_bytes(leaf, fi) == 0)

No, this is not correct.
It's btrfs_file_extent_disk_bytenr() that we want, not btrfs_file_extent_disk_num_bytes().
In fact a disk_num_bytes of 0 should ve invalid and never happen.

Quoted 92 lines
fs/btrfs/inode.c +16 -17
                goto out;
 
        /* Compressed/encrypted/encoded extents must be COWed. */
@@ -1951,8 +1941,8 @@ static int can_nocow_file_extent(struct btrfs_path *path,
        btrfs_release_path(path);
 
        ret = btrfs_cross_ref_exist(root, btrfs_ino(inode),
-                                   key->offset - args->extent_offset,
-                                   args->disk_bytenr, args->strict, path);
+                                   key->offset - args->file_extent.offset,
+                                   args->file_extent.disk_bytenr, args->strict, path);
        WARN_ON_ONCE(ret > 0 && is_freespace_inode);
        if (ret != 0)
                goto out;
@@ -1973,21 +1963,18 @@ static int can_nocow_file_extent(struct btrfs_path *path,
            atomic_read(&root->snapshot_force_cow))
                goto out;
 
-       args->disk_bytenr += args->extent_offset;
-       args->disk_bytenr += args->start - key->offset;
-       args->num_bytes = min(args->end + 1, extent_end) - args->start;
-
-       args->file_extent.num_bytes = args->num_bytes;
+       args->file_extent.num_bytes = min(args->end + 1, extent_end) - args->start;
        args->file_extent.offset += args->start - key->offset;
+       io_start = args->file_extent.disk_bytenr + args->file_extent.offset;
 
        /*
         * Force COW if csums exist in the range. This ensures that csums for a
         * given extent are either valid or do not exist.
         */
 
-       csum_root = btrfs_csum_root(root->fs_info, args->disk_bytenr);
-       ret = btrfs_lookup_csums_list(csum_root, args->disk_bytenr,
-                                     args->disk_bytenr + args->num_bytes - 1,
+       csum_root = btrfs_csum_root(root->fs_info, io_start);
+       ret = btrfs_lookup_csums_list(csum_root, io_start,
+                                     io_start + args->file_extent.num_bytes - 1,
                                      NULL, nowait);
        WARN_ON_ONCE(ret > 0 && is_freespace_inode);
        if (ret != 0)
@@ -2046,7 +2033,6 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                struct extent_buffer *leaf;
                struct extent_state *cached_state = NULL;
                u64 extent_end;
-               u64 ram_bytes;
                u64 nocow_end;
                int extent_type;
                bool is_prealloc;
@@ -2125,7 +2111,6 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                        ret = -EUCLEAN;
                        goto error;
                }
-               ram_bytes = btrfs_file_extent_ram_bytes(leaf, fi);
                extent_end = btrfs_file_extent_end(path);
 
                /*
@@ -2145,7 +2130,9 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                        goto must_cow;
 
                ret = 0;
-               nocow_bg = btrfs_inc_nocow_writers(fs_info, nocow_args.disk_bytenr);
+               nocow_bg = btrfs_inc_nocow_writers(fs_info,
+                               nocow_args.file_extent.disk_bytenr +
+                               nocow_args.file_extent.offset);
                if (!nocow_bg) {
 must_cow:
                        /*
@@ -2181,16 +2168,18 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                        }
                }
 
-               nocow_end = cur_offset + nocow_args.num_bytes - 1;
+               nocow_end = cur_offset + nocow_args.file_extent.num_bytes - 1;
                lock_extent(&inode->io_tree, cur_offset, nocow_end, &cached_state);
 
                is_prealloc = extent_type == BTRFS_FILE_EXTENT_PREALLOC;
                if (is_prealloc) {
                        struct extent_map *em;
 
-                       em = create_io_em(inode, cur_offset, nocow_args.num_bytes,
-                                         nocow_args.disk_num_bytes, /* orig_block_len */
-                                         ram_bytes, BTRFS_COMPRESS_NONE,
+                       em = create_io_em(inode, cur_offset,
+                                         nocow_args.file_extent.num_bytes,
+                                         nocow_args.file_extent.disk_num_bytes,
+                                         nocow_args.file_extent.ram_bytes,
+                                         BTRFS_COMPRESS_NONE,
                                          &nocow_args.file_extent,
                                          BTRFS_ORDERED_PREALLOC);
                        if (IS_ERR(em)) {

fs/btrfs/inode.c static noinline int run_delalloc_nocow(struct btrfs_inode *inode, · line 2198

Quoted 7 lines
fs/btrfs/inode.c +4 -0
@@ -2203,9 +2192,16 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                        free_extent_map(em);
                }
 
+               /*
+                * Check btrfs_create_dio_extent() for why we intentionally pass
+                * incorrect value for NOCOW/PREALLOC OEs.
+                */

If in the next version you remove that similar comment/rant about OEs and disk_bytenr, also remove this one.

Everything else in this patch looks fine, thanks.

Quoted 104 lines
fs/btrfs/inode.c +14 -17
                ordered = btrfs_alloc_ordered_extent(inode, cur_offset,
-                               nocow_args.num_bytes, nocow_args.num_bytes,
-                               nocow_args.disk_bytenr, nocow_args.num_bytes, 0,
+                               nocow_args.file_extent.num_bytes,
+                               nocow_args.file_extent.num_bytes,
+                               nocow_args.file_extent.disk_bytenr +
+                               nocow_args.file_extent.offset,
+                               nocow_args.file_extent.num_bytes, 0,
                                is_prealloc
                                ? (1 << BTRFS_ORDERED_PREALLOC)
                                : (1 << BTRFS_ORDERED_NOCOW),
@@ -7144,8 +7140,7 @@ static bool btrfs_extent_readonly(struct btrfs_fs_info *fs_info, u64 bytenr)
  *      any ordered extents.
  */
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
-                             u64 *orig_block_len,
-                             u64 *ram_bytes, struct btrfs_file_extent *file_extent,
+                             struct btrfs_file_extent *file_extent,
                              bool nowait, bool strict)
 {
        struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
@@ -7196,8 +7191,6 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
 
        fi = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_file_extent_item);
        found_type = btrfs_file_extent_type(leaf, fi);
-       if (ram_bytes)
-               *ram_bytes = btrfs_file_extent_ram_bytes(leaf, fi);
 
        nocow_args.start = offset;
        nocow_args.end = offset + *len - 1;
@@ -7215,14 +7208,15 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
        }
 
        ret = 0;
-       if (btrfs_extent_readonly(fs_info, nocow_args.disk_bytenr))
+       if (btrfs_extent_readonly(fs_info,
+                               nocow_args.file_extent.disk_bytenr + nocow_args.file_extent.offset))
                goto out;
 
        if (!(BTRFS_I(inode)->flags & BTRFS_INODE_NODATACOW) &&
            found_type == BTRFS_FILE_EXTENT_PREALLOC) {
                u64 range_end;
 
-               range_end = round_up(offset + nocow_args.num_bytes,
+               range_end = round_up(offset + nocow_args.file_extent.num_bytes,
                                     root->fs_info->sectorsize) - 1;
                ret = test_range_bit_exists(io_tree, offset, range_end, EXTENT_DELALLOC);
                if (ret) {
@@ -7231,13 +7225,11 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
                }
        }
 
-       if (orig_block_len)
-               *orig_block_len = nocow_args.disk_num_bytes;
        if (file_extent)
                memcpy(file_extent, &nocow_args.file_extent,
                       sizeof(struct btrfs_file_extent));
 
-       *len = nocow_args.num_bytes;
+       *len = nocow_args.file_extent.num_bytes;
        ret = 1;
 out:
        btrfs_free_path(path);
@@ -7422,7 +7414,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
        struct btrfs_file_extent file_extent = { 0 };
        struct extent_map *em = *map;
        int type;
-       u64 block_start, orig_block_len, ram_bytes;
+       u64 block_start;
        struct btrfs_block_group *bg;
        bool can_nocow = false;
        bool space_reserved = false;
@@ -7450,7 +7442,6 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                block_start = extent_map_block_start(em) + (start - em->start);
 
                if (can_nocow_extent(inode, start, &len,
-                                    &orig_block_len, &ram_bytes,
                                     &file_extent, false, false) == 1) {
                        bg = btrfs_inc_nocow_writers(fs_info, block_start);
                        if (bg)
@@ -7477,8 +7468,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                space_reserved = true;
 
                em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
-                                             orig_block_len,
-                                             ram_bytes, type,
+                                             file_extent.disk_num_bytes,
+                                             file_extent.ram_bytes, type,
                                              &file_extent);
                btrfs_dec_nocow_writers(bg);
                if (type == BTRFS_ORDERED_PREALLOC) {
@@ -10709,7 +10700,7 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file,
                free_extent_map(em);
                em = NULL;
 
-               ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, false, true);
+               ret = can_nocow_extent(inode, start, &len, NULL, false, true);
                if (ret < 0) {
                        goto out;
                } else if (ret) {
--

2.45.0

#28 lore raw

Re: [PATCH v2 08/11] btrfs: cleanup duplicated parameters related to can_nocow_file_extent_args

Quoted 1 line · Filipe Manana

On Fri, May 3, 2024 at 7:02 AM Qu Wenruo <[email protected]> wrote:

[...]

Quoted 8 lines

fs/btrfs/inode.c static int can_nocow_file_extent(struct btrfs_path *path, · line 1919

Quoted 6 lines
fs/btrfs/inode.c +1 -1
@@ -1926,7 +1916,7 @@ static int can_nocow_file_extent(struct btrfs_path *path,
                 goto out;
 
         /* An explicit hole, must COW. */
-       if (args->disk_bytenr == 0)
+       if (btrfs_file_extent_disk_num_bytes(leaf, fi) == 0)

No, this is not correct.

All my bad, will fix it definitely.

Quoted 3 lines

It's btrfs_file_extent_disk_bytenr() that we want, not btrfs_file_extent_disk_num_bytes().
In fact a disk_num_bytes of 0 should ve invalid and never happen.

Thankfully for most cases, a explict hole has both its disk_num_bytes and disk_bytenr as zeros, thus I didn't get any test case triggered:

item 6 key (257 EXTENT_DATA 0) itemoff 15816 itemsize 53
generation 7 type 1 (regular)
extent data disk byte 0 nr 0
extent data offset 0 nr 65536 ram 65536 extent compression 0 (none)

But still I should fix that.

Thanks,
Qu

Quoted 212 lines
Quoted 93 lines
fs/btrfs/inode.c +16 -17
                 goto out;
 
         /* Compressed/encrypted/encoded extents must be COWed. */
@@ -1951,8 +1941,8 @@ static int can_nocow_file_extent(struct btrfs_path *path,
         btrfs_release_path(path);
 
         ret = btrfs_cross_ref_exist(root, btrfs_ino(inode),
-                                   key->offset - args->extent_offset,
-                                   args->disk_bytenr, args->strict, path);
+                                   key->offset - args->file_extent.offset,
+                                   args->file_extent.disk_bytenr, args->strict, path);
         WARN_ON_ONCE(ret > 0 && is_freespace_inode);
         if (ret != 0)
                 goto out;
@@ -1973,21 +1963,18 @@ static int can_nocow_file_extent(struct btrfs_path *path,
             atomic_read(&root->snapshot_force_cow))
                 goto out;
 
-       args->disk_bytenr += args->extent_offset;
-       args->disk_bytenr += args->start - key->offset;
-       args->num_bytes = min(args->end + 1, extent_end) - args->start;
-
-       args->file_extent.num_bytes = args->num_bytes;
+       args->file_extent.num_bytes = min(args->end + 1, extent_end) - args->start;
         args->file_extent.offset += args->start - key->offset;
+       io_start = args->file_extent.disk_bytenr + args->file_extent.offset;
 
         /*
          * Force COW if csums exist in the range. This ensures that csums for a
          * given extent are either valid or do not exist.
          */
 
-       csum_root = btrfs_csum_root(root->fs_info, args->disk_bytenr);
-       ret = btrfs_lookup_csums_list(csum_root, args->disk_bytenr,
-                                     args->disk_bytenr + args->num_bytes - 1,
+       csum_root = btrfs_csum_root(root->fs_info, io_start);
+       ret = btrfs_lookup_csums_list(csum_root, io_start,
+                                     io_start + args->file_extent.num_bytes - 1,
                                       NULL, nowait);
         WARN_ON_ONCE(ret > 0 && is_freespace_inode);
         if (ret != 0)
@@ -2046,7 +2033,6 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                 struct extent_buffer *leaf;
                 struct extent_state *cached_state = NULL;
                 u64 extent_end;
-               u64 ram_bytes;
                 u64 nocow_end;
                 int extent_type;
                 bool is_prealloc;
@@ -2125,7 +2111,6 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                         ret = -EUCLEAN;
                         goto error;
                 }
-               ram_bytes = btrfs_file_extent_ram_bytes(leaf, fi);
                 extent_end = btrfs_file_extent_end(path);
 
                 /*
@@ -2145,7 +2130,9 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                         goto must_cow;
 
                 ret = 0;
-               nocow_bg = btrfs_inc_nocow_writers(fs_info, nocow_args.disk_bytenr);
+               nocow_bg = btrfs_inc_nocow_writers(fs_info,
+                               nocow_args.file_extent.disk_bytenr +
+                               nocow_args.file_extent.offset);
                 if (!nocow_bg) {
  must_cow:
                         /*
@@ -2181,16 +2168,18 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                         }
                 }
 
-               nocow_end = cur_offset + nocow_args.num_bytes - 1;
+               nocow_end = cur_offset + nocow_args.file_extent.num_bytes - 1;
                 lock_extent(&inode->io_tree, cur_offset, nocow_end, &cached_state);
 
                 is_prealloc = extent_type == BTRFS_FILE_EXTENT_PREALLOC;
                 if (is_prealloc) {
                         struct extent_map *em;
 
-                       em = create_io_em(inode, cur_offset, nocow_args.num_bytes,
-                                         nocow_args.disk_num_bytes, /* orig_block_len */
-                                         ram_bytes, BTRFS_COMPRESS_NONE,
+                       em = create_io_em(inode, cur_offset,
+                                         nocow_args.file_extent.num_bytes,
+                                         nocow_args.file_extent.disk_num_bytes,
+                                         nocow_args.file_extent.ram_bytes,
+                                         BTRFS_COMPRESS_NONE,
                                           &nocow_args.file_extent,
                                           BTRFS_ORDERED_PREALLOC);
                         if (IS_ERR(em)) {

fs/btrfs/inode.c static noinline int run_delalloc_nocow(struct btrfs_inode *inode, · line 2198

Quoted 7 lines
fs/btrfs/inode.c +4 -0
@@ -2203,9 +2192,16 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                         free_extent_map(em);
                 }
 
+               /*
+                * Check btrfs_create_dio_extent() for why we intentionally pass
+                * incorrect value for NOCOW/PREALLOC OEs.
+                */

If in the next version you remove that similar comment/rant about OEs and disk_bytenr, also remove this one.

Everything else in this patch looks fine, thanks.

Quoted 104 lines
fs/btrfs/inode.c +14 -17
                 ordered = btrfs_alloc_ordered_extent(inode, cur_offset,
-                               nocow_args.num_bytes, nocow_args.num_bytes,
-                               nocow_args.disk_bytenr, nocow_args.num_bytes, 0,
+                               nocow_args.file_extent.num_bytes,
+                               nocow_args.file_extent.num_bytes,
+                               nocow_args.file_extent.disk_bytenr +
+                               nocow_args.file_extent.offset,
+                               nocow_args.file_extent.num_bytes, 0,
                                 is_prealloc
                                 ? (1 << BTRFS_ORDERED_PREALLOC)
                                 : (1 << BTRFS_ORDERED_NOCOW),
@@ -7144,8 +7140,7 @@ static bool btrfs_extent_readonly(struct btrfs_fs_info *fs_info, u64 bytenr)
   *      any ordered extents.
   */
  noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
-                             u64 *orig_block_len,
-                             u64 *ram_bytes, struct btrfs_file_extent *file_extent,
+                             struct btrfs_file_extent *file_extent,
                               bool nowait, bool strict)
  {
         struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
@@ -7196,8 +7191,6 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
 
         fi = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_file_extent_item);
         found_type = btrfs_file_extent_type(leaf, fi);
-       if (ram_bytes)
-               *ram_bytes = btrfs_file_extent_ram_bytes(leaf, fi);
 
         nocow_args.start = offset;
         nocow_args.end = offset + *len - 1;
@@ -7215,14 +7208,15 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
         }
 
         ret = 0;
-       if (btrfs_extent_readonly(fs_info, nocow_args.disk_bytenr))
+       if (btrfs_extent_readonly(fs_info,
+                               nocow_args.file_extent.disk_bytenr + nocow_args.file_extent.offset))
                 goto out;
 
         if (!(BTRFS_I(inode)->flags & BTRFS_INODE_NODATACOW) &&
             found_type == BTRFS_FILE_EXTENT_PREALLOC) {
                 u64 range_end;
 
-               range_end = round_up(offset + nocow_args.num_bytes,
+               range_end = round_up(offset + nocow_args.file_extent.num_bytes,
                                      root->fs_info->sectorsize) - 1;
                 ret = test_range_bit_exists(io_tree, offset, range_end, EXTENT_DELALLOC);
                 if (ret) {
@@ -7231,13 +7225,11 @@ noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
                 }
         }
 
-       if (orig_block_len)
-               *orig_block_len = nocow_args.disk_num_bytes;
         if (file_extent)
                 memcpy(file_extent, &nocow_args.file_extent,
                        sizeof(struct btrfs_file_extent));
 
-       *len = nocow_args.num_bytes;
+       *len = nocow_args.file_extent.num_bytes;
         ret = 1;
  out:
         btrfs_free_path(path);
@@ -7422,7 +7414,7 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
         struct btrfs_file_extent file_extent = { 0 };
         struct extent_map *em = *map;
         int type;
-       u64 block_start, orig_block_len, ram_bytes;
+       u64 block_start;
         struct btrfs_block_group *bg;
         bool can_nocow = false;
         bool space_reserved = false;
@@ -7450,7 +7442,6 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                 block_start = extent_map_block_start(em) + (start - em->start);
 
                 if (can_nocow_extent(inode, start, &len,
-                                    &orig_block_len, &ram_bytes,
                                      &file_extent, false, false) == 1) {
                         bg = btrfs_inc_nocow_writers(fs_info, block_start);
                         if (bg)
@@ -7477,8 +7468,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                 space_reserved = true;
 
                 em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
-                                             orig_block_len,
-                                             ram_bytes, type,
+                                             file_extent.disk_num_bytes,
+                                             file_extent.ram_bytes, type,
                                               &file_extent);
                 btrfs_dec_nocow_writers(bg);
                 if (type == BTRFS_ORDERED_PREALLOC) {
@@ -10709,7 +10700,7 @@ static int btrfs_swap_activate(struct swap_info_struct *sis, struct file *file,
                 free_extent_map(em);
                 em = NULL;
 
-               ret = can_nocow_extent(inode, start, &len, NULL, NULL, NULL, false, true);
+               ret = can_nocow_extent(inode, start, &len, NULL, false, true);
                 if (ret < 0) {
                         goto out;
                 } else if (ret) {
--

2.45.0

author
#29 lore raw

PATCH v2 9/11 btrfs: cleanup duplicated parameters related to btrfs_alloc_ordered_extent

In reply to Qu Wenruo

All parameters after @filepos of btrfs_alloc_ordered_extent() can be replaced with btrfs_file_extent structure.

This patch would do the cleanup, meanwhile some points to note:

- Move btrfs_file_extent structure to ordered-data.h
The structure is needed by both btrfs_alloc_ordered_extent() and can_nocow_extent(), but since btrfs_inode.h would include ordered-data.h, so we need to move the structure to ordered-data.h.

- Move the special handling of NOCOW/PREALLOC into btrfs_alloc_ordered_extent()
Previously we have two call sites intentionally forging the numbers, but now with accurate btrfs_file_extent results, it's better to move the special handling into btrfs_alloc_ordered_extent(), so callers can just pass the accurate file_extent.

---
 fs/btrfs/btrfs_inode.h  | 17 -----------
 fs/btrfs/inode.c        | 64 +++++++----------------------------------
 fs/btrfs/ordered-data.c | 36 +++++++++++++++++++----
 fs/btrfs/ordered-data.h | 22 ++++++++++++--
 4 files changed, 59 insertions(+), 80 deletions(-)
fs/btrfs/btrfs_inode.h +0 -17
index bea343615ad1..6622485389dc 100644
@@ -443,23 +443,6 @@ int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page,
 			    u32 pgoff, u8 *csum, const u8 * const csum_expected);
 bool btrfs_data_csum_ok(struct btrfs_bio *bbio, struct btrfs_device *dev,
 			u32 bio_offset, struct bio_vec *bv);
-
-/*
- * A more access-friendly representation of btrfs_file_extent_item.
- *
- * Unused members are excluded.
- */
-struct btrfs_file_extent {
-	u64 disk_bytenr;
-	u64 disk_num_bytes;
-
-	u64 num_bytes;
-	u64 ram_bytes;
-	u64 offset;
-
-	u8 compression;
-};
-
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
 			      struct btrfs_file_extent *file_extent,
 			      bool nowait, bool strict);
fs/btrfs/inode.c +10 -54

99 lines not on this page · raw message

fs/btrfs/ordered-data.c +30 -6
index c5bdd674f55c..371a85250d6a 100644
@@ -263,17 +263,41 @@ static void insert_ordered_extent(struct btrfs_ordered_extent *entry)
  */
 struct btrfs_ordered_extent *btrfs_alloc_ordered_extent(
 			struct btrfs_inode *inode, u64 file_offset,
-			u64 num_bytes, u64 ram_bytes, u64 disk_bytenr,
-			u64 disk_num_bytes, u64 offset, unsigned long flags,
-			int compress_type)
+			struct btrfs_file_extent *file_extent,
+			unsigned long flags)
 {
 	struct btrfs_ordered_extent *entry;
 
 	ASSERT((flags & ~BTRFS_ORDERED_TYPE_FLAGS) == 0);
 
-	entry = alloc_ordered_extent(inode, file_offset, num_bytes, ram_bytes,
-				     disk_bytenr, disk_num_bytes, offset, flags,
-				     compress_type);
+	/*
+	 * NOTE: I know the numbers are totally wrong for NOCOW/PREALLOC,
+	 * but it doesn't cause problem at least for now.
+	 *
+	 * For NOCOW, we don't really care about the numbers except @file_pos
+	 * and @num_bytes, as we won't insert a file extent item at all.
+	 *
+	 * For PREALLOC, we do not use ordered extent's member, but
+	 * btrfs_mark_extent_written() would handle everything.
+	 *
+	 * So here we intentionally go with pseudo numbers for the NOCOW/PREALLOC
+	 * OEs, or btrfs_extract_ordered_extent() would need a completely new
+	 * routine to handle NOCOW/PREALLOC splits, meanwhile result nothing
+	 * different.
+	 */
+	if (flags & ((1 << BTRFS_ORDERED_NOCOW) | (1 << BTRFS_ORDERED_PREALLOC)))
+		entry = alloc_ordered_extent(inode, file_offset,
+				file_extent->num_bytes, file_extent->num_bytes,
+				file_extent->disk_bytenr + file_extent->offset,
+				file_extent->num_bytes, 0, flags,
+				file_extent->compression);
+	else
+		entry = alloc_ordered_extent(inode, file_offset,
+				file_extent->num_bytes, file_extent->ram_bytes,
+				file_extent->disk_bytenr,
+				file_extent->disk_num_bytes,
+				file_extent->offset, flags,
+				file_extent->compression);
 	if (!IS_ERR(entry))
 		insert_ordered_extent(entry);
 	return entry;
fs/btrfs/ordered-data.h +19 -3
index b6f6c6b91732..5bbec06fbc8d 100644
@@ -171,11 +171,27 @@ void btrfs_mark_ordered_io_finished(struct btrfs_inode *inode,
 bool btrfs_dec_test_ordered_pending(struct btrfs_inode *inode,
 				    struct btrfs_ordered_extent **cached,
 				    u64 file_offset, u64 io_size);
+
+/*
+ * A more access-friendly representation of btrfs_file_extent_item.
+ *
+ * Unused members are excluded.
+ */
+struct btrfs_file_extent {
+	u64 disk_bytenr;
+	u64 disk_num_bytes;
+
+	u64 num_bytes;
+	u64 ram_bytes;
+	u64 offset;
+
+	u8 compression;
+};
+
 struct btrfs_ordered_extent *btrfs_alloc_ordered_extent(
 			struct btrfs_inode *inode, u64 file_offset,
-			u64 num_bytes, u64 ram_bytes, u64 disk_bytenr,
-			u64 disk_num_bytes, u64 offset, unsigned long flags,
-			int compress_type);
+			struct btrfs_file_extent *file_extent,
+			unsigned long flags);
 void btrfs_add_ordered_sum(struct btrfs_ordered_extent *entry,
 			   struct btrfs_ordered_sum *sum);
 struct btrfs_ordered_extent *btrfs_lookup_ordered_extent(struct btrfs_inode *inode,
Signature · 1 line
2.45.0
#30 lore raw

Re: [PATCH v2 09/11] btrfs: cleanup duplicated parameters related to btrfs_alloc_ordered_extent

Quoted 5 lines · Qu Wenruo

All parameters after @filepos of btrfs_alloc_ordered_extent() can be replaced with btrfs_file_extent structure.

This patch would do the cleanup, meanwhile some points to note:

would do -> does

Quoted 4 lines

- Move btrfs_file_extent structure to ordered-data.h The structure is needed by both btrfs_alloc_ordered_extent() and can_nocow_extent(), but since btrfs_inode.h would include

would include -> includes (it's including at the moment, before this patchset - there's an include of ordered-data.h in btrfs_inode.h)

Quoted 5 lines

ordered-data.h, so we need to move the structure to ordered-data.h.

- Move the special handling of NOCOW/PREALLOC into btrfs_alloc_ordered_extent()
Previously we have two call sites intentionally forging the numbers,

So this forging is the thing you don't like about disk_bytenr not matching the disk_bytenr of a file extent item, but rather that plus some offset.
I would leave the rant about it from the changelog the comments in code. Leave that for a patch that specifically addresses that.

Otherwise the rest of this patch looks fine.
Thanks.

Quoted 239 lines

but now with accurate btrfs_file_extent results, it's better to move the special handling into btrfs_alloc_ordered_extent(), so callers can just pass the accurate file_extent.

---
 fs/btrfs/btrfs_inode.h  | 17 -----------
 fs/btrfs/inode.c        | 64 +++++++----------------------------------
 fs/btrfs/ordered-data.c | 36 +++++++++++++++++++----
 fs/btrfs/ordered-data.h | 22 ++++++++++++--
 4 files changed, 59 insertions(+), 80 deletions(-)
fs/btrfs/btrfs_inode.h +0 -17
index bea343615ad1..6622485389dc 100644
@@ -443,23 +443,6 @@ int btrfs_check_sector_csum(struct btrfs_fs_info *fs_info, struct page *page,
                            u32 pgoff, u8 *csum, const u8 * const csum_expected);
 bool btrfs_data_csum_ok(struct btrfs_bio *bbio, struct btrfs_device *dev,
                        u32 bio_offset, struct bio_vec *bv);
-
-/*
- * A more access-friendly representation of btrfs_file_extent_item.
- *
- * Unused members are excluded.
- */
-struct btrfs_file_extent {
-       u64 disk_bytenr;
-       u64 disk_num_bytes;
-
-       u64 num_bytes;
-       u64 ram_bytes;
-       u64 offset;
-
-       u8 compression;
-};
-
 noinline int can_nocow_extent(struct inode *inode, u64 offset, u64 *len,
                              struct btrfs_file_extent *file_extent,
                              bool nowait, bool strict);
fs/btrfs/inode.c +10 -54
index 89f284ae26a4..eec5ecb917d8 100644
@@ -1220,14 +1220,8 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        }
        free_extent_map(em);
 
-       ordered = btrfs_alloc_ordered_extent(inode, start,      /* file_offset */
-                                      async_extent->ram_size,  /* num_bytes */
-                                      async_extent->ram_size,  /* ram_bytes */
-                                      ins.objectid,            /* disk_bytenr */
-                                      ins.offset,              /* disk_num_bytes */
-                                      0,                       /* offset */
-                                      1 << BTRFS_ORDERED_COMPRESSED,
-                                      async_extent->compress_type);
+       ordered = btrfs_alloc_ordered_extent(inode, start, &file_extent,
+                                      1 << BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(ordered)) {
                btrfs_drop_extent_map_range(inode, start, end, false);
                ret = PTR_ERR(ordered);
@@ -1463,10 +1457,8 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                }
                free_extent_map(em);
 
-               ordered = btrfs_alloc_ordered_extent(inode, start, ram_size,
-                                       ram_size, ins.objectid, cur_alloc_size,
-                                       0, 1 << BTRFS_ORDERED_REGULAR,
-                                       BTRFS_COMPRESS_NONE);
+               ordered = btrfs_alloc_ordered_extent(inode, start, &file_extent,
+                                                    1 << BTRFS_ORDERED_REGULAR);
                if (IS_ERR(ordered)) {
                        unlock_extent(&inode->io_tree, start,
                                      start + ram_size - 1, &cached);
@@ -2192,20 +2184,11 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                        free_extent_map(em);
                }
 
-               /*
-                * Check btrfs_create_dio_extent() for why we intentionally pass
-                * incorrect value for NOCOW/PREALLOC OEs.
-                */
                ordered = btrfs_alloc_ordered_extent(inode, cur_offset,
-                               nocow_args.file_extent.num_bytes,
-                               nocow_args.file_extent.num_bytes,
-                               nocow_args.file_extent.disk_bytenr +
-                               nocow_args.file_extent.offset,
-                               nocow_args.file_extent.num_bytes, 0,
+                               &nocow_args.file_extent,
                                is_prealloc
                                ? (1 << BTRFS_ORDERED_PREALLOC)
-                               : (1 << BTRFS_ORDERED_NOCOW),
-                               BTRFS_COMPRESS_NONE);
+                               : (1 << BTRFS_ORDERED_NOCOW));
                btrfs_dec_nocow_writers(nocow_bg);
                if (IS_ERR(ordered)) {
                        if (is_prealloc) {
@@ -7020,33 +7003,9 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                        goto out;
        }
 
-       /*
-        * NOTE: I know the numbers are totally wrong for NOCOW/PREALLOC,
-        * but it doesn't cause problem at least for now.
-        *
-        * For regular writes, we would have file_extent->offset as 0,
-        * thus we really only need disk_bytenr, every other length
-        * (disk_num_bytes/ram_bytes) would match @len and fe->num_bytes.
-        * The current numbers are totally fine.
-        *
-        * For NOCOW, we don't really care about the numbers except @file_pos
-        * and @num_bytes, as we won't insert a file extent item at all.
-        *
-        * For PREALLOC, we do not use ordered extent's member, but
-        * btrfs_mark_extent_written() would handle everything.
-        *
-        * So here we intentionally go with pseudo numbers for the NOCOW/PREALLOC
-        * OEs, or btrfs_extract_ordered_extent() would need a completely new
-        * routine to handle NOCOW/PREALLOC splits, meanwhile result nothing
-        * different.
-        */
-       ordered = btrfs_alloc_ordered_extent(inode, start, len, len,
-                                            file_extent->disk_bytenr +
-                                            file_extent->offset,
-                                            len, 0,
+       ordered = btrfs_alloc_ordered_extent(inode, start, file_extent,
                                             (1 << type) |
-                                            (1 << BTRFS_ORDERED_DIRECT),
-                                            BTRFS_COMPRESS_NONE);
+                                            (1 << BTRFS_ORDERED_DIRECT));
        if (IS_ERR(ordered)) {
                if (em) {
                        free_extent_map(em);
@@ -10377,12 +10336,9 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
        }
        free_extent_map(em);
 
-       ordered = btrfs_alloc_ordered_extent(inode, start, num_bytes, ram_bytes,
-                                      ins.objectid, ins.offset,
-                                      encoded->unencoded_offset,
+       ordered = btrfs_alloc_ordered_extent(inode, start, &file_extent,
                                       (1 << BTRFS_ORDERED_ENCODED) |
-                                      (1 << BTRFS_ORDERED_COMPRESSED),
-                                      compression);
+                                      (1 << BTRFS_ORDERED_COMPRESSED));
        if (IS_ERR(ordered)) {
                btrfs_drop_extent_map_range(inode, start, end, false);
                ret = PTR_ERR(ordered);
fs/btrfs/ordered-data.c +30 -6
index c5bdd674f55c..371a85250d6a 100644
@@ -263,17 +263,41 @@ static void insert_ordered_extent(struct btrfs_ordered_extent *entry)
  */
 struct btrfs_ordered_extent *btrfs_alloc_ordered_extent(
                        struct btrfs_inode *inode, u64 file_offset,
-                       u64 num_bytes, u64 ram_bytes, u64 disk_bytenr,
-                       u64 disk_num_bytes, u64 offset, unsigned long flags,
-                       int compress_type)
+                       struct btrfs_file_extent *file_extent,
+                       unsigned long flags)
 {
        struct btrfs_ordered_extent *entry;
 
        ASSERT((flags & ~BTRFS_ORDERED_TYPE_FLAGS) == 0);
 
-       entry = alloc_ordered_extent(inode, file_offset, num_bytes, ram_bytes,
-                                    disk_bytenr, disk_num_bytes, offset, flags,
-                                    compress_type);
+       /*
+        * NOTE: I know the numbers are totally wrong for NOCOW/PREALLOC,
+        * but it doesn't cause problem at least for now.
+        *
+        * For NOCOW, we don't really care about the numbers except @file_pos
+        * and @num_bytes, as we won't insert a file extent item at all.
+        *
+        * For PREALLOC, we do not use ordered extent's member, but
+        * btrfs_mark_extent_written() would handle everything.
+        *
+        * So here we intentionally go with pseudo numbers for the NOCOW/PREALLOC
+        * OEs, or btrfs_extract_ordered_extent() would need a completely new
+        * routine to handle NOCOW/PREALLOC splits, meanwhile result nothing
+        * different.
+        */
+       if (flags & ((1 << BTRFS_ORDERED_NOCOW) | (1 << BTRFS_ORDERED_PREALLOC)))
+               entry = alloc_ordered_extent(inode, file_offset,
+                               file_extent->num_bytes, file_extent->num_bytes,
+                               file_extent->disk_bytenr + file_extent->offset,
+                               file_extent->num_bytes, 0, flags,
+                               file_extent->compression);
+       else
+               entry = alloc_ordered_extent(inode, file_offset,
+                               file_extent->num_bytes, file_extent->ram_bytes,
+                               file_extent->disk_bytenr,
+                               file_extent->disk_num_bytes,
+                               file_extent->offset, flags,
+                               file_extent->compression);
        if (!IS_ERR(entry))
                insert_ordered_extent(entry);
        return entry;
fs/btrfs/ordered-data.h +19 -4
index b6f6c6b91732..5bbec06fbc8d 100644
@@ -171,11 +171,27 @@ void btrfs_mark_ordered_io_finished(struct btrfs_inode *inode,
 bool btrfs_dec_test_ordered_pending(struct btrfs_inode *inode,
                                    struct btrfs_ordered_extent **cached,
                                    u64 file_offset, u64 io_size);
+
+/*
+ * A more access-friendly representation of btrfs_file_extent_item.
+ *
+ * Unused members are excluded.
+ */
+struct btrfs_file_extent {
+       u64 disk_bytenr;
+       u64 disk_num_bytes;
+
+       u64 num_bytes;
+       u64 ram_bytes;
+       u64 offset;
+
+       u8 compression;
+};
+
 struct btrfs_ordered_extent *btrfs_alloc_ordered_extent(
                        struct btrfs_inode *inode, u64 file_offset,
-                       u64 num_bytes, u64 ram_bytes, u64 disk_bytenr,
-                       u64 disk_num_bytes, u64 offset, unsigned long flags,
-                       int compress_type);
+                       struct btrfs_file_extent *file_extent,
+                       unsigned long flags);
 void btrfs_add_ordered_sum(struct btrfs_ordered_extent *entry,
                           struct btrfs_ordered_sum *sum);
 struct btrfs_ordered_extent *btrfs_lookup_ordered_extent(struct btrfs_inode *inode,
--

2.45.0

author
#31 lore raw

PATCH v2 10/11 btrfs: cleanup duplicated parameters related to create_io_em()

In reply to Qu Wenruo

Most parameters of create_io_em() can be replaced by the members with the same name inside btrfs_file_extent.

Do a straight parameters cleanup here.

---
 fs/btrfs/inode.c | 50 +++++++++++++-----------------------------------
 1 file changed, 13 insertions(+), 37 deletions(-)
fs/btrfs/inode.c +13 -37
index eec5ecb917d8..a95dc2333972 100644
@@ -138,9 +138,6 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
 				     u64 end, struct writeback_control *wbc,
 				     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-				       u64 len,
-				       u64 disk_num_bytes,
-				       u64 ram_bytes, int compress_type,
 				       struct btrfs_file_extent *file_extent,
 				       int type);
 
@@ -1207,12 +1204,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
 	file_extent.offset = 0;
 	file_extent.compression = async_extent->compress_type;
 
-	em = create_io_em(inode, start,
-			  async_extent->ram_size,	/* len */
-			  ins.offset,			/* orig_block_len */
-			  async_extent->ram_size,	/* ram_bytes */
-			  async_extent->compress_type,
-			  &file_extent,
+	em = create_io_em(inode, start, &file_extent,
 			  BTRFS_ORDERED_COMPRESSED);
 	if (IS_ERR(em)) {
 		ret = PTR_ERR(em);
@@ -1443,11 +1435,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
 		lock_extent(&inode->io_tree, start, start + ram_size - 1,
 			    &cached);
 
-		em = create_io_em(inode, start, ins.offset, /* len */
-				  ins.offset, /* orig_block_len */
-				  ram_size, /* ram_bytes */
-				  BTRFS_COMPRESS_NONE, /* compress_type */
-				  &file_extent,
+		em = create_io_em(inode, start, &file_extent,
 				  BTRFS_ORDERED_REGULAR /* type */);
 		if (IS_ERR(em)) {
 			unlock_extent(&inode->io_tree, start,
@@ -2168,10 +2156,6 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
 			struct extent_map *em;
 
 			em = create_io_em(inode, cur_offset,
-					  nocow_args.file_extent.num_bytes,
-					  nocow_args.file_extent.disk_num_bytes,
-					  nocow_args.file_extent.ram_bytes,
-					  BTRFS_COMPRESS_NONE,
 					  &nocow_args.file_extent,
 					  BTRFS_ORDERED_PREALLOC);
 			if (IS_ERR(em)) {
@@ -6995,10 +6979,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
 	struct btrfs_ordered_extent *ordered;
 
 	if (type != BTRFS_ORDERED_NOCOW) {
-		em = create_io_em(inode, start, len,
-				  orig_block_len, ram_bytes,
-				  BTRFS_COMPRESS_NONE, /* compress_type */
-				  file_extent, type);
+		em = create_io_em(inode, start, file_extent, type);
 		if (IS_ERR(em))
 			goto out;
 	}
@@ -7290,9 +7271,6 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
 
 /* The callers of this must take lock_extent() */
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-				       u64 len,
-				       u64 disk_num_bytes,
-				       u64 ram_bytes, int compress_type,
 				       struct btrfs_file_extent *file_extent,
 				       int type)
 {
@@ -7314,25 +7292,25 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 	switch (type) {
 	case BTRFS_ORDERED_PREALLOC:
 		/* We're only referring part of a larger preallocated extent. */
-		ASSERT(len <= ram_bytes);
+		ASSERT(file_extent->num_bytes <= file_extent->ram_bytes);
 		break;
 	case BTRFS_ORDERED_REGULAR:
 		/* COW results a new extent matching our file extent size. */
-		ASSERT(disk_num_bytes == len);
-		ASSERT(ram_bytes == len);
+		ASSERT(file_extent->disk_num_bytes == file_extent->num_bytes);
+		ASSERT(file_extent->ram_bytes == file_extent->num_bytes);
 
 		/* Since it's a new extent, we should not have any offset. */
 		ASSERT(file_extent->offset == 0);
 		break;
 	case BTRFS_ORDERED_COMPRESSED:
 		/* Must be compressed. */
-		ASSERT(compress_type != BTRFS_COMPRESS_NONE);
+		ASSERT(file_extent->compression != BTRFS_COMPRESS_NONE);
 
 		/*
 		 * Encoded write can make us to refer to part of the
 		 * uncompressed extent.
 		 */
-		ASSERT(len <= ram_bytes);
+		ASSERT(file_extent->num_bytes <= file_extent->ram_bytes);
 		break;
 	}
 
@@ -7341,15 +7319,15 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
 		return ERR_PTR(-ENOMEM);
 
 	em->start = start;
-	em->len = len;
+	em->len = file_extent->num_bytes;
 	em->disk_bytenr = file_extent->disk_bytenr;
-	em->disk_num_bytes = disk_num_bytes;
-	em->ram_bytes = ram_bytes;
+	em->disk_num_bytes = file_extent->disk_num_bytes;
+	em->ram_bytes = file_extent->ram_bytes;
 	em->generation = -1;
 	em->offset = file_extent->offset;
 	em->flags |= EXTENT_FLAG_PINNED;
 	if (type == BTRFS_ORDERED_COMPRESSED)
-		extent_map_set_compression(em, compress_type);
+		extent_map_set_compression(em, file_extent->compression);
 
 	ret = btrfs_replace_extent_map_range(inode, em, true);
 	if (ret) {
@@ -10327,9 +10305,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
 	file_extent.ram_bytes = ram_bytes;
 	file_extent.offset = encoded->unencoded_offset;
 	file_extent.compression = compression;
-	em = create_io_em(inode, start, num_bytes,
-			  ins.offset, ram_bytes, compression,
-			  &file_extent, BTRFS_ORDERED_COMPRESSED);
+	em = create_io_em(inode, start, &file_extent, BTRFS_ORDERED_COMPRESSED);
 	if (IS_ERR(em)) {
 		ret = PTR_ERR(em);
 		goto out_free_reserved;
Signature · 1 line
2.45.0
#32 lore raw

Re: [PATCH v2 10/11] btrfs: cleanup duplicated parameters related to create_io_em()

Quoted 27 lines · Qu Wenruo

Most parameters of create_io_em() can be replaced by the members with the same name inside btrfs_file_extent.

Do a straight parameters cleanup here.

---
 fs/btrfs/inode.c | 50 +++++++++++++-----------------------------------
 1 file changed, 13 insertions(+), 37 deletions(-)
fs/btrfs/inode.c +0 -3
index eec5ecb917d8..a95dc2333972 100644
@@ -138,9 +138,6 @@ static noinline int run_delalloc_cow(struct btrfs_inode *inode,
                                     u64 end, struct writeback_control *wbc,
                                     bool pages_dirty);
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-                                      u64 len,
-                                      u64 disk_num_bytes,
-                                      u64 ram_bytes, int compress_type,
                                       struct btrfs_file_extent *file_extent,
                                       int type);
 
@@ -1207,12 +1204,7 @@ static void submit_one_async_extent(struct async_chunk *async_chunk,
        file_extent.offset = 0;

fs/btrfs/inode.c static void submit_one_async_extent(struct async_chunk *async_chunk, · line 1208

Quoted 10 lines
fs/btrfs/inode.c +1 -6
        file_extent.compression = async_extent->compress_type;
 
-       em = create_io_em(inode, start,
-                         async_extent->ram_size,       /* len */
-                         ins.offset,                   /* orig_block_len */
-                         async_extent->ram_size,       /* ram_bytes */
-                         async_extent->compress_type,
-                         &file_extent,
+       em = create_io_em(inode, start, &file_extent,
                          BTRFS_ORDERED_COMPRESSED);

Btw, as we're changing this, we can take the chance to make everything in a single line since it fits and gets more readable.

fs/btrfs/inode.c static noinline int cow_file_range(struct btrfs_inode *inode, · line 1439

Quoted 13 lines
fs/btrfs/inode.c +1 -5
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
@@ -1443,11 +1435,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode,
                lock_extent(&inode->io_tree, start, start + ram_size - 1,
                            &cached);
 
-               em = create_io_em(inode, start, ins.offset, /* len */
-                                 ins.offset, /* orig_block_len */
-                                 ram_size, /* ram_bytes */
-                                 BTRFS_COMPRESS_NONE, /* compress_type */
-                                 &file_extent,
+               em = create_io_em(inode, start, &file_extent,
                                  BTRFS_ORDERED_REGULAR /* type */);

Same here, and remove the /* type */ comment there which is superfluous anyway.

fs/btrfs/inode.c static noinline int run_delalloc_nocow(struct btrfs_inode *inode, · line 2160

Quoted 12 lines
fs/btrfs/inode.c +0 -4
                if (IS_ERR(em)) {
                        unlock_extent(&inode->io_tree, start,
@@ -2168,10 +2156,6 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,
                        struct extent_map *em;
 
                        em = create_io_em(inode, cur_offset,
-                                         nocow_args.file_extent.num_bytes,
-                                         nocow_args.file_extent.disk_num_bytes,
-                                         nocow_args.file_extent.ram_bytes,
-                                         BTRFS_COMPRESS_NONE,
                                          &nocow_args.file_extent,
                                          BTRFS_ORDERED_PREALLOC);

Same here.

The rest looks good, simple patch.
Thanks.

Quoted 89 lines
fs/btrfs/inode.c +11 -20
                        if (IS_ERR(em)) {
@@ -6995,10 +6979,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
        struct btrfs_ordered_extent *ordered;
 
        if (type != BTRFS_ORDERED_NOCOW) {
-               em = create_io_em(inode, start, len,
-                                 orig_block_len, ram_bytes,
-                                 BTRFS_COMPRESS_NONE, /* compress_type */
-                                 file_extent, type);
+               em = create_io_em(inode, start, file_extent, type);
                if (IS_ERR(em))
                        goto out;
        }
@@ -7290,9 +7271,6 @@ static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
 
 /* The callers of this must take lock_extent() */
 static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
-                                      u64 len,
-                                      u64 disk_num_bytes,
-                                      u64 ram_bytes, int compress_type,
                                       struct btrfs_file_extent *file_extent,
                                       int type)
 {
@@ -7314,25 +7292,25 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
        switch (type) {
        case BTRFS_ORDERED_PREALLOC:
                /* We're only referring part of a larger preallocated extent. */
-               ASSERT(len <= ram_bytes);
+               ASSERT(file_extent->num_bytes <= file_extent->ram_bytes);
                break;
        case BTRFS_ORDERED_REGULAR:
                /* COW results a new extent matching our file extent size. */
-               ASSERT(disk_num_bytes == len);
-               ASSERT(ram_bytes == len);
+               ASSERT(file_extent->disk_num_bytes == file_extent->num_bytes);
+               ASSERT(file_extent->ram_bytes == file_extent->num_bytes);
 
                /* Since it's a new extent, we should not have any offset. */
                ASSERT(file_extent->offset == 0);
                break;
        case BTRFS_ORDERED_COMPRESSED:
                /* Must be compressed. */
-               ASSERT(compress_type != BTRFS_COMPRESS_NONE);
+               ASSERT(file_extent->compression != BTRFS_COMPRESS_NONE);
 
                /*
                 * Encoded write can make us to refer to part of the
                 * uncompressed extent.
                 */
-               ASSERT(len <= ram_bytes);
+               ASSERT(file_extent->num_bytes <= file_extent->ram_bytes);
                break;
        }
 
@@ -7341,15 +7319,15 @@ static struct extent_map *create_io_em(struct btrfs_inode *inode, u64 start,
                return ERR_PTR(-ENOMEM);
 
        em->start = start;
-       em->len = len;
+       em->len = file_extent->num_bytes;
        em->disk_bytenr = file_extent->disk_bytenr;
-       em->disk_num_bytes = disk_num_bytes;
-       em->ram_bytes = ram_bytes;
+       em->disk_num_bytes = file_extent->disk_num_bytes;
+       em->ram_bytes = file_extent->ram_bytes;
        em->generation = -1;
        em->offset = file_extent->offset;
        em->flags |= EXTENT_FLAG_PINNED;
        if (type == BTRFS_ORDERED_COMPRESSED)
-               extent_map_set_compression(em, compress_type);
+               extent_map_set_compression(em, file_extent->compression);
 
        ret = btrfs_replace_extent_map_range(inode, em, true);
        if (ret) {
@@ -10327,9 +10305,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,
        file_extent.ram_bytes = ram_bytes;
        file_extent.offset = encoded->unencoded_offset;
        file_extent.compression = compression;
-       em = create_io_em(inode, start, num_bytes,
-                         ins.offset, ram_bytes, compression,
-                         &file_extent, BTRFS_ORDERED_COMPRESSED);
+       em = create_io_em(inode, start, &file_extent, BTRFS_ORDERED_COMPRESSED);
        if (IS_ERR(em)) {
                ret = PTR_ERR(em);
                goto out_free_reserved;
--

2.45.0

author
#33 lore raw

PATCH v2 11/11 btrfs: cleanup duplicated parameters related to btrfs_create_dio_extent()

In reply to Qu Wenruo

The following 3 parameters can be cleaned up using btrfs_file_extent structure:

- len
btrfs_file_extent::num_bytes

- orig_block_len
btrfs_file_extent::disk_num_bytes

- ram_bytes
btrfs_file_extent::ram_bytes

---
 fs/btrfs/inode.c | 22 ++++++++--------------
 1 file changed, 8 insertions(+), 14 deletions(-)
fs/btrfs/inode.c +8 -14
index a95dc2333972..09974c86d3d1 100644
@@ -6969,11 +6969,8 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
 static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
 						  struct btrfs_dio_data *dio_data,
 						  const u64 start,
-						  const u64 len,
-						  const u64 orig_block_len,
-						  const u64 ram_bytes,
-						  const int type,
-						  struct btrfs_file_extent *file_extent)
+						  struct btrfs_file_extent *file_extent,
+						  const int type)
 {
 	struct extent_map *em = NULL;
 	struct btrfs_ordered_extent *ordered;
@@ -6991,7 +6988,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
 		if (em) {
 			free_extent_map(em);
 			btrfs_drop_extent_map_range(inode, start,
-						    start + len - 1, false);
+					start + file_extent->num_bytes - 1, false);
 		}
 		em = ERR_CAST(ordered);
 	} else {
@@ -7034,10 +7031,9 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
 	file_extent.ram_bytes = ins.offset;
 	file_extent.offset = 0;
 	file_extent.compression = BTRFS_COMPRESS_NONE;
-	em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset,
-				     ins.offset,
-				     ins.offset, BTRFS_ORDERED_REGULAR,
-				     &file_extent);
+	em = btrfs_create_dio_extent(inode, dio_data, start,
+				     &file_extent,
+				     BTRFS_ORDERED_REGULAR);
 	btrfs_dec_block_group_reservations(fs_info, ins.objectid);
 	if (IS_ERR(em))
 		btrfs_free_reserved_extent(fs_info, ins.objectid, ins.offset,
@@ -7404,10 +7400,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
 		}
 		space_reserved = true;
 
-		em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
-					      file_extent.disk_num_bytes,
-					      file_extent.ram_bytes, type,
-					      &file_extent);
+		em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start,
+					      &file_extent, type);
 		btrfs_dec_nocow_writers(bg);
 		if (type == BTRFS_ORDERED_PREALLOC) {
 			free_extent_map(em);
Signature · 1 line
2.45.0
#34 lore raw

Re: [PATCH v2 11/11] btrfs: cleanup duplicated parameters related to btrfs_create_dio_extent()

Quoted 46 lines · Qu Wenruo

The following 3 parameters can be cleaned up using btrfs_file_extent structure:

- len
btrfs_file_extent::num_bytes

- orig_block_len
btrfs_file_extent::disk_num_bytes

- ram_bytes
btrfs_file_extent::ram_bytes

---
 fs/btrfs/inode.c | 22 ++++++++--------------
 1 file changed, 8 insertions(+), 14 deletions(-)
fs/btrfs/inode.c +3 -6
index a95dc2333972..09974c86d3d1 100644
@@ -6969,11 +6969,8 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
 static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                                                  struct btrfs_dio_data *dio_data,
                                                  const u64 start,
-                                                 const u64 len,
-                                                 const u64 orig_block_len,
-                                                 const u64 ram_bytes,
-                                                 const int type,
-                                                 struct btrfs_file_extent *file_extent)
+                                                 struct btrfs_file_extent *file_extent,
+                                                 const int type)
 {
        struct extent_map *em = NULL;
        struct btrfs_ordered_extent *ordered;
@@ -6991,7 +6988,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                if (em) {
                        free_extent_map(em);
                        btrfs_drop_extent_map_range(inode, start,
-                                                   start + len - 1, false);
+                                       start + file_extent->num_bytes - 1, false);
                }
                em = ERR_CAST(ordered);
        } else {

fs/btrfs/inode.c static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode, · line 7036

Quoted 10 lines
fs/btrfs/inode.c +3 -4
@@ -7034,10 +7031,9 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
        file_extent.ram_bytes = ins.offset;
        file_extent.offset = 0;
        file_extent.compression = BTRFS_COMPRESS_NONE;
-       em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset,
-                                    ins.offset,
-                                    ins.offset, BTRFS_ORDERED_REGULAR,
-                                    &file_extent);
+       em = btrfs_create_dio_extent(inode, dio_data, start,
+                                    &file_extent,
+                                    BTRFS_ORDERED_REGULAR);

As we're changing this, we can leave this in a single line as it fits.

fs/btrfs/inode.c static int btrfs_get_blocks_direct_write(struct extent_map **map, · line 7404

Quoted 13 lines
fs/btrfs/inode.c +2 -4
        btrfs_dec_block_group_reservations(fs_info, ins.objectid);
        if (IS_ERR(em))
                btrfs_free_reserved_extent(fs_info, ins.objectid, ins.offset,
@@ -7404,10 +7400,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                }
                space_reserved = true;
 
-               em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
-                                             file_extent.disk_num_bytes,
-                                             file_extent.ram_bytes, type,
-                                             &file_extent);
+               em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start,
+                                             &file_extent, type);

Same here.

The rest looks good, thanks.

Quoted 7 lines
fs/btrfs/inode.c +0 -1
                btrfs_dec_nocow_writers(bg);
                if (type == BTRFS_ORDERED_PREALLOC) {
                        free_extent_map(em);
--

2.45.0

#35 lore raw

Re: [PATCH v2 11/11] btrfs: cleanup duplicated parameters related to btrfs_create_dio_extent()

Quoted 67 lines · Filipe Manana
Quoted 46 lines · Qu Wenruo

The following 3 parameters can be cleaned up using btrfs_file_extent structure:

- len
btrfs_file_extent::num_bytes

- orig_block_len
btrfs_file_extent::disk_num_bytes

- ram_bytes
btrfs_file_extent::ram_bytes

---
  fs/btrfs/inode.c | 22 ++++++++--------------
  1 file changed, 8 insertions(+), 14 deletions(-)
fs/btrfs/inode.c +3 -6
index a95dc2333972..09974c86d3d1 100644
@@ -6969,11 +6969,8 @@ struct extent_map *btrfs_get_extent(struct btrfs_inode *inode,
  static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                                                   struct btrfs_dio_data *dio_data,
                                                   const u64 start,
-                                                 const u64 len,
-                                                 const u64 orig_block_len,
-                                                 const u64 ram_bytes,
-                                                 const int type,
-                                                 struct btrfs_file_extent *file_extent)
+                                                 struct btrfs_file_extent *file_extent,
+                                                 const int type)
  {
         struct extent_map *em = NULL;
         struct btrfs_ordered_extent *ordered;
@@ -6991,7 +6988,7 @@ static struct extent_map *btrfs_create_dio_extent(struct btrfs_inode *inode,
                 if (em) {
                         free_extent_map(em);
                         btrfs_drop_extent_map_range(inode, start,
-                                                   start + len - 1, false);
+                                       start + file_extent->num_bytes - 1, false);
                 }
                 em = ERR_CAST(ordered);
         } else {

fs/btrfs/inode.c static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode, · line 7036

Quoted 10 lines
fs/btrfs/inode.c +3 -4
@@ -7034,10 +7031,9 @@ static struct extent_map *btrfs_new_extent_direct(struct btrfs_inode *inode,
         file_extent.ram_bytes = ins.offset;
         file_extent.offset = 0;
         file_extent.compression = BTRFS_COMPRESS_NONE;
-       em = btrfs_create_dio_extent(inode, dio_data, start, ins.offset,
-                                    ins.offset,
-                                    ins.offset, BTRFS_ORDERED_REGULAR,
-                                    &file_extent);
+       em = btrfs_create_dio_extent(inode, dio_data, start,
+                                    &file_extent,
+                                    BTRFS_ORDERED_REGULAR);

As we're changing this, we can leave this in a single line as it fits.

Quoted 7 lines
fs/btrfs/inode.c +0 -0
         btrfs_dec_block_group_reservations(fs_info, ins.objectid);
         if (IS_ERR(em))
                 btrfs_free_reserved_extent(fs_info, ins.objectid, ins.offset,
@@ -7404,10 +7400,8 @@ static int btrfs_get_blocks_direct_write(struct extent_map **map,
                 }
                 space_reserved = true;
 
Quoted 8 lines

fs/btrfs/inode.c static int btrfs_get_blocks_direct_write(struct extent_map **map, · line 7404

Quoted 6 lines
fs/btrfs/inode.c +2 -4
-               em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start, len,
-                                             file_extent.disk_num_bytes,
-                                             file_extent.ram_bytes, type,
-                                             &file_extent);
+               em2 = btrfs_create_dio_extent(BTRFS_I(inode), dio_data, start,
+                                             &file_extent, type);

Same here.

Just a small question related to the single line one.

The parameter @start with its tailing ',' is already at 80 chars, do we still need to follow the old 80 chars width recommendation?

With previous several patches, I re-checked the lines, some can indeed be improved a little, but some BTRFS_ORDERED_* flags can not be merged without exceeding the 80 chars limits.

Thanks,
Qu

Quoted 11 lines

The rest looks good, thanks.

Quoted 7 lines
fs/btrfs/inode.c +0 -1
                 btrfs_dec_nocow_writers(bg);
                 if (type == BTRFS_ORDERED_PREALLOC) {
                         free_extent_map(em);
--

2.45.0

#36 lore raw
In reply to Qu Wenruo
Quoted 8 lines · Qu Wenruo

[CHANGELOG]
v2:
- Rebased to the latest for-next
There is a conflicts with extent locking, and maybe some other hidden conflicts for NOCOW/PREALLOC?
As previously the patchset passes fstests auto group, but after the merging with other patches, it always crashes as btrfs/060.

Quoted 8 lines

- Fix an error in the final cleanup patch
It's the NOCOW/PREALLOC shenanigans again, in the buffered NOCOW path, that we have to use the old inaccurate numbers for NOCOW/PREALLOC OEs.

- Split the final cleanup into 4 patches
Most cleanups are very straightforward, but the cleanup for btrfs_alloc_ordered_extent() needs extra special handling for NOCOW/PREALLOC.

Regarding the merge target, it's too close to 6.10 merge window freeze so this series is scheduled for 6.11.

#37 lore raw
In reply to Qu Wenruo
Quoted 35 lines · Qu Wenruo

[CHANGELOG]
v2:
- Rebased to the latest for-next
There is a conflicts with extent locking, and maybe some other hidden conflicts for NOCOW/PREALLOC?
As previously the patchset passes fstests auto group, but after the merging with other patches, it always crashes as btrfs/060.

- Fix an error in the final cleanup patch
It's the NOCOW/PREALLOC shenanigans again, in the buffered NOCOW path, that we have to use the old inaccurate numbers for NOCOW/PREALLOC OEs.

- Split the final cleanup into 4 patches
Most cleanups are very straightforward, but the cleanup for btrfs_alloc_ordered_extent() needs extra special handling for NOCOW/PREALLOC.

v1:
- Rebased to the latest for-next
To resolve the conflicts with the recently introduced extent map shrinker

- A new cleanup patch to remove the recursive header inclusion

- Use a new structure to pass the file extent item related members around

- Add a new comment on why we're intentionally passing incorrect numbers for NOCOW/PREALLOC ordered extents inside btrfs_create_dio_extent()

[REPO]
https://github.com/adam900710/linux/tree/em_cleanup

Quoted 7 lines

This series introduce two new members (disk_bytenr/offset) to extent_map, and removes three old members (block_start/block_len/offset), finally rename one member (orig_block_len -> disk_num_bytes).

This should save us one u64 for extent_map, although with the recent extent map shrinker, the saving is not that useful.

The shrinker doesn't invalidate or make this patchset less useful.

It's always good to reduce the size of a structure like this for which we can easily have millions of instances, it reduces the number of pages we consume.

Things are a bit hard to review here, because a lot of code is added and then removed later and fields at a time, so a lot of cross reference checks are needed.
Changing the approach here would be a lot of work, and probably would be more bike shedding than anything else.

But it looks fine, and all the comments on the individual patches are minor, except for a bug in patch 8/11.

Thanks!

Quoted 72 lines

But to make things safe to migrate, I introduce extra sanity checks for extent_map, and do cross check for both old and new members.

The extra sanity checks already exposed one bug (thankfully harmless) causing em::block_start to be incorrect.

But so far, the patchset is fine for default fstests run.

Furthermore, since we're already having too long parameter lists for extent_map/ordered_extent/can_nocow_extent, here is a new structure, btrfs_file_extent, a memory-access-friendly structure to represent a btrfs_file_extent_item.

With the help of that structure, we can use that to represent a file extent item without a super long parameter list.

The patchset would rename orig_block_len to disk_num_bytes first. Then introduce the new member, the extra sanity checks, and introduce the new btrfs_file_extent structure and use that to remove the older 3 members from extent_map.

After all above works done, use btrfs_file_extent to further cleanup can_nocow_file_extent_args()/btrfs_alloc_ordered_extent()/create_io_em()/ btrfs_create_dio_extent().

The cleanup is in fact pretty tricky, the current code base never expects correct numbers for NOCOW/PREALLOC OEs, thus we have to keep the old but incorrect numbers just for NOCOW/PREALLOC.

I will address the NOCOW/PREALLOC shenanigans the future, but after the huge cleanup across multiple core structures.

Qu Wenruo (11):
btrfs: rename extent_map::orig_block_len to disk_num_bytes
btrfs: export the expected file extent through can_nocow_extent()
btrfs: introduce new members for extent_map
btrfs: introduce extra sanity checks for extent maps
btrfs: remove extent_map::orig_start member
btrfs: remove extent_map::block_len member
btrfs: remove extent_map::block_start member
btrfs: cleanup duplicated parameters related to
can_nocow_file_extent_args
btrfs: cleanup duplicated parameters related to
btrfs_alloc_ordered_extent
btrfs: cleanup duplicated parameters related to create_io_em()
btrfs: cleanup duplicated parameters related to
btrfs_create_dio_extent()

 fs/btrfs/btrfs_inode.h            |   4 +-
 fs/btrfs/compression.c            |   7 +-
 fs/btrfs/defrag.c                 |  14 +-
 fs/btrfs/extent_io.c              |  10 +-
 fs/btrfs/extent_map.c             | 187 ++++++++++++------
 fs/btrfs/extent_map.h             |  51 +++--
 fs/btrfs/file-item.c              |  23 +--
 fs/btrfs/file.c                   |  18 +-
 fs/btrfs/inode.c                  | 308 +++++++++++++-----------------
 fs/btrfs/ordered-data.c           |  36 +++-
 fs/btrfs/ordered-data.h           |  22 ++-
 fs/btrfs/relocation.c             |   5 +-
 fs/btrfs/tests/extent-map-tests.c | 114 ++++++-----
 fs/btrfs/tests/inode-tests.c      | 177 ++++++++---------
 fs/btrfs/tree-log.c               |  25 +--
 fs/btrfs/zoned.c                  |   4 +-
 include/trace/events/btrfs.h      |  26 +--
 17 files changed, 548 insertions(+), 483 deletions(-)

--
2.45.0