Skip to contentSkip to conversation

Btrfs: fix physical offset reported by fiemap for inline extents

PATCH Superseded [email protected] 4 participants 6 messages Jun 19, 2018 – Jun 25, 2018 Btrfs file system

lore
Share & tools
lore

[email protected]

author
#0 lore raw

PATCH 1/1 Btrfs: fix physical offset reported by fiemap for inline extents

From: Filipe Manana <[email protected]>

Commit 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero") introduced a regression where we no longer report 0 as the physical offset for inline extents. This is because it always sets the variable used to report the physical offset ("disko") as em->block_start plus some offset, and em->block_start has the value 18446744073709551614 ((u64) -2) for inline extents.

This made the btrfs test 004 (from fstests) often fail, for example, for a file with an inline extent we have the following items in the subvolume tree:

    item 101 key (418 INODE_ITEM 0) itemoff 11029 itemsize 160
           generation 25 transid 38 size 1525 nbytes 1525
           block group 0 mode 100666 links 1 uid 0 gid 0 rdev 0
           sequence 0 flags 0x2(none)
           atime 1529342058.461891730 (2018-06-18 18:14:18)
           ctime 1529342058.461891730 (2018-06-18 18:14:18)
           mtime 1529342058.461891730 (2018-06-18 18:14:18)
           otime 1529342055.869892885 (2018-06-18 18:14:15)
    item 102 key (418 INODE_REF 264) itemoff 11016 itemsize 13
           index 25 namelen 3 name: fc7
    item 103 key (418 EXTENT_DATA 0) itemoff 9470 itemsize 1546
           generation 38 type 0 (inline)
           inline extent data size 1525 ram_bytes 1525 compression 0 (none)

Then when test 004 invoked fiemap against the file it got a non-zero physical offset:

 $ filefrag -v /mnt/p0/d4/d7/fc7
 Filesystem type is: 9123683e
 File size of /mnt/p0/d4/d7/fc7 is 1525 (1 block of 4096 bytes)
  ext:     logical_offset:        physical_offset: length:   expected: flags:
    0:        0..    4095: 18446744073709551614..      4093:   4096:             last,not_aligned,inline,eof
 /mnt/p0/d4/d7/fc7: 1 extent found

This resulted in the test failing like this:

btrfs/004 49s ... [failed, exit status 1]- output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad)

    --- tests/btrfs/004.out	2016-08-23 10:17:35.027012095 +0100
    +++ /home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad	2018-06-18 18:15:02.385872155 +0100
    @@ -1,3 +1,10 @@
     QA output created by 004
     *** test backref walking
    -*** done
    +./tests/btrfs/004: line 227: [: 7.55578637259143e+22: integer expression expected
    +ERROR: 7.55578637259143e+22 is not a valid numeric value.
    +unexpected output from
    +	/home/fdmanana/git/hub/btrfs-progs/btrfs inspect-internal logical-resolve -s 65536 -P 7.55578637259143e+22 /home/fdmanana/btrfs-tests/scratch_1
    ...
    (Run 'diff -u tests/btrfs/004.out /home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad'  to see the entire diff)

Ran: btrfs/004

The large number in scientific notation reported as an invalid numeric value is the result from the filter passed to perl which multiplies the physical offset by the block size reported by fiemap.

So fix this by ensuring the physical offset is always set to 0 when we are processing an inline extent.

Fixes 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero")
Signed-off-by Filipe Manana <[email protected]>
---
 fs/btrfs/extent_io.c | 1 +
 1 file changed, 1 insertion(+)
fs/btrfs/extent_io.c +1 -0
index 8e4a7cdbc9f5..978327d98fc5 100644
@@ -4559,6 +4559,7 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			end = 1;
 			flags |= FIEMAP_EXTENT_LAST;
 		} else if (em->block_start == EXTENT_MAP_INLINE) {
+			disko = 0;
 			flags |= (FIEMAP_EXTENT_DATA_INLINE |
 				  FIEMAP_EXTENT_NOT_ALIGNED);
 		} else if (em->block_start == EXTENT_MAP_DELALLOC) {
Signature · 1 line
2.11.0
#1 lore raw
Quoted 6 lines · On Tue, Jun 19, 2018 at 12:31:42PM +0100, [email protected] wrote:

From: Filipe Manana <[email protected]>
So fix this by ensuring the physical offset is always set to 0 when we are processing an inline extent.

Fixes 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero")
Signed-off-by Filipe Manana <[email protected]>

Added to 4.18 queue, thanks.

This is a fix for a patch that was in for-next for a few weeks but the bug was discovered only after the patch got merged to master. I wonder if there's something to be improved in the patch flow.

I think it should take less time to catch bugs while the patches are unmerged, either in the mailinglist or in the development branches. Post-merge fixes will happen of course, but in this particular case it looks like something that slipped too easily. I did the "fallback" review and checked that tests regarding fiemap pass, the occasional failure you mention has not happened on any of my testing setups.

There's another patch for fiemap that seems to have bigger impact and for that reason I have postponed merging it to 4.18 unlike the first one, but now I think this requires a testcase.

https://patchwork.kernel.org/patch/10383491/

The patch will be in for-next topic branch until then.

#2 lore raw
In reply to [email protected]

[email protected] 於 2018-06-19 19:31 寫到:

Quoted 91 lines

From: Filipe Manana <[email protected]>

Commit 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero") introduced a regression where we no longer report 0 as the physical offset for inline extents. This is because it always sets the variable used to report the physical offset ("disko") as em->block_start plus some offset, and em->block_start has the value 18446744073709551614 ((u64) -2) for inline extents.

This made the btrfs test 004 (from fstests) often fail, for example, for a file with an inline extent we have the following items in the subvolume tree:

    item 101 key (418 INODE_ITEM 0) itemoff 11029 itemsize 160
           generation 25 transid 38 size 1525 nbytes 1525
           block group 0 mode 100666 links 1 uid 0 gid 0 rdev 0
           sequence 0 flags 0x2(none)
           atime 1529342058.461891730 (2018-06-18 18:14:18)
           ctime 1529342058.461891730 (2018-06-18 18:14:18)
           mtime 1529342058.461891730 (2018-06-18 18:14:18)
           otime 1529342055.869892885 (2018-06-18 18:14:15)
    item 102 key (418 INODE_REF 264) itemoff 11016 itemsize 13
           index 25 namelen 3 name: fc7
    item 103 key (418 EXTENT_DATA 0) itemoff 9470 itemsize 1546
           generation 38 type 0 (inline)
           inline extent data size 1525 ram_bytes 1525 compression 0

(none)

Then when test 004 invoked fiemap against the file it got a non-zero physical offset:

 $ filefrag -v /mnt/p0/d4/d7/fc7
 Filesystem type is: 9123683e
 File size of /mnt/p0/d4/d7/fc7 is 1525 (1 block of 4096 bytes)
  ext:     logical_offset:        physical_offset: length:   expected: 
flags:
    0:        0..    4095: 18446744073709551614..      4093:   4096:
          last,not_aligned,inline,eof
 /mnt/p0/d4/d7/fc7: 1 extent found

This resulted in the test failing like this:

btrfs/004 49s ... [failed, exit status 1]- output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad)

    --- tests/btrfs/004.out	2016-08-23 10:17:35.027012095 +0100
    +++
/home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad	2018-06-18
18:15:02.385872155 +0100
    @@ -1,3 +1,10 @@
     QA output created by 004
     *** test backref walking
    -*** done
    +./tests/btrfs/004: line 227: [: 7.55578637259143e+22: integer

expression expected
+ERROR: 7.55578637259143e+22 is not a valid numeric value. +unexpected output from
+ /home/fdmanana/git/hub/btrfs-progs/btrfs inspect-internal
logical-resolve -s 65536 -P 7.55578637259143e+22 /home/fdmanana/btrfs-tests/scratch_1

    ...
    (Run 'diff -u tests/btrfs/004.out

/home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad' to see the entire diff)
Ran: btrfs/004

The large number in scientific notation reported as an invalid numeric value is the result from the filter passed to perl which multiplies the physical offset by the block size reported by fiemap.

So fix this by ensuring the physical offset is always set to 0 when we are processing an inline extent.

Fixes 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when

fm_extent_count is zero")

Signed-off-by Filipe Manana <[email protected]>
---
 fs/btrfs/extent_io.c | 1 +
 1 file changed, 1 insertion(+)
fs/btrfs/extent_io.c +1 -0
index 8e4a7cdbc9f5..978327d98fc5 100644
@@ -4559,6 +4559,7 @@ int extent_fiemap(struct inode *inode, struct
 fiemap_extent_info *fieinfo,
 			end = 1;
 			flags |= FIEMAP_EXTENT_LAST;
 		} else if (em->block_start == EXTENT_MAP_INLINE) {
+			disko = 0;
 			flags |= (FIEMAP_EXTENT_DATA_INLINE |
 				  FIEMAP_EXTENT_NOT_ALIGNED);
Quoted 1 line

} else if (em->block_start == EXTENT_MAP_DELALLOC) {

EXTENT_MAP_DELALLOC should have the same problem.

em->block_start has some special values. The following values ​​should not be considered disko #define EXTENT_MAP_LAST_BYTE((u64)-4)
#define EXTENT_MAP_HOLE((u64)-3)
#define EXTENT_MAP_INLINE((u64)-2)
#define EXTENT_MAP_DELALLOC((u64)-1)

Is the following change more suitable?
if (em->block_start >= EXTENT_MAP_LAST_BYTE)

    disko = 0;
else
    disko = em->block_start + offset_in_extent;

Thanks.
Robbie Ko

author
#3 lore raw
Quoted 106 lines · robbieko

[email protected] 於 2018-06-19 19:31 寫到:

Quoted 91 lines

From: Filipe Manana <[email protected]>

Commit 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero") introduced a regression where we no longer report 0 as the physical offset for inline extents. This is because it always sets the variable used to report the physical offset ("disko") as em->block_start plus some offset, and em->block_start has the value 18446744073709551614 ((u64) -2) for inline extents.

This made the btrfs test 004 (from fstests) often fail, for example, for a file with an inline extent we have the following items in the subvolume tree:

    item 101 key (418 INODE_ITEM 0) itemoff 11029 itemsize 160
           generation 25 transid 38 size 1525 nbytes 1525
           block group 0 mode 100666 links 1 uid 0 gid 0 rdev 0
           sequence 0 flags 0x2(none)
           atime 1529342058.461891730 (2018-06-18 18:14:18)
           ctime 1529342058.461891730 (2018-06-18 18:14:18)
           mtime 1529342058.461891730 (2018-06-18 18:14:18)
           otime 1529342055.869892885 (2018-06-18 18:14:15)
    item 102 key (418 INODE_REF 264) itemoff 11016 itemsize 13
           index 25 namelen 3 name: fc7
    item 103 key (418 EXTENT_DATA 0) itemoff 9470 itemsize 1546
           generation 38 type 0 (inline)
           inline extent data size 1525 ram_bytes 1525 compression 0

(none)

Then when test 004 invoked fiemap against the file it got a non-zero physical offset:

 $ filefrag -v /mnt/p0/d4/d7/fc7
 Filesystem type is: 9123683e
 File size of /mnt/p0/d4/d7/fc7 is 1525 (1 block of 4096 bytes)
  ext:     logical_offset:        physical_offset: length:   expected:
flags:
    0:        0..    4095: 18446744073709551614..      4093:   4096:
          last,not_aligned,inline,eof
 /mnt/p0/d4/d7/fc7: 1 extent found

This resulted in the test failing like this:

btrfs/004 49s ... [failed, exit status 1]- output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad)

    --- tests/btrfs/004.out     2016-08-23 10:17:35.027012095 +0100
    +++
/home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad      2018-06-18
18:15:02.385872155 +0100
    @@ -1,3 +1,10 @@
     QA output created by 004
     *** test backref walking
    -*** done
    +./tests/btrfs/004: line 227: [: 7.55578637259143e+22: integer

expression expected
+ERROR: 7.55578637259143e+22 is not a valid numeric value. +unexpected output from
+ /home/fdmanana/git/hub/btrfs-progs/btrfs inspect-internal
logical-resolve -s 65536 -P 7.55578637259143e+22 /home/fdmanana/btrfs-tests/scratch_1

    ...
    (Run 'diff -u tests/btrfs/004.out

/home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad' to see the entire diff)
Ran: btrfs/004

The large number in scientific notation reported as an invalid numeric value is the result from the filter passed to perl which multiplies the physical offset by the block size reported by fiemap.

So fix this by ensuring the physical offset is always set to 0 when we are processing an inline extent.

Fixes 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when

fm_extent_count is zero")

Signed-off-by Filipe Manana <[email protected]>
---
 fs/btrfs/extent_io.c | 1 +
 1 file changed, 1 insertion(+)
fs/btrfs/extent_io.c +1 -0
index 8e4a7cdbc9f5..978327d98fc5 100644
@@ -4559,6 +4559,7 @@ int extent_fiemap(struct inode *inode, struct
 fiemap_extent_info *fieinfo,
                        end = 1;
                        flags |= FIEMAP_EXTENT_LAST;
                } else if (em->block_start == EXTENT_MAP_INLINE) {
+                       disko = 0;
                        flags |= (FIEMAP_EXTENT_DATA_INLINE |
                                  FIEMAP_EXTENT_NOT_ALIGNED);
Quoted 1 line
                } else if (em->block_start == EXTENT_MAP_DELALLOC) {

EXTENT_MAP_DELALLOC should have the same problem.

em->block_start has some special values. The following values should not be considered disko
#define EXTENT_MAP_LAST_BYTE((u64)-4)
#define EXTENT_MAP_HOLE((u64)-3)
#define EXTENT_MAP_INLINE((u64)-2)
#define EXTENT_MAP_DELALLOC((u64)-1)

Quoted 5 lines

Is the following change more suitable?
if (em->block_start >= EXTENT_MAP_LAST_BYTE)

    disko = 0;
else
    disko = em->block_start + offset_in_extent;

Yes, I was thinking about it yesterday's evening regarding at least holes/delalloc and leaving it for today's morning after leaving some tests running during the evening.

Quoted 10 lines

Thanks.
Robbie Ko

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#4 lore raw

PATCH v2 1/1 Btrfs: fix physical offset reported by fiemap for inline extents

In reply to [email protected]

From: Filipe Manana <[email protected]>

Commit 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero") introduced a regression where we no longer report 0 as the physical offset for inline extents (and other extents with a special block_start value). This is because it always sets the variable used to report the physical offset ("disko") as em->block_start plus some offset, and em->block_start has the value 18446744073709551614 ((u64) -2) for inline extents.

This made the btrfs test 004 (from fstests) often fail, for example, for a file with an inline extent we have the following items in the subvolume tree:

    item 101 key (418 INODE_ITEM 0) itemoff 11029 itemsize 160
           generation 25 transid 38 size 1525 nbytes 1525
           block group 0 mode 100666 links 1 uid 0 gid 0 rdev 0
           sequence 0 flags 0x2(none)
           atime 1529342058.461891730 (2018-06-18 18:14:18)
           ctime 1529342058.461891730 (2018-06-18 18:14:18)
           mtime 1529342058.461891730 (2018-06-18 18:14:18)
           otime 1529342055.869892885 (2018-06-18 18:14:15)
    item 102 key (418 INODE_REF 264) itemoff 11016 itemsize 13
           index 25 namelen 3 name: fc7
    item 103 key (418 EXTENT_DATA 0) itemoff 9470 itemsize 1546
           generation 38 type 0 (inline)
           inline extent data size 1525 ram_bytes 1525 compression 0 (none)

Then when test 004 invoked fiemap against the file it got a non-zero physical offset:

 $ filefrag -v /mnt/p0/d4/d7/fc7
 Filesystem type is: 9123683e
 File size of /mnt/p0/d4/d7/fc7 is 1525 (1 block of 4096 bytes)
  ext:     logical_offset:        physical_offset: length:   expected: flags:
    0:        0..    4095: 18446744073709551614..      4093:   4096:             last,not_aligned,inline,eof
 /mnt/p0/d4/d7/fc7: 1 extent found

This resulted in the test failing like this:

btrfs/004 49s ... [failed, exit status 1]- output mismatch (see /home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad)

    --- tests/btrfs/004.out	2016-08-23 10:17:35.027012095 +0100
    +++ /home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad	2018-06-18 18:15:02.385872155 +0100
    @@ -1,3 +1,10 @@
     QA output created by 004
     *** test backref walking
    -*** done
    +./tests/btrfs/004: line 227: [: 7.55578637259143e+22: integer expression expected
    +ERROR: 7.55578637259143e+22 is not a valid numeric value.
    +unexpected output from
    +	/home/fdmanana/git/hub/btrfs-progs/btrfs inspect-internal logical-resolve -s 65536 -P 7.55578637259143e+22 /home/fdmanana/btrfs-tests/scratch_1
    ...
    (Run 'diff -u tests/btrfs/004.out /home/fdmanana/git/hub/xfstests/results//btrfs/004.out.bad'  to see the entire diff)

Ran: btrfs/004

The large number in scientific notation reported as an invalid numeric value is the result from the filter passed to perl which multiplies the physical offset by the block size reported by fiemap.

So fix this by ensuring the physical offset is always set to 0 when we are processing an extent with a special block_start value.

Fixes 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero")
Signed-off-by Filipe Manana <[email protected]>

---

v2: Set the physical offset to 0 for other extent maps with a special
block_start value as well.

 fs/btrfs/extent_io.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
fs/btrfs/extent_io.c +4 -1
index 8e4a7cdbc9f5..1aa91d57404a 100644
@@ -4545,8 +4545,11 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			offset_in_extent = em_start - em->start;
 		em_end = extent_map_end(em);
 		em_len = em_end - em_start;
-		disko = em->block_start + offset_in_extent;
 		flags = 0;
+		if (em->block_start < EXTENT_MAP_LAST_BYTE)
+			disko = em->block_start + offset_in_extent;
+		else
+			disko = 0;
 
 		/*
 		 * bump off for our next call to get_extent
Signature · 1 line
2.11.0
Reviewed-by on v2 1/1
#5 lore raw
Quoted 2 lines · On 20.06.2018 12:02, [email protected] wrote:

From: Filipe Manana <[email protected]>

<snip>

Quoted 5 lines

So fix this by ensuring the physical offset is always set to 0 when we are processing an extent with a special block_start value.

Fixes 9d311e11fc1f ("Btrfs: fiemap: pass correct bytenr when fm_extent_count is zero")
Signed-off-by Filipe Manana <[email protected]>

At first I was "wtf" since I had missed the u64 cast. After realising that the constants are in fact always parsed as positive numbers then it makes sense.

Reviewed-by Nikolay Borisov <[email protected]>
Quoted 26 lines

---

v2: Set the physical offset to 0 for other extent maps with a special
block_start value as well.

 fs/btrfs/extent_io.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
fs/btrfs/extent_io.c +4 -1
index 8e4a7cdbc9f5..1aa91d57404a 100644
@@ -4545,8 +4545,11 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 			offset_in_extent = em_start - em->start;
 		em_end = extent_map_end(em);
 		em_len = em_end - em_start;
-		disko = em->block_start + offset_in_extent;
 		flags = 0;
+		if (em->block_start < EXTENT_MAP_LAST_BYTE)
+			disko = em->block_start + offset_in_extent;
+		else
+			disko = 0;
 
 		/*
 		 * bump off for our next call to get_extent