Skip to contentSkip to conversation

Physical address offset of the inline extent

DISCUSSION Rebraca Dejan (BSOT/PJ-ES1-Bg) 3 participants 6 messages Jun 18, 2020 – Jun 19, 2020 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hi,

Does anybody know how to get an address offset of the file data inline extent?
Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:

In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).

I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.

Thanks very much,

Dejan

#1 lore raw
Quoted 3 lines · On 2020/6/18 下午9:21, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:

Hi,

Does anybody know how to get an address offset of the file data inline extent?

Since it's inline, it's inlined into metadata (tree block), thus makes no sense to get the "address".

The address will be inside a tree block anyway.

If you really want the address, you can check it with dump-tree.

$ btrfs ins dump-tree -t 5 /dev/nvme/btrfs
fs tree key (FS_TREE ROOT_ITEM 0)
leaf 5324800 items 7 free space 13619 generation 6 owner FS_TREE
leaf 5324800 flags 0x1(WRITTEN) backref revision 1
     ^^^^^^^ Tree block bytenr
...
        item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
                generation 6 type 0 (inline)
                inline extent data size 2048 ram_bytes 2048 compression
0 (none)

You need to use extent_buffer to grab the inline file extent item, and use the btrfs_file_extent_inline_start() to get the inline file extent offset inside the leaf.

Anyway, you need a solid understanding of btrfs on-disk format to grasp the above dump.

Quoted 5 lines

Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:

In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).

I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.

Key.objectid of EXTENT_ITEM only means the inode number. Key.offset of EXTENT_ITEM means the file offset, for inline file extent it's always 0.

So it looks like you're not familiar with btrfs on-disk format, thus I doubt the usefulness of known the inline file extent offset inside the tree block.

Thanks,
Qu

Quoted 7 lines

Thanks very much,

Dejan

#2 lore raw

Hi Qu,

I've read this:
https://btrfs.wiki.kernel.org/index.php/Data_Structures#btrfs_key_.2F_btrfs_disk_key
where key.objectid of EXTENT_ITEM contains the starting byte offset of the extent it describes and key.offset of EXTENT_ITEM stores the size of the extent the item describes.

From your dump:
item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069

itremoff = 13794
This is the offset relative to tree block or tree leaf, right?

Thanks,
Dejan

-----Original Message-----
From: Qu Wenruo <[email protected]>
Sent: četvrtak, 18. jun 2020. 15:46
To: Rebraca Dejan (BSOT/PJ-ES1-Bg) <[email protected]>; [email protected]
Subject: Re: Physical address offset of the inline extent

Quoted 3 lines · On 2020/6/18 下午9:21, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:

Hi,

Does anybody know how to get an address offset of the file data inline extent?

Since it's inline, it's inlined into metadata (tree block), thus makes no sense to get the "address".

The address will be inside a tree block anyway.

If you really want the address, you can check it with dump-tree.

$ btrfs ins dump-tree -t 5 /dev/nvme/btrfs fs tree key (FS_TREE ROOT_ITEM 0) leaf 5324800 items 7 free space 13619 generation 6 owner FS_TREE leaf 5324800 flags 0x1(WRITTEN) backref revision 1
     ^^^^^^^ Tree block bytenr
...
        item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
                generation 6 type 0 (inline)
                inline extent data size 2048 ram_bytes 2048 compression
0 (none)

You need to use extent_buffer to grab the inline file extent item, and use the btrfs_file_extent_inline_start() to get the inline file extent offset inside the leaf.

Anyway, you need a solid understanding of btrfs on-disk format to grasp the above dump.

Quoted 5 lines

Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:

In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).

I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.

Key.objectid of EXTENT_ITEM only means the inode number.
Key.offset of EXTENT_ITEM means the file offset, for inline file extent it's always 0.

So it looks like you're not familiar with btrfs on-disk format, thus I doubt the usefulness of known the inline file extent offset inside the tree block.

Thanks,
Qu

Quoted 7 lines

Thanks very much,

Dejan

#3 lore raw
Quoted 6 lines · On 18.06.20 г. 17:53 ч., Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:

Hi Qu,

I've read this:
https://btrfs.wiki.kernel.org/index.php/Data_Structures#btrfs_key_.2F_btrfs_disk_key
where key.objectid of EXTENT_ITEM contains the starting byte offset of the extent it describes and key.offset of EXTENT_ITEM stores the size of the extent the item describes.

Quoted 6 lines

From your dump:
item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069

itremoff = 13794
This is the offset relative to tree block or tree leaf, right?

This is the offset relative to the tree leaf block, which is 5324800. HOwever this address 5324800 is within btrfs' logical address space and not physical space. Then you need to map this logical address to a particular btrfs block group which then has to be mapped to a chunk, which then has to be mapped to a physical device extent to eventually arrive at the physical location of this data.

Before proceeding with this I strongly advise you spend time getting familiar with
https://github.com/btrfs/btrfs-dev-docs/blob/master/tree-items.txt and
https://github.com/btrfs/btrfs-dev-docs/blob/master/trees.txt

<snip>

#4 lore raw
In reply to Rebraca Dejan (BSOT/PJ-ES1-Bg)
Quoted 11 lines · On 2020/6/18 下午10:53, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:

Hi Qu,

I've read this:
https://btrfs.wiki.kernel.org/index.php/Data_Structures#btrfs_key_.2F_btrfs_disk_key
where key.objectid of EXTENT_ITEM contains the starting byte offset of the extent it describes and key.offset of EXTENT_ITEM stores the size of the extent the item describes.

From your dump:
item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069

itremoff = 13794
This is the offset relative to tree block or tree leaf, right?

Right, but you still need to consider the offset inside btrfs_file_extent_item for inline data, that's why I'm using btrfs_file_extent_inline_start() to find out the real offset.

Thanks,
Qu

Quoted 58 lines

Thanks,
Dejan

-----Original Message-----
From: Qu Wenruo <[email protected]>
Sent: četvrtak, 18. jun 2020. 15:46
To: Rebraca Dejan (BSOT/PJ-ES1-Bg) <[email protected]>; [email protected]
Subject: Re: Physical address offset of the inline extent

Quoted 3 lines · On 2020/6/18 下午9:21, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:

Hi,

Does anybody know how to get an address offset of the file data inline extent?

Since it's inline, it's inlined into metadata (tree block), thus makes no sense to get the "address".

The address will be inside a tree block anyway.

If you really want the address, you can check it with dump-tree.

$ btrfs ins dump-tree -t 5 /dev/nvme/btrfs fs tree key (FS_TREE ROOT_ITEM 0) leaf 5324800 items 7 free space 13619 generation 6 owner FS_TREE leaf 5324800 flags 0x1(WRITTEN) backref revision 1
     ^^^^^^^ Tree block bytenr
...
        item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
                generation 6 type 0 (inline)
                inline extent data size 2048 ram_bytes 2048 compression
0 (none)

You need to use extent_buffer to grab the inline file extent item, and use the btrfs_file_extent_inline_start() to get the inline file extent offset inside the leaf.

Anyway, you need a solid understanding of btrfs on-disk format to grasp the above dump.

Quoted 5 lines

Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:

In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).

I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.

Key.objectid of EXTENT_ITEM only means the inode number.
Key.offset of EXTENT_ITEM means the file offset, for inline file extent it's always 0.

So it looks like you're not familiar with btrfs on-disk format, thus I doubt the usefulness of known the inline file extent offset inside the tree block.

Thanks,
Qu

Quoted 7 lines

Thanks very much,

Dejan

#5 lore raw

Thanks a lot for your help guys.
As I thought, it's not so straightforward, I'll see what I can do.

Thanks,
Dejan

-----Original Message-----
From: Qu Wenruo <[email protected]>
Sent: petak, 19. jun 2020. 00:49
To: Rebraca Dejan (BSOT/PJ-ES1-Bg) <[email protected]>; [email protected]
Subject: Re: Physical address offset of the inline extent

Quoted 7 lines · On 2020/6/18 下午10:53, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:

Hi Qu,

I've read this:
https://btrfs.wiki.kernel.org/index.php/Data_Structures#btrfs_key_.2F_
btrfs_disk_key where key.objectid of EXTENT_ITEM contains the starting
byte offset of the extent it describes and key.offset of EXTENT_ITEM stores the size of the extent the item describes.

Quoted 5 lines

From your dump:
item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069

itremoff = 13794
This is the offset relative to tree block or tree leaf, right?

Right, but you still need to consider the offset inside btrfs_file_extent_item for inline data, that's why I'm using btrfs_file_extent_inline_start() to find out the real offset.

Thanks,
Qu

Quoted 60 lines

Thanks,
Dejan

-----Original Message-----
From: Qu Wenruo <[email protected]>
Sent: četvrtak, 18. jun 2020. 15:46
To: Rebraca Dejan (BSOT/PJ-ES1-Bg) <[email protected]>; [email protected]
Subject: Re: Physical address offset of the inline extent

Quoted 3 lines · On 2020/6/18 下午9:21, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:

Hi,

Does anybody know how to get an address offset of the file data inline extent?

Since it's inline, it's inlined into metadata (tree block), thus makes no sense to get the "address".

The address will be inside a tree block anyway.

If you really want the address, you can check it with dump-tree.

$ btrfs ins dump-tree -t 5 /dev/nvme/btrfs fs tree key (FS_TREE ROOT_ITEM 0) leaf 5324800 items 7 free space 13619 generation 6 owner FS_TREE leaf 5324800 flags 0x1(WRITTEN) backref revision 1
     ^^^^^^^ Tree block bytenr
...
        item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
                generation 6 type 0 (inline)
                inline extent data size 2048 ram_bytes 2048 
compression
0 (none)

You need to use extent_buffer to grab the inline file extent item, and use the btrfs_file_extent_inline_start() to get the inline file extent offset inside the leaf.

Anyway, you need a solid understanding of btrfs on-disk format to grasp the above dump.

Quoted 5 lines

Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:

In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).

I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.

Key.objectid of EXTENT_ITEM only means the inode number.
Key.offset of EXTENT_ITEM means the file offset, for inline file extent it's always 0.

So it looks like you're not familiar with btrfs on-disk format, thus I doubt the usefulness of known the inline file extent offset inside the tree block.

Thanks,
Qu

Quoted 7 lines

Thanks very much,

Dejan