design: document the revisions to the realtime rmap formats
PATCH 1/2 design: document the revisions to the realtime rmap formats
From: Darrick J. Wong <[email protected]>
Update the realtime rmap ondisk format documentation, since it's changed a bit since the introduction of the metadata directory tree.
Signed-off-by Darrick J. Wong <[email protected]>
--- .../internal_inodes.asciidoc | 4 .../journaling_log.asciidoc | 22 +- design/XFS_Filesystem_Structure/magic.asciidoc | 4 .../XFS_Filesystem_Structure/ondisk_inode.asciidoc | 1 design/XFS_Filesystem_Structure/realtime.asciidoc | 5 design/XFS_Filesystem_Structure/rtrmapbt.asciidoc | 215 ++++++++------------ 6 files changed, 115 insertions(+), 136 deletions(-)
design/XFS_Filesystem_Structure/internal_inodes.asciidoc +4 -0
@@ -26,6 +26,7 @@ of those inodes have been deallocated and may be reused by future features. | xref:Quota_Inodes[Project Quota] | /quota/project | xref:Real-Time_Bitmap_Inode[Realtime Bitmap] | /rtgroups/*.bitmap | xref:Real-Time_Summary_Inode[Realtime Summary] | /rtgroups/*.summary +| xref:Real_time_Reverse_Mapping_Btree[Realtime Reverse Mapping B+tree] | /rtgroups/*.rmap |===== Metadata files are flagged by the +XFS_DIFLAG2_METADATA+ flag in the
@@ -298,3 +299,6 @@ Checksum of the DQ block. There are two inodes allocated to managing the real-time device's space, the xref:Real-Time_Bitmap_Inode[Bitmap Inode] and the xref:Real-Time_Summary_Inode[Summary Inode]. + +Each realtime group can allocate one inode to managing a +xref:Real_time_Reverse_Mapping_Btree[reverse-index of space] usage.
design/XFS_Filesystem_Structure/journaling_log.asciidoc +13 -9
@@ -352,8 +352,9 @@ typedef struct xfs_efi_log_format { ---- *efi_type*:: -The signature of an EFI operation, 0x1236. This value is in host-endian order, -not big-endian like the rest of XFS. +The signature of an EFI operation, 0x1236. For a realtime extent, this +value will be 0x124a. This value is in host-endian order, not big-endian like +the rest of XFS. *efi_size*:: Size of this log item. Should be 1.
@@ -390,8 +391,9 @@ typedef struct xfs_efd_log_format { ---- *efd_type*:: -The signature of an EFD operation, 0x1237. This value is in host-endian order, -not big-endian like the rest of XFS. +The signature of an EFD operation, 0x1237. For a realtime EFD, this value will +be 0x124b. This value is in host-endian order, not big-endian like the rest of +XFS. *efd_size*:: Size of this log item. Should be 1.
@@ -478,14 +480,15 @@ struct xfs_rui_log_format { __uint16_t rui_type; __uint16_t rui_size; __uint32_t rui_nextents; - __uint64_t rui_id; + __uint64_t rui_id; struct xfs_map_extent rui_extents[1]; }; ---- *rui_type*:: -The signature of an RUI operation, 0x1240. This value is in host-endian order, -not big-endian like the rest of XFS. +The signature of an RUI operation, 0x1240. For a realtime RUI, this value will +be 0x124c. This value is in host-endian order, not big-endian like the rest of +XFS. *rui_size*:: Size of this log item. Should be 1.
@@ -519,8 +522,9 @@ struct xfs_rud_log_format { ---- *rud_type*:: -The signature of an RUD operation, 0x1241. This value is in host-endian order, -not big-endian like the rest of XFS. +The signature of an RUD operation, 0x1241. For a realtime RUD, this value will +be 0x124d. This value is in host-endian order, not big-endian like the rest of +XFS. *rud_size*:: Size of this log item. Should be 1.
design/XFS_Filesystem_Structure/magic.asciidoc +4 -0
@@ -78,6 +78,10 @@ are not aligned to blocks. | +XFS_LI_ATTRD+ | 0x1247 | | xref:ATTRD_Log_Item[Extended Attribute Update Done] | +XFS_LI_XMI+ | 0x1248 | | xref:XMI_Log_Item[File Mapping Exchange Intent] | +XFS_LI_XMD+ | 0x1249 | | xref:XMD_Log_Item[File Mapping Exchange Done] +| +XFS_LI_EFI_RT+ | 0x124a | | xref:EFI_Log_Item[Extent Freeing Intent Log Item] +| +XFS_LI_EFD_RT+ | 0x124b | | xref:EFD_Log_Item[Extent Freeing Done Log Item] +| +XFS_LI_RUI_RT+ | 0x124c | | xref:RUI_Log_Item[Reverse Mapping Update Intent] +| +XFS_LI_RUD_RT+ | 0x124d | | xref:RUD_Log_Item[Reverse Mapping Update Done] |===== = Theoretical Limits
design/XFS_Filesystem_Structure/ondisk_inode.asciidoc +1 -0
@@ -204,6 +204,7 @@ enum xfs_metafile_type { XFS_METAFILE_PRJQUOTA, XFS_METAFILE_RTBITMAP, XFS_METAFILE_RTSUMMARY, + XFS_METAFILE_RTRMAP, }; ----
design/XFS_Filesystem_Structure/realtime.asciidoc +4 -1
@@ -299,7 +299,9 @@ The summary counts were excerpted for brevity. To reduce metadata contention for space allocation and remapping activities being applied to realtime files, the realtime volume can be split into allocation groups, just like the data volume. The free space information is -still contained in a single file that applies to the entire volume. +still contained in a single file that applies to the entire volume. This +sharding enables code reuse between the data and realtime reverse mapping +indexes and supports parallelism of reverse mapping and online fsck activities. Each realtime allocation group can contain up to (2^31^ - 1) filesystem blocks, regardless of the underlying realtime extent size.
@@ -309,6 +311,7 @@ Each realtime group has the following characteristics: * Group 0 has a super block describing overall filesystem info * Free space bitmap * Summary of free space + * Reverse space mapping btree The free space metadata are the same as described in the previous sections, except that their scope covers only a single rtgroup. The other structures are
design/XFS_Filesystem_Structure/rtrmapbt.asciidoc +89 -126
304 lines not on this page · raw message
PATCH 2/2 design: document changes for the realtime refcount btree
From: Darrick J. Wong <[email protected]>
Update the ondisk format documentation to reflect the realtime refcount btree.
Signed-off-by Darrick J. Wong <[email protected]>
--- .../internal_inodes.asciidoc | 5 - .../journaling_log.asciidoc | 10 + design/XFS_Filesystem_Structure/magic.asciidoc | 3 .../XFS_Filesystem_Structure/ondisk_inode.asciidoc | 2 design/XFS_Filesystem_Structure/realtime.asciidoc | 6 - .../XFS_Filesystem_Structure/rtrefcountbt.asciidoc | 172 ++++++++++++++++++++ 6 files changed, 191 insertions(+), 7 deletions(-) create mode 100644 design/XFS_Filesystem_Structure/rtrefcountbt.asciidoc
design/XFS_Filesystem_Structure/internal_inodes.asciidoc +4 -1
@@ -27,6 +27,7 @@ of those inodes have been deallocated and may be reused by future features. | xref:Real-Time_Bitmap_Inode[Realtime Bitmap] | /rtgroups/*.bitmap | xref:Real-Time_Summary_Inode[Realtime Summary] | /rtgroups/*.summary | xref:Real_time_Reverse_Mapping_Btree[Realtime Reverse Mapping B+tree] | /rtgroups/*.rmap +| xref:Real_time_Refcount_Btree[Realtime Reference Count+tree] | /rtgroups/*.refcount |===== Metadata files are flagged by the +XFS_DIFLAG2_METADATA+ flag in the
@@ -301,4 +302,6 @@ xref:Real-Time_Bitmap_Inode[Bitmap Inode] and the xref:Real-Time_Summary_Inode[Summary Inode]. Each realtime group can allocate one inode to managing a -xref:Real_time_Reverse_Mapping_Btree[reverse-index of space] usage. +xref:Real_time_Reverse_Mapping_Btree[reverse-index of space] usage, and +a second one to manage xref:Real_time_Refcount_Btree[reference counts] of space +usage.
design/XFS_Filesystem_Structure/journaling_log.asciidoc +6 -4
@@ -586,8 +586,9 @@ struct xfs_cui_log_format { ---- *cui_type*:: -The signature of an CUI operation, 0x1242. This value is in host-endian order, -not big-endian like the rest of XFS. +The signature of an CUI operation, 0x1242. For a realtime CUI, this vlaue is +0x124e. This value is in host-endian order, not big-endian like the rest of +XFS. *cui_size*:: Size of this log item. Should be 1.
@@ -621,8 +622,9 @@ struct xfs_cud_log_format { ---- *cud_type*:: -The signature of an RUD operation, 0x1243. This value is in host-endian order, -not big-endian like the rest of XFS. +The signature of an RUD operation, 0x1243. For a realtime CUD, this value is +0x124f. This value is in host-endian order, not big-endian like the rest of +XFS. *cud_size*:: Size of this log item. Should be 1.
design/XFS_Filesystem_Structure/magic.asciidoc +3 -0
@@ -51,6 +51,7 @@ relevant chapters. Magic numbers tend to have consistent locations: | +XFS_REFC_CRC_MAGIC+ | 0x52334643 | R3FC | xref:Reference_Count_Btree[Reference Count B+tree], v5 only | +XFS_MD_MAGIC+ | 0x5846534d | XFSM | xref:Metadata_Dumps[Metadata Dumps] | +XFS_RTSB_MAGIC+ | 0x46726F67 | Frog | xref:Realtime_Groups[Realtime Groups] +| +XFS_RTREFC_CRC_MAGIC+ | 0x52434e54 | RCNT | xref:Real_time_Refcount_Btree[Real-Time Reference Count B+tree], v5 only |===== The magic numbers for log items are at offset zero in each log item, but items
@@ -82,6 +83,8 @@ are not aligned to blocks. | +XFS_LI_EFD_RT+ | 0x124b | | xref:EFD_Log_Item[Extent Freeing Done Log Item] | +XFS_LI_RUI_RT+ | 0x124c | | xref:RUI_Log_Item[Reverse Mapping Update Intent] | +XFS_LI_RUD_RT+ | 0x124d | | xref:RUD_Log_Item[Reverse Mapping Update Done] +| +XFS_LI_CUI_RT+ | 0x124e | | xref:CUI_Log_Item[Reference Count Update Intent] +| +XFS_LI_CUD_RT+ | 0x124f | | xref:CUD_Log_Item[Reference Count Update Done] |===== = Theoretical Limits
design/XFS_Filesystem_Structure/ondisk_inode.asciidoc +2 -0
@@ -183,6 +183,7 @@ typedef enum xfs_dinode_fmt { XFS_DINODE_FMT_BTREE, XFS_DINODE_FMT_UUID, XFS_DINODE_FMT_RMAP, + XFS_DINODE_FMT_REFCOUNT } xfs_dinode_fmt_t; ----
@@ -205,6 +206,7 @@ enum xfs_metafile_type { XFS_METAFILE_RTBITMAP, XFS_METAFILE_RTSUMMARY, XFS_METAFILE_RTRMAP, + XFS_METAFILE_RTREFCOUNT, }; ----
design/XFS_Filesystem_Structure/realtime.asciidoc +4 -2
@@ -14,8 +14,7 @@ By placing the real time device (and the journal) on separate high-performance storage devices, it is possible to reduce most of the unpredictability in I/O response times that come from metadata operations. -None of the XFS per-AG B+trees are involved with real time files. It is not -possible for real time files to share data blocks. +None of the XFS per-AG B+trees are involved with real time files. [[Real-Time_Bitmap_Inode]] == Free Space Bitmap Inode
@@ -312,6 +311,7 @@ Each realtime group has the following characteristics: * Free space bitmap * Summary of free space * Reverse space mapping btree + * Reference count btree The free space metadata are the same as described in the previous sections, except that their scope covers only a single rtgroup. The other structures are
@@ -395,3 +395,5 @@ meta_uuid = 7e55b909-8728-4d69-a1fa-891427314eea ---- include::rtrmapbt.asciidoc[] + +include::rtrefcountbt.asciidoc[]
design/XFS_Filesystem_Structure/rtrefcountbt.asciidoc new file +172 -0
172 lines not on this page · raw message
Looks good:
Reviewed-by Christoph Hellwig <[email protected]>
Re: [PATCH 2/2] design: document changes for the realtime refcount btree
In reply to Darrick J. WongLooks good:
Reviewed-by Christoph Hellwig <[email protected]>