xfsprogs: metadata directories and realtime groups
Hi Andrey and Christoph,
This begins the second round of review of userspace support for the metadata directory tree and realtime allocation group code that was merged into kernel 6.13-rc1. I've added all of Christoph's RVB tags from last time and made a few corrections and adjustments as noted in the replies.
However, the xfs_mdrestore patches have expanded -- now there's a cleanup patch to add the mdrestore device instead of opencoding the fd/flags parameters, which changed the patch to support mdrestoring rt superblocks quite a bit. That patch never got an RVB tag, so that's why there are two left.
As always, you can browse / pull the branch from here:
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-quotas-6.13_2024-12-12
These are the patches that have not yet been reviewed:
[PATCHSET v5.9] xfsprogs: shard the realtime section
[PATCH 34/50] xfs_mdrestore: refactor open-coded fd/is_file into a [PATCH 35/50] xfs_mdrestore: restore rt group superblocks to realtime
I'm resending only these two patches to reduce pointless retransmission.
--D
PATCH 1/1 xfsprogs: shard the realtime section
Hi all,
Right now, the realtime section uses a single pair of metadata inodes to store the free space information. This presents a scalability problem since every thread trying to allocate or free rt extents have to lock these files. Solve this problem by sharding the realtime section into separate realtime allocation groups.
While we're at it, define a superblock to be stamped into the start of the rt section. This enables utilities such as blkid to identify block devices containing realtime sections, and avoids the situation where anything written into block 0 of the realtime extent can be misinterpreted as file data.
The best advantage for rtgroups will become evident later when we get to adding rmap and reflink to the realtime volume, since the geometry constraints are the same for rt groups and AGs. Hence we can reuse all that code directly.
This is a very large patchset, but it catches us up with 20 years of technical debt that have accumulated.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
Comments and questions are, as always, welcome.
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=realtime-groups-6.13
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-groups-6.13
---
Commits in this patchset:
* libxfs: remove XFS_ILOCK_RT*
* libxfs: adjust xfs_fsb_to_db to handle segmented rtblocks
* xfs_repair,mkfs: port to libxfs_rt{bitmap,summary}_create
* libxfs: use correct rtx count to block count conversion
* libfrog: scrub the realtime group superblock
* man: document the rt group geometry ioctl
* libxfs: port userspace deferred log item to handle rtgroups
* libxfs: implement some sanity checking for enormous rgcount
* libfrog: support scrubbing rtgroup metadata paths
* libfrog: report rt groups in output
* libfrog: add bitmap_clear
* xfs_logprint: report realtime EFIs
* xfs_repair: adjust rtbitmap/rtsummary word updates to handle big endian values
* xfs_repair: refactor phase4
* xfs_repair: refactor offsetof+sizeof to offsetofend
* xfs_repair: improve rtbitmap discrepancy reporting
* xfs_repair: simplify rt_lock handling
* xfs_repair: add a real per-AG bitmap abstraction
* xfs_repair: support realtime groups
* xfs_repair: find and clobber rtgroup bitmap and summary files
* xfs_repair: support realtime superblocks
* xfs_repair: repair rtbitmap and rtsummary block headers
* xfs_db: enable the rtblock and rtextent commands for segmented rt block numbers
* xfs_db: enable rtconvert to handle segmented rtblocks
* xfs_db: listify the definition of enum typnm
* xfs_db: support dumping realtime group data and superblocks
* xfs_db: support changing the label and uuid of rt superblocks
* xfs_db: enable conversion of rt space units
* xfs_db: metadump metadir rt bitmap and summary files
* xfs_db: metadump realtime devices
* xfs_db: dump rt bitmap blocks
* xfs_db: dump rt summary blocks
* xfs_db: report rt group and block number in the bmap command
* xfs_mdrestore: refactor open-coded fd/is_file into a structure
* xfs_mdrestore: restore rt group superblocks to realtime device
* xfs_io: support scrubbing rtgroup metadata
* xfs_io: support scrubbing rtgroup metadata paths
* xfs_io: add a command to display allocation group information
* xfs_io: add a command to display realtime group information
* xfs_io: display rt group in verbose bmap output
* xfs_io: display rt group in verbose fsmap output
* xfs_spaceman: report on realtime group health
* xfs_scrub: scrub realtime allocation group metadata
* xfs_scrub: check rtgroup metadata directory connections
* xfs_scrub: cleanup fsmap keys initialization
* xfs_scrub: call GETFSMAP for each rt group in parallel
* xfs_scrub: trim realtime volumes too
* xfs_scrub: use histograms to speed up phase 8 on the realtime volume
* mkfs: add headers to realtime bitmap blocks
* mkfs: format realtime groups
--- db/Makefile | 1 db/block.c | 34 ++ db/bmap.c | 56 +++- db/command.c | 2 db/convert.c | 118 +++++++- db/field.c | 20 + db/field.h | 10 + db/inode.c | 36 ++ db/metadump.c | 59 ++++ db/rtgroup.c | 154 ++++++++++ db/rtgroup.h | 21 + db/sb.c | 117 +++++++- db/type.c | 16 + db/type.h | 32 ++ db/xfs_metadump.sh | 5 include/xfs.h | 15 + include/xfs_metadump.h | 8 + io/Makefile | 1 io/aginfo.c | 215 ++++++++++++++ io/bmap.c | 27 +- io/fsmap.c | 22 + io/init.c | 1 io/io.h | 1 io/scrub.c | 81 +++++ libfrog/bitmap.c | 25 +- libfrog/bitmap.h | 1 libfrog/div64.h | 6 libfrog/fsgeom.c | 24 +- libfrog/fsgeom.h | 16 + libfrog/scrub.c | 24 +- libfrog/scrub.h | 1 libfrog/util.c | 12 + libfrog/util.h | 1 libxfs/defer_item.c | 73 +++-- libxfs/init.c | 46 +++ libxfs/libxfs_api_defs.h | 5 libxfs/libxfs_priv.h | 8 - libxfs/topology.c | 42 +++ libxfs/topology.h | 3 libxfs/trans.c | 2 libxfs/util.c | 2 logprint/log_misc.c | 2 logprint/log_print_all.c | 8 + logprint/log_redo.c | 57 +++- man/man2/ioctl_xfs_rtgroup_geometry.2 | 99 ++++++ man/man2/ioctl_xfs_scrub_metadata.2 | 9 + man/man8/mkfs.xfs.8.in | 31 ++ man/man8/xfs_db.8 | 34 ++ man/man8/xfs_io.8 | 29 ++ man/man8/xfs_mdrestore.8 | 10 + man/man8/xfs_metadump.8 | 11 + man/man8/xfs_spaceman.8 | 5 mdrestore/xfs_mdrestore.c | 163 ++++++----- mkfs/proto.c | 139 ++++++--- mkfs/xfs_mkfs.c | 282 ++++++++++++++++++ repair/agheader.c | 27 -- repair/agheader.h | 10 + repair/dino_chunks.c | 58 +++- repair/dinode.c | 162 +++++++---- repair/dir2.c | 13 + repair/globals.c | 3 repair/globals.h | 6 repair/incore.c | 235 +++++++++++---- repair/incore.h | 36 ++ repair/incore_ext.c | 3 repair/phase2.c | 51 ++- repair/phase3.c | 4 repair/phase4.c | 221 ++++++++------ repair/phase5.c | 2 repair/phase6.c | 180 +++++++++++- repair/rmap.c | 4 repair/rt.c | 506 +++++++++++++++++++++++++++++---- repair/rt.h | 23 ++ repair/sb.c | 41 +++ repair/scan.c | 36 +- repair/xfs_repair.c | 19 + scrub/phase2.c | 124 ++++++-- scrub/phase5.c | 24 +- scrub/phase6.c | 17 + scrub/phase7.c | 7 scrub/phase8.c | 36 ++ scrub/repair.c | 1 scrub/scrub.c | 7 scrub/scrub.h | 13 + scrub/spacemap.c | 102 +++++-- scrub/xfs_scrub.c | 2 scrub/xfs_scrub.h | 1 spaceman/health.c | 63 ++++ 88 files changed, 3541 insertions(+), 718 deletions(-) create mode 100644 db/rtgroup.c create mode 100644 db/rtgroup.h create mode 100644 io/aginfo.c create mode 100644 man/man2/ioctl_xfs_rtgroup_geometry.2
PATCH 34/50 xfs_mdrestore: refactor open-coded fd/is_file into a structure
From: Darrick J. Wong <[email protected]>
Create an explicit object to track the fd and flags associated with a device onto which we are restoring metadata, and use it to reduce the amount of open-coded arguments to ->restore. This avoids some grossness in the next patch.
--- mdrestore/xfs_mdrestore.c | 123 +++++++++++++++++++++++---------------------- 1 file changed, 63 insertions(+), 60 deletions(-)
mdrestore/xfs_mdrestore.c +63 -60
@@ -15,12 +15,20 @@ union mdrestore_headers {struct xfs_metadump_header v2; }; +struct mdrestore_dev { +int fd; +bool is_file; +}; + +#define DEFINE_MDRESTORE_DEV (name) \ + struct mdrestore_dev name = { .fd = -1 } +struct mdrestore_ops {void (*read_header)(union mdrestore_headers *header,FILE *md_fp);void (*show_info)(union mdrestore_headers *header,const char *md_file);void (*restore)(union mdrestore_headers *header,FILE *md_fp, -int ddev_fd,bool is_data_target_file,int logdev_fd, -bool is_log_target_file); +const struct mdrestore_dev *ddev, +const struct mdrestore_dev *logdev); }; static struct mdrestore {
@@ -108,25 +116,24 @@ fixup_superblock(fatal ("error writing primary superblock: %s\n" ,strerror (errno)); } -static int +static void open_device ( -char *path, -bool *is_file) +struct mdrestore_dev *dev, +char *path) { -struct stat statbuf; -int open_flags; -int fd; +struct stat statbuf; +int open_flags; open_flags =O_RDWR ; - *is_file = false; + dev->is_file = false;if (stat (path, &statbuf) <0 ) {/* ok, assume it's a file and create it */ open_flags |=O_CREAT ; - *is_file = true; + dev->is_file = true; }else if (S_ISREG (statbuf.st_mode)) { open_flags |=O_TRUNC ; - *is_file = true; + dev->is_file = true; }else if (platform_check_ismounted (path,NULL , &statbuf,0 )) {/* * check to make sure a filesystem isn't mounted on the device
@@ -136,23 +143,30 @@ open_device( path); } - fd =open (path, open_flags,0644 ); -if (fd <0 ) + dev->fd =open (path, open_flags,0644 ); +if (dev->fd <0 )fatal ("couldn't open \"%s\"\n" , path); +} -return fd; +static void +close_device ( +struct mdrestore_dev *dev) +{ +if (dev->fd >=0 ) +close (dev->fd); + dev->fd =-1 ; + dev->is_file = false; }static void verify_device_size ( -int dev_fd, -bool is_file, -xfs_rfsblock_t nr_blocks, -uint32_t blocksize) +const struct mdrestore_dev *dev, +xfs_rfsblock_t nr_blocks, +uint32_t blocksize) { -if (is_file) { +if (dev->is_file) {/* ensure regular files are correctly sized */ -if (ftruncate (dev_fd, nr_blocks * blocksize)) +if (ftruncate (dev->fd, nr_blocks * blocksize))fatal ("cannot set filesystem image size: %s\n" ,strerror (errno)); }else {
@@ -161,7 +175,7 @@ verify_device_size(off_t off; off = nr_blocks * blocksize -sizeof (lb); -if (pwrite (dev_fd, lb,sizeof (lb), off) <0 ) +if (pwrite (dev->fd, lb,sizeof (lb), off) <0 )fatal ("failed to write last block, is target too " "small? (error: %s)\n" ,strerror (errno)); }
@@ -195,12 +209,10 @@ show_info_v1(static void restore_v1 ( -union mdrestore_headers *h, -FILE *md_fp, -int ddev_fd, -bool is_data_target_file, -int logdev_fd, -bool is_log_target_file) +union mdrestore_headers *h, +FILE *md_fp, +const struct mdrestore_dev *ddev, +const struct mdrestore_dev *logdev) {struct xfs_metablock *metablock;/* header + index + blocks */ __be64 *block_index;
@@ -254,8 +266,7 @@ restore_v1( ((struct xfs_dsb *)block_buffer)->sb_inprogress =1 ; -verify_device_size (ddev_fd, is_data_target_file, sb.sb_dblocks, - sb.sb_blocksize); +verify_device_size (ddev, sb.sb_dblocks, sb.sb_blocksize); bytes_read =0 ;
@@ -263,7 +274,7 @@ restore_v1(maybe_print_progress (&mb_read, bytes_read);for (cur_index =0 ; cur_index < mb_count; cur_index++) { -if (pwrite (ddev_fd, &block_buffer[cur_index << +if (pwrite (ddev->fd, &block_buffer[cur_index << h->v1.mb_blocklog], block_size,be64_to_cpu (block_index[cur_index]) <<BBSHIFT ) <0 )
@@ -292,7 +303,7 @@ restore_v1(final_print_progress (&mb_read, bytes_read); -fixup_superblock (ddev_fd, block_buffer, &sb); +fixup_superblock (ddev->fd, block_buffer, &sb);free (metablock); }
@@ -376,12 +387,10 @@ restore_meta_extent(static void restore_v2 ( -union mdrestore_headers *h, -FILE *md_fp, -int ddev_fd, -bool is_data_target_file, -int logdev_fd, -bool is_log_target_file) +union mdrestore_headers *h, +FILE *md_fp, +const struct mdrestore_dev *ddev, +const struct mdrestore_dev *logdev) {struct xfs_sb sb;struct xfs_meta_extent xme;
@@ -415,16 +424,14 @@ restore_v2( ((struct xfs_dsb *)block_buffer)->sb_inprogress =1 ; -verify_device_size (ddev_fd, is_data_target_file, sb.sb_dblocks, - sb.sb_blocksize); +verify_device_size (ddev, sb.sb_dblocks, sb.sb_blocksize);if (sb.sb_logstart ==0 ) {ASSERT (mdrestore.external_log == true); -verify_device_size (logdev_fd, is_log_target_file, sb.sb_logblocks, - sb.sb_blocksize); +verify_device_size (logdev, sb.sb_logblocks, sb.sb_blocksize); } -if (pwrite (ddev_fd, block_buffer, len,0 ) <0 ) +if (pwrite (ddev->fd, block_buffer, len,0 ) <0 )fatal ("error writing primary superblock: %s\n" ,strerror (errno));
@@ -446,11 +453,11 @@ restore_v2(switch (be64_to_cpu (xme.xme_addr) &XME_ADDR_DEVICE_MASK ) {case XME_ADDR_DATA_DEVICE : device ="data" ; - fd = ddev_fd; + fd = ddev->fd;break ;case XME_ADDR_LOG_DEVICE : device ="log" ; - fd = logdev_fd; + fd = logdev->fd;break ;default :fatal ("Invalid device found in metadump\n" );
@@ -467,7 +474,7 @@ restore_v2(final_print_progress (&mb_read, bytes_read); -fixup_superblock (ddev_fd, block_buffer, &sb); +fixup_superblock (ddev->fd, block_buffer, &sb);free (block_buffer); }
@@ -492,13 +499,11 @@ main( char **argv) {union mdrestore_headers headers; +DEFINE_MDRESTORE_DEV (ddev); +DEFINE_MDRESTORE_DEV (logdev);FILE *src_f; -char *logdev =NULL ; -int data_dev_fd =-1 ; -int log_dev_fd =-1 ; +char *logdev_path =NULL ;int c; -bool is_data_dev_file = false; -bool is_log_dev_file = false; mdrestore.show_progress = false; mdrestore.show_info = false;
@@ -516,7 +521,7 @@ main( mdrestore.show_info = true;break ;case 'l' : - logdev = optarg; + logdev_path = optarg; mdrestore.external_log = true;break ;case 'V' :
@@ -555,7 +560,7 @@ main(switch (be32_to_cpu (headers.magic)) {case XFS_MD_MAGIC_V1 : -if (logdev !=NULL ) +if (logdev_path !=NULL )usage (); mdrestore.mdrops = &mdrestore_ops_v1;break ;
@@ -581,18 +586,16 @@ main( optind++;/* check and open data device */ - data_dev_fd =open_device (argv[optind], &is_data_dev_file); +open_device (&ddev, argv[optind]); +/* check and open log device */ if (mdrestore.external_log) -/* check and open log device */ - log_dev_fd =open_device (logdev, &is_log_dev_file); +open_device (&logdev, logdev_path); - mdrestore.mdrops->restore (&headers, src_f, data_dev_fd, - is_data_dev_file, log_dev_fd, is_log_dev_file); + mdrestore.mdrops->restore (&headers, src_f, &ddev, &logdev); -close (data_dev_fd); -if (mdrestore.external_log) -close (log_dev_fd); +close_device (&ddev); +close_device (&logdev);if (src_f != stdin)fclose (src_f);
Re: [PATCH 34/50] xfs_mdrestore: refactor open-coded fd/is_file into a structure
Quoted 8 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Create an explicit object to track the fd and flags associated with a device onto which we are restoring metadata, and use it to reduce the amount of open-coded arguments to ->restore. This avoids some grossness in the next patch.
Signed-off-by "Darrick J. Wong" <[email protected]>
LGTM
Quoted 281 lines
--- mdrestore/xfs_mdrestore.c | 123 +++++++++++++++++++++++---------------------- 1 file changed, 63 insertions(+), 60 deletions(-)mdrestore/xfs_mdrestore.c +63
-60@@ -15,12 +15,20 @@ union mdrestore_headers {struct xfs_metadump_header v2; }; +struct mdrestore_dev { +int fd; +bool is_file; +}; + +#define DEFINE_MDRESTORE_DEV (name) \ + struct mdrestore_dev name = { .fd = -1 } +struct mdrestore_ops {void (*read_header)(union mdrestore_headers *header,FILE *md_fp);void (*show_info)(union mdrestore_headers *header,const char *md_file);void (*restore)(union mdrestore_headers *header,FILE *md_fp, -int ddev_fd,bool is_data_target_file,int logdev_fd, -bool is_log_target_file); +const struct mdrestore_dev *ddev, +const struct mdrestore_dev *logdev); }; static struct mdrestore {@@ -108,25 +116,24 @@ fixup_superblock(fatal ("error writing primary superblock: %s\n" ,strerror (errno)); } -static int +static void open_device ( -char *path, -bool *is_file) +struct mdrestore_dev *dev, +char *path) { -struct stat statbuf; -int open_flags; -int fd; +struct stat statbuf; +int open_flags; open_flags =O_RDWR ; - *is_file = false; + dev->is_file = false;if (stat (path, &statbuf) <0 ) {/* ok, assume it's a file and create it */ open_flags |=O_CREAT ; - *is_file = true; + dev->is_file = true; }else if (S_ISREG (statbuf.st_mode)) { open_flags |=O_TRUNC ; - *is_file = true; + dev->is_file = true; }else if (platform_check_ismounted (path,NULL , &statbuf,0 )) {/* * check to make sure a filesystem isn't mounted on the device @@ -136,23 +143,30 @@ open_device( path); } - fd =open (path, open_flags,0644 ); -if (fd <0 ) + dev->fd =open (path, open_flags,0644 ); +if (dev->fd <0 )fatal ("couldn't open \"%s\"\n" , path); +} -return fd; +static void +close_device ( +struct mdrestore_dev *dev) +{ +if (dev->fd >=0 ) +close (dev->fd); + dev->fd =-1 ; + dev->is_file = false; }static void verify_device_size ( -int dev_fd, -bool is_file, -xfs_rfsblock_t nr_blocks, -uint32_t blocksize) +const struct mdrestore_dev *dev, +xfs_rfsblock_t nr_blocks, +uint32_t blocksize) { -if (is_file) { +if (dev->is_file) {/* ensure regular files are correctly sized */ -if (ftruncate (dev_fd, nr_blocks * blocksize)) +if (ftruncate (dev->fd, nr_blocks * blocksize))fatal ("cannot set filesystem image size: %s\n" ,strerror (errno)); }else {@@ -161,7 +175,7 @@ verify_device_size(off_t off; off = nr_blocks * blocksize -sizeof (lb); -if (pwrite (dev_fd, lb,sizeof (lb), off) <0 ) +if (pwrite (dev->fd, lb,sizeof (lb), off) <0 )fatal ("failed to write last block, is target too " "small? (error: %s)\n" ,strerror (errno)); }@@ -195,12 +209,10 @@ show_info_v1(static void restore_v1 ( -union mdrestore_headers *h, -FILE *md_fp, -int ddev_fd, -bool is_data_target_file, -int logdev_fd, -bool is_log_target_file) +union mdrestore_headers *h, +FILE *md_fp, +const struct mdrestore_dev *ddev, +const struct mdrestore_dev *logdev) {struct xfs_metablock *metablock;/* header + index + blocks */ __be64 *block_index;@@ -254,8 +266,7 @@ restore_v1( ((struct xfs_dsb *)block_buffer)->sb_inprogress =1 ; -verify_device_size (ddev_fd, is_data_target_file, sb.sb_dblocks, - sb.sb_blocksize); +verify_device_size (ddev, sb.sb_dblocks, sb.sb_blocksize); bytes_read =0 ;@@ -263,7 +274,7 @@ restore_v1(maybe_print_progress (&mb_read, bytes_read);for (cur_index =0 ; cur_index < mb_count; cur_index++) { -if (pwrite (ddev_fd, &block_buffer[cur_index << +if (pwrite (ddev->fd, &block_buffer[cur_index << h->v1.mb_blocklog], block_size,be64_to_cpu (block_index[cur_index]) <<BBSHIFT ) <0 )@@ -292,7 +303,7 @@ restore_v1(final_print_progress (&mb_read, bytes_read); -fixup_superblock (ddev_fd, block_buffer, &sb); +fixup_superblock (ddev->fd, block_buffer, &sb);free (metablock); }@@ -376,12 +387,10 @@ restore_meta_extent(static void restore_v2 ( -union mdrestore_headers *h, -FILE *md_fp, -int ddev_fd, -bool is_data_target_file, -int logdev_fd, -bool is_log_target_file) +union mdrestore_headers *h, +FILE *md_fp, +const struct mdrestore_dev *ddev, +const struct mdrestore_dev *logdev) {struct xfs_sb sb;struct xfs_meta_extent xme;@@ -415,16 +424,14 @@ restore_v2( ((struct xfs_dsb *)block_buffer)->sb_inprogress =1 ; -verify_device_size (ddev_fd, is_data_target_file, sb.sb_dblocks, - sb.sb_blocksize); +verify_device_size (ddev, sb.sb_dblocks, sb.sb_blocksize);if (sb.sb_logstart ==0 ) {ASSERT (mdrestore.external_log == true); -verify_device_size (logdev_fd, is_log_target_file, sb.sb_logblocks, - sb.sb_blocksize); +verify_device_size (logdev, sb.sb_logblocks, sb.sb_blocksize); } -if (pwrite (ddev_fd, block_buffer, len,0 ) <0 ) +if (pwrite (ddev->fd, block_buffer, len,0 ) <0 )fatal ("error writing primary superblock: %s\n" ,strerror (errno));@@ -446,11 +453,11 @@ restore_v2(switch (be64_to_cpu (xme.xme_addr) &XME_ADDR_DEVICE_MASK ) {case XME_ADDR_DATA_DEVICE : device ="data" ; - fd = ddev_fd; + fd = ddev->fd;break ;case XME_ADDR_LOG_DEVICE : device ="log" ; - fd = logdev_fd; + fd = logdev->fd;break ;default :fatal ("Invalid device found in metadump\n" );@@ -467,7 +474,7 @@ restore_v2(final_print_progress (&mb_read, bytes_read); -fixup_superblock (ddev_fd, block_buffer, &sb); +fixup_superblock (ddev->fd, block_buffer, &sb);free (block_buffer); }@@ -492,13 +499,11 @@ main( char **argv) {union mdrestore_headers headers; +DEFINE_MDRESTORE_DEV (ddev); +DEFINE_MDRESTORE_DEV (logdev);FILE *src_f; -char *logdev =NULL ; -int data_dev_fd =-1 ; -int log_dev_fd =-1 ; +char *logdev_path =NULL ;int c; -bool is_data_dev_file = false; -bool is_log_dev_file = false; mdrestore.show_progress = false; mdrestore.show_info = false;@@ -516,7 +521,7 @@ main( mdrestore.show_info = true;break ;case 'l' : - logdev = optarg; + logdev_path = optarg; mdrestore.external_log = true;break ;case 'V' :@@ -555,7 +560,7 @@ main(switch (be32_to_cpu (headers.magic)) {case XFS_MD_MAGIC_V1 : -if (logdev !=NULL ) +if (logdev_path !=NULL )usage (); mdrestore.mdrops = &mdrestore_ops_v1;break ;@@ -581,18 +586,16 @@ main( optind++;/* check and open data device */ - data_dev_fd =open_device (argv[optind], &is_data_dev_file); +open_device (&ddev, argv[optind]); +/* check and open log device */ if (mdrestore.external_log) -/* check and open log device */ - log_dev_fd =open_device (logdev, &is_log_dev_file); +open_device (&logdev, logdev_path); - mdrestore.mdrops->restore (&headers, src_f, data_dev_fd, - is_data_dev_file, log_dev_fd, is_log_dev_file); + mdrestore.mdrops->restore (&headers, src_f, &ddev, &logdev); -close (data_dev_fd); -if (mdrestore.external_log) -close (log_dev_fd); +close_device (&ddev); +close_device (&logdev);if (src_f != stdin)fclose (src_f);
Signature · 1 line
- Andrey
PATCH 35/50 xfs_mdrestore: restore rt group superblocks to realtime device
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Support restoring realtime device metadata to the realtime device, if the dumped filesystem had one.
--- man/man8/xfs_mdrestore.8 | 10 ++++++++++ mdrestore/xfs_mdrestore.c | 47 ++++++++++++++++++++++++++++++++++++--------- 2 files changed, 48 insertions(+), 9 deletions(-)
man/man8/xfs_mdrestore.8 +10 -0
@@ -8,6 +8,9 @@ .SH SYNOPSIS ] [ .B \-l .I logdev +] [ +.B \-r +.I rtdev ] .I source .I target
@@ -17,6 +20,9 @@ .SH SYNOPSIS [ .B \-l .I logdev +] [ +.B \-r +.I rtdev ] .I source .br
@@ -61,6 +67,10 @@ .SH OPTIONS In such a scenario, the user has to provide a device to which the log device contents from the metadump file are copied. .TP +.BI \-r " rtdev" +Restore realtime device metadata to this device. +This is only required for a metadump in v2 format. +.TP .B \-V Prints the version number and exits. .SH DIAGNOSTICS
mdrestore/xfs_mdrestore.c +38 -9
140 lines not on this page · raw message
Re: [PATCH 35/50] xfs_mdrestore: restore rt group superblocks to realtime device
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Support restoring realtime device metadata to the realtime device, if the dumped filesystem had one.
Signed-off-by "Darrick J. Wong" <[email protected]>
LGTM
Quoted 198 lines
--- man/man8/xfs_mdrestore.8 | 10 ++++++++++ mdrestore/xfs_mdrestore.c | 47 ++++++++++++++++++++++++++++++++++++--------- 2 files changed, 48 insertions(+), 9 deletions(-)man/man8/xfs_mdrestore.8 +10
-0@@ -8,6 +8,9 @@ .SH SYNOPSIS ] [ .B \-l .I logdev +] [ +.B \-r +.I rtdev ] .I source .I target@@ -17,6 +20,9 @@ .SH SYNOPSIS [ .B \-l .I logdev +] [ +.B \-r +.I rtdev ] .I source .br@@ -61,6 +67,10 @@ .SH OPTIONS In such a scenario, the user has to provide a device to which the log device contents from the metadump file are copied. .TP +.BI \-r " rtdev" +Restore realtime device metadata to this device. +This is only required for a metadump in v2 format. +.TP .B \-V Prints the version number and exits. .SH DIAGNOSTICSmdrestore/xfs_mdrestore.c +38
-9@@ -28,7 +28,8 @@ struct mdrestore_ops {void (*show_info)(union mdrestore_headers *header,const char *md_file);void (*restore)(union mdrestore_headers *header,FILE *md_fp,const struct mdrestore_dev *ddev, -const struct mdrestore_dev *logdev); +const struct mdrestore_dev *logdev, +const struct mdrestore_dev *rtdev); }; static struct mdrestore {@@ -37,6 +38,7 @@ static struct mdrestore {bool show_info;bool progress_since_warning;bool external_log; +bool realtime_data; } mdrestore; static void@@ -212,7 +214,8 @@ restore_v1(union mdrestore_headers *h, FILE *md_fp, const struct mdrestore_dev *ddev, - const struct mdrestore_dev *logdev) + const struct mdrestore_dev *logdev, + const struct mdrestore_dev *rtdev) { struct xfs_metablock *metablock; /* header + index + blocks */__be64 *block_index;@@ -336,8 +339,9 @@ read_header_v2(if (!mdrestore.external_log && (compat &XFS_MD2_COMPAT_EXTERNALLOG ))fatal ("External Log device is required\n" ); -if (h->v2.xmh_incompat_flags &cpu_to_be32 (XFS_MD2_INCOMPAT_RTDEVICE )) -fatal ("Realtime device not yet supported\n" ); +if ((h->v2.xmh_incompat_flags &cpu_to_be32 (XFS_MD2_INCOMPAT_RTDEVICE )) && + !mdrestore.realtime_data) +fatal ("Realtime device is required\n" ); } static void@@ -346,14 +350,17 @@ show_info_v2( const char *md_file) {uint32_t compat_flags; +uint32_t incompat_flags; compat_flags =be32_to_cpu (h->v2.xmh_compat_flags); + incompat_flags =be32_to_cpu (h->v2.xmh_incompat_flags); -printf ("%s: %sobfuscated, %s log, external log contents are %sdumped, %s metadata blocks,\n" , +printf ("%s: %sobfuscated, %s log, external log contents are %sdumped, rt device contents are %sdumped, %s metadata blocks,\n" , md_file, compat_flags &XFS_MD2_COMPAT_OBFUSCATED ?"" :"not " , compat_flags &XFS_MD2_COMPAT_DIRTYLOG ?"dirty" :"clean" , compat_flags &XFS_MD2_COMPAT_EXTERNALLOG ?"" :"not " , + incompat_flags &XFS_MD2_INCOMPAT_RTDEVICE ?"" :"not " , compat_flags &XFS_MD2_COMPAT_FULLBLOCKS ?"full" :"zeroed" ); }@@ -390,7 +397,8 @@ restore_v2(union mdrestore_headers *h, FILE *md_fp, const struct mdrestore_dev *ddev, - const struct mdrestore_dev *logdev) + const struct mdrestore_dev *logdev, + const struct mdrestore_dev *rtdev) { struct xfs_sb sb;struct xfs_meta_extent xme;@@ -431,6 +439,11 @@ restore_v2(verify_device_size (logdev, sb.sb_logblocks, sb.sb_blocksize); } +if (sb.sb_rblocks >0 ) { +ASSERT (mdrestore.realtime_data == true); +verify_device_size (rtdev, sb.sb_rblocks, sb.sb_blocksize); + } +if (pwrite (ddev->fd, block_buffer, len,0 ) <0 )fatal ("error writing primary superblock: %s\n" ,strerror (errno));@@ -459,6 +472,10 @@ restore_v2( device ="log" ; fd = logdev->fd;break ; +case XME_ADDR_RT_DEVICE : + device ="rt" ; + fd = rtdev->fd; +break ;default :fatal ("Invalid device found in metadump\n" );break ;@@ -488,7 +505,7 @@ static struct mdrestore_ops mdrestore_ops_v2 = {static void usage (void ) { -fprintf (stderr,"Usage: %s [-V] [-g] [-i] [-l logdev] source target\n" , +fprintf (stderr,"Usage: %s [-V] [-g] [-i] [-l logdev] [-r rtdev] source target\n" , progname);exit (1 ); }@@ -501,18 +518,21 @@ main(union mdrestore_headers headers;DEFINE_MDRESTORE_DEV (ddev);DEFINE_MDRESTORE_DEV (logdev); +DEFINE_MDRESTORE_DEV (rtdev);FILE *src_f;char *logdev_path =NULL ; +char *rtdev_path =NULL ;int c; mdrestore.show_progress = false; mdrestore.show_info = false; mdrestore.progress_since_warning = false; mdrestore.external_log = false; + mdrestore.realtime_data = false; progname =basename (argv[0 ]); -while ((c =getopt (argc, argv,"gil:V" )) !=EOF ) { +while ((c =getopt (argc, argv,"gil:r:V" )) !=EOF ) {switch (c) {case 'g' : mdrestore.show_progress = true;@@ -524,6 +544,10 @@ main( logdev_path = optarg; mdrestore.external_log = true;break ; +case 'r' : + rtdev_path = optarg; + mdrestore.realtime_data = true; +break ;case 'V' :printf ("%s version %s\n" , progname,VERSION );exit (0 );@@ -592,10 +616,15 @@ main(if (mdrestore.external_log)open_device (&logdev, logdev_path); - mdrestore.mdrops->restore (&headers, src_f, &ddev, &logdev); +/* check and open realtime device */ +if (mdrestore.realtime_data) +open_device (&rtdev, rtdev_path); + + mdrestore.mdrops->restore (&headers, src_f, &ddev, &logdev, &rtdev);close_device (&ddev);close_device (&logdev); +close_device (&rtdev);if (src_f != stdin)fclose (src_f);
Signature · 1 line
- Andrey