mkfs: Break block discard into chunks of 2 GB
PATCH v3 1/1 mkfs: Break block discard into chunks of 2 GB
Some users are not happy about the BLKDISCARD taking too long and at the same time not being informed about that - so they think that the command actually hung.
This commit changes code so that progress reporting is possible and also typing the ^C will cancel the ongoing BLKDISCARD.
--- mkfs/xfs_mkfs.c | 50 ++++++++++++++++++++++++++++++++++++------------- 1 file changed, 37 insertions(+), 13 deletions(-)
mkfs/xfs_mkfs.c +37 -13
@@ -1240,17 +1240,40 @@ done: }static void -discard_blocks (dev_t dev,uint64_t nsectors) +discard_blocks (dev_t dev,uint64_t nsectors,int quiet) { -int fd; +int fd; +uint64_t offset =0 ; +/* Discard the device 2G at a time */ +const uint64_t step =2ULL <<30 ; +const uint64_t count =BBTOB (nsectors); -/* -* We intentionally ignore errors from the discard ioctl. It is -* not necessary for the mkfs functionality but just an optimization. -*/ fd =libxfs_device_to_fd (dev); -if (fd >0 ) -platform_discard_blocks (fd,0 , nsectors <<9 ); +if (fd <=0 ) +return ; +if (!quiet) { +printf ("Discarding blocks...\n" ); +printf ("...\n" ); + } + +/* The block discarding happens in smaller batches so it can be +* interrupted prematurely +*/ +while (offset < count) { +uint64_t tmp_step =min (step, count - offset); + +/* +* We intentionally ignore errors from the discard ioctl. It is +* not necessary for the mkfs functionality but just an +* optimization. However we should stop on error. +*/ +if (platform_discard_blocks (fd, offset, tmp_step)) +return ; + + offset += tmp_step; + } +if (!quiet) +printf ("Done.\n" ); } static __attribute__((noreturn)) void
@@ -2507,18 +2530,19 @@ open_devices(static void discard_devices ( -struct libxfs_xinit *xi) +struct libxfs_xinit *xi, +int quiet) {/* * This function has to be called after libxfs has been initialized. */ if (!xi->disfile) -discard_blocks (xi->ddev, xi->dsize); +discard_blocks (xi->ddev, xi->dsize, quiet);if (xi->rtdev && !xi->risfile) -discard_blocks (xi->rtdev, xi->rtsize); +discard_blocks (xi->rtdev, xi->rtsize, quiet);if (xi->logdev && xi->logdev != xi->ddev && !xi->lisfile) -discard_blocks (xi->logdev, xi->logBBsize); +discard_blocks (xi->logdev, xi->logBBsize, quiet); } static void
@@ -3749,7 +3773,7 @@ main(* All values have been validated, discard the old device layout. */ if (discard && !dry_run) -discard_devices (&xi); +discard_devices (&xi, quiet);/* * we need the libxfs buffer cache from here on in.
Signature · 1 line
2.23.0
Not an XFS expert, but patch to handle ^C is been discussed on the block layer mailing list which includes discard operations. [1]
This solution seems specific to one file system, which will lead to code repetition for all the file systems which are in question.
How about we come up with the generic solution in the block-layer so it can be reused for all the file systems ?
(fyi, I'm not aware of any drawbacks of handling ^C it in the block layer and would like to learn if any).
[1] https://patchwork.kernel.org/patch/11234607/
-Chaitanya
Quoted 9 lines · Pavel Reichl
Some users are not happy about the BLKDISCARD taking too long and at the same time not being informed about that - so they think that the command actually hung.
This commit changes code so that progress reporting is possible and also typing the ^C will cancel the ongoing BLKDISCARD.
Signed-off-by Pavel Reichl<[email protected]>---
Quoted 2 lines · Chaitanya Kulkarni
Not an XFS expert, but patch to handle ^C is been discussed on the block layer mailing list which includes discard operations. [1]
Heh, I wasn't aware of that. :)
Quoted 8 lines
This solution seems specific to one file system, which will lead to code repetition for all the file systems which are in question.
How about we come up with the generic solution in the block-layer so it can be reused for all the file systems ?
(fyi, I'm not aware of any drawbacks of handling ^C it in the block layer and would like to learn if any).
The only one that I can think of is how to signal a partial completion, but if you're only aborting on *fatal* signals then that doesn't matter.
Fixing the block layer seems like a better answer anyway.
Quoted 1 line
Though looking through that patch raises the question of whether xfs' control loops also need to check for fatal signals, similar to what the online scrub loops do?
--D
Quoted 13 lines
-Chaitanya
Quoted 9 lines · Pavel Reichl
Some users are not happy about the BLKDISCARD taking too long and at the same time not being informed about that - so they think that the command actually hung.
This commit changes code so that progress reporting is possible and also typing the ^C will cancel the ongoing BLKDISCARD.
Signed-off-by Pavel Reichl<[email protected]>---
Quoted 9 lines · Chaitanya Kulkarni
Not an XFS expert, but patch to handle ^C is been discussed on the block layer mailing list which includes discard operations. [1]
This solution seems specific to one file system, which will lead to code repetition for all the file systems which are in question.
How about we come up with the generic solution in the block-layer so it can be reused for all the file systems ?
Quoted 6 lines
(fyi, I'm not aware of any drawbacks of handling ^C it in the block layer and would like to learn if any).
[1] https://patchwork.kernel.org/patch/11234607/
-Chaitanya
It'd be great to fix this universally in the kernel but it seems like that patch is in discussion for now, and TBH I don't see any real drawbacks to looping in mkfs - it would also solve the problem on any old kernel w/o the block layer change.
I'd propose that we go ahead w/ the mkfs change, and if/when the kernel handles this better, and it's reasonable to expect that we're running on a kernel where it can be interrupted, we could remove the mkfs loop at a later date if we wanted to.
-Eric
Quoted 4 lines · Eric Sandeen
It'd be great to fix this universally in the kernel but it seems like that patch is in discussion for now, and TBH I don't see any real drawbacks to looping in mkfs - it would also solve the problem on any old kernel w/o the block layer change.
The problem is that we throw out efficiency for no good reason.
Quoted 4 lines
I'd propose that we go ahead w/ the mkfs change, and if/when the kernel handles this better, and it's reasonable to expect that we're running on a kernel where it can be interrupted, we could remove the mkfs loop at a later date if we wanted to.
I'd rather not touch mkfs if a trivial kernel patch handles the issue.
Quoted 7 lines · Christoph Hellwig
Quoted 4 lines · Eric Sandeen
It'd be great to fix this universally in the kernel but it seems like that patch is in discussion for now, and TBH I don't see any real drawbacks to looping in mkfs - it would also solve the problem on any old kernel w/o the block layer change.
The problem is that we throw out efficiency for no good reason.
The reason, as I stated earlier, is that up to this day, no kernel properly handles this, and people are hitting this problem today.
And nobody has shown a significant efficiency loss. (To be fair, nobody has shown that there isn't a loss either, but in my admittedly small test set I didn't see any meaningful overhead from the loop.)
Quoted 6 lines
Quoted 4 lines
I'd propose that we go ahead w/ the mkfs change, and if/when the kernel handles this better, and it's reasonable to expect that we're running on a kernel where it can be interrupted, we could remove the mkfs loop at a later date if we wanted to.
I'd rather not touch mkfs if a trivial kernel patch handles the issue.
I don't think a trivial kernel patch to handle the issue even exists yet...
-Eric
Quoted 7 lines · Christoph Hellwig
Quoted 4 lines · Eric Sandeen
It'd be great to fix this universally in the kernel but it seems like that patch is in discussion for now, and TBH I don't see any real drawbacks to looping in mkfs - it would also solve the problem on any old kernel w/o the block layer change.
The problem is that we throw out efficiency for no good reason.
True...
Quoted 2 lines
Quoted 2 lines
I'd propose that we go ahead w/ the mkfs change, and if/when the kernel handles this better, and it's reasonable to expect that we're running
How do we detect that the kernel will handle it better?
Quoted 4 lines
Quoted 2 lines
on a kernel where it can be interrupted, we could remove the mkfs loop at a later date if we wanted to.
I'd rather not touch mkfs if a trivial kernel patch handles the issue.
Did some version of Tetsuo's patch even make it for 5.5? It seemed to call submit_bio_wait from within a blk_plug region, which seems way worse.
--D
Quoted 17 lines · Darrick J. Wong
Quoted 7 lines · Christoph Hellwig
Quoted 4 lines · Eric Sandeen
It'd be great to fix this universally in the kernel but it seems like that patch is in discussion for now, and TBH I don't see any real drawbacks to looping in mkfs - it would also solve the problem on any old kernel w/o the block layer change.
The problem is that we throw out efficiency for no good reason.
True...
Quoted 2 lines
Quoted 2 lines
I'd propose that we go ahead w/ the mkfs change, and if/when the kernel handles this better, and it's reasonable to expect that we're running
How do we detect that the kernel will handle it better?
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines
on a kernel where it can be interrupted, we could remove the mkfs loop at a later date if we wanted to.
I'd rather not touch mkfs if a trivial kernel patch handles the issue.
Did some version of Tetsuo's patch even make it for 5.5? It seemed to call
submit_bio_waitfrom within ablk_plugregion, which seems way worse.
It did not yet, I can ping on the series with reference to this discussion.
Quoted 2 lines
--D
Quoted 21 lines · Chaitanya Kulkarni
Quoted 20 lines · Darrick J. Wong
Quoted 7 lines · Christoph Hellwig
Quoted 4 lines · Eric Sandeen
It'd be great to fix this universally in the kernel but it seems like that patch is in discussion for now, and TBH I don't see any real drawbacks to looping in mkfs - it would also solve the problem on any old kernel w/o the block layer change.
The problem is that we throw out efficiency for no good reason.
True...
Quoted 2 lines
Quoted 2 lines
I'd propose that we go ahead w/ the mkfs change, and if/when the kernel handles this better, and it's reasonable to expect that we're running
How do we detect that the kernel will handle it better?
Quoted 3 lines
Quoted 2 lines
on a kernel where it can be interrupted, we could remove the mkfs loop at a later date if we wanted to.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
I'd rather not touch mkfs if a trivial kernel patch handles the issue.
Did some version of Tetsuo's patch even make it for 5.5? It seemed to call
submit_bio_waitfrom within ablk_plugregion, which seems way worse.It did not yet, I can ping on the series with reference to this discussion.
That's fine, though I'm going to merge this patch in any case; we need a solution for kernels that are not bleeding-edge new as well.
-eric
Quoted 6 lines · Pavel Reichl
Some users are not happy about the BLKDISCARD taking too long and at the same time not being informed about that - so they think that the command actually hung.
This commit changes code so that progress reporting is possible and also typing the ^C will cancel the ongoing BLKDISCARD.
Ok I'm going to nitpick this just a little bit more...
Quoted 26 lines
Signed-off-by Pavel Reichl <[email protected]>--- mkfs/xfs_mkfs.c | 50 ++++++++++++++++++++++++++++++++++++------------- 1 file changed, 37 insertions(+), 13 deletions(-)mkfs/xfs_mkfs.c +6
-5@@ -1240,17 +1240,40 @@ done: }static void -discard_blocks (dev_t dev,uint64_t nsectors) +discard_blocks (dev_t dev,uint64_t nsectors,int quiet) { -int fd; +int fd; +uint64_t offset =0 ; +/* Discard the device 2G at a time */ +const uint64_t step =2ULL <<30 ; +const uint64_t count =BBTOB (nsectors); -/* -* We intentionally ignore errors from the discard ioctl. It is -* not necessary for the mkfs functionality but just an optimization.
mkfs/xfs_mkfs.c done: · line 1257
Quoted 10 lines
mkfs/xfs_mkfs.c +6
-3-*/ fd =libxfs_device_to_fd (dev); -if (fd >0 ) -platform_discard_blocks (fd,0 , nsectors <<9 ); +if (fd <=0 ) +return ; +if (!quiet) { +printf ("Discarding blocks...\n" ); +printf ("...\n" ); + }
Let's change this output so that it prints only a single line, i.e.
+ if (!quiet) {
+ printf("Discarding blocks... ");
...
+ if (!quiet)
+ printf("Done.\n");
Then in xfstests there's only one line to filter.
I think there was a suggestion to test isatty() to see if we're on a terminal, but I have some concern that a hung script will lead to confusion as well if there's no status message. We can add terminal testing in a later patch if that really seems like the right way to go.
If you're ok with the above, I can make the change on commit to save another patch round trip, and
Quoted 56 lines
mkfs/xfs_mkfs.c +25
-5+ +/* The block discarding happens in smaller batches so it can be +* interrupted prematurely +*/ +while (offset < count) { +uint64_t tmp_step =min (step, count - offset); + +/* +* We intentionally ignore errors from the discard ioctl. It is +* not necessary for the mkfs functionality but just an +* optimization. However we should stop on error. +*/ +if (platform_discard_blocks (fd, offset, tmp_step)) +return ; + + offset += tmp_step; + } +if (!quiet) +printf ("Done.\n" ); } static __attribute__((noreturn)) void@@ -2507,18 +2530,19 @@ open_devices(static void discard_devices ( -struct libxfs_xinit *xi) +struct libxfs_xinit *xi, +int quiet) {/* * This function has to be called after libxfs has been initialized. */ if (!xi->disfile) -discard_blocks (xi->ddev, xi->dsize); +discard_blocks (xi->ddev, xi->dsize, quiet);if (xi->rtdev && !xi->risfile) -discard_blocks (xi->rtdev, xi->rtsize); +discard_blocks (xi->rtdev, xi->rtsize, quiet);if (xi->logdev && xi->logdev != xi->ddev && !xi->lisfile) -discard_blocks (xi->logdev, xi->logBBsize); +discard_blocks (xi->logdev, xi->logBBsize, quiet); } static void@@ -3749,7 +3773,7 @@ main(* All values have been validated, discard the old device layout. */ if (discard && !dry_run) -discard_devices (&xi); +discard_devices (&xi, quiet);/* * we need the libxfs buffer cache from here on in.
Quoted 49 lines · Eric Sandeen
Quoted 6 lines · Pavel Reichl
Some users are not happy about the BLKDISCARD taking too long and at the same time not being informed about that - so they think that the command actually hung.
This commit changes code so that progress reporting is possible and also typing the ^C will cancel the ongoing BLKDISCARD.
Ok I'm going to nitpick this just a little bit more...
Quoted 26 lines
Signed-off-by Pavel Reichl <[email protected]>--- mkfs/xfs_mkfs.c | 50 ++++++++++++++++++++++++++++++++++++------------- 1 file changed, 37 insertions(+), 13 deletions(-)mkfs/xfs_mkfs.c +6
-5@@ -1240,17 +1240,40 @@ done: }static void -discard_blocks (dev_t dev,uint64_t nsectors) +discard_blocks (dev_t dev,uint64_t nsectors,int quiet) { -int fd; +int fd; +uint64_t offset =0 ; +/* Discard the device 2G at a time */ +const uint64_t step =2ULL <<30 ; +const uint64_t count =BBTOB (nsectors); -/* -* We intentionally ignore errors from the discard ioctl. It is -* not necessary for the mkfs functionality but just an optimization. mkfs/xfs_mkfs.c done: · line 1257
Quoted 10 lines
mkfs/xfs_mkfs.c +6
-3-*/ fd =libxfs_device_to_fd (dev); -if (fd >0 ) -platform_discard_blocks (fd,0 , nsectors <<9 ); +if (fd <=0 ) +return ; +if (!quiet) { +printf ("Discarding blocks...\n" ); +printf ("...\n" ); + }Let's change this output so that it prints only a single line, i.e.
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (!quiet) { + printf("Discarding blocks... ");
This needs fflush(stdout); here to force the message out of stdio output buffering.
--D
Quoted 74 lines
...
mkfs/xfs_mkfs.c +2
-0+if (!quiet) +printf ("Done.\n" );Then in xfstests there's only one line to filter.
I think there was a suggestion to test
isatty()to see if we're on a terminal, but I have some concern that a hung script will lead to confusion as well if there's no status message. We can add terminal testing in a later patch if that really seems like the right way to go.If you're ok with the above, I can make the change on commit to save another patch round trip, and
Reviewed-by Eric Sandeen <[email protected]>Quoted 56 lines
mkfs/xfs_mkfs.c +25
-5+ +/* The block discarding happens in smaller batches so it can be +* interrupted prematurely +*/ +while (offset < count) { +uint64_t tmp_step =min (step, count - offset); + +/* +* We intentionally ignore errors from the discard ioctl. It is +* not necessary for the mkfs functionality but just an +* optimization. However we should stop on error. +*/ +if (platform_discard_blocks (fd, offset, tmp_step)) +return ; + + offset += tmp_step; + } +if (!quiet) +printf ("Done.\n" ); } static __attribute__((noreturn)) void@@ -2507,18 +2530,19 @@ open_devices(static void discard_devices ( -struct libxfs_xinit *xi) +struct libxfs_xinit *xi, +int quiet) {/* * This function has to be called after libxfs has been initialized. */ if (!xi->disfile) -discard_blocks (xi->ddev, xi->dsize); +discard_blocks (xi->ddev, xi->dsize, quiet);if (xi->rtdev && !xi->risfile) -discard_blocks (xi->rtdev, xi->rtsize); +discard_blocks (xi->rtdev, xi->rtsize, quiet);if (xi->logdev && xi->logdev != xi->ddev && !xi->lisfile) -discard_blocks (xi->logdev, xi->logBBsize); +discard_blocks (xi->logdev, xi->logBBsize, quiet); } static void@@ -3749,7 +3773,7 @@ main(* All values have been validated, discard the old device layout. */ if (discard && !dry_run) -discard_devices (&xi); +discard_devices (&xi, quiet);/* * we need the libxfs buffer cache from here on in.
Quoted 48 lines · Darrick J. Wong
Quoted 47 lines · Eric Sandeen
Quoted 6 lines · Pavel Reichl
Some users are not happy about the BLKDISCARD taking too long and at the same time not being informed about that - so they think that the command actually hung.
This commit changes code so that progress reporting is possible and also typing the ^C will cancel the ongoing BLKDISCARD.
Ok I'm going to nitpick this just a little bit more...
Quoted 26 lines
Signed-off-by Pavel Reichl <[email protected]>--- mkfs/xfs_mkfs.c | 50 ++++++++++++++++++++++++++++++++++++------------- 1 file changed, 37 insertions(+), 13 deletions(-)mkfs/xfs_mkfs.c +6
-5@@ -1240,17 +1240,40 @@ done: }static void -discard_blocks (dev_t dev,uint64_t nsectors) +discard_blocks (dev_t dev,uint64_t nsectors,int quiet) { -int fd; +int fd; +uint64_t offset =0 ; +/* Discard the device 2G at a time */ +const uint64_t step =2ULL <<30 ; +const uint64_t count =BBTOB (nsectors); -/* -* We intentionally ignore errors from the discard ioctl. It is -* not necessary for the mkfs functionality but just an optimization. mkfs/xfs_mkfs.c done: · line 1257
Quoted 10 lines
mkfs/xfs_mkfs.c +6
-3-*/ fd =libxfs_device_to_fd (dev); -if (fd >0 ) -platform_discard_blocks (fd,0 , nsectors <<9 ); +if (fd <=0 ) +return ; +if (!quiet) { +printf ("Discarding blocks...\n" ); +printf ("...\n" ); + }Let's change this output so that it prints only a single line, i.e.
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (!quiet) { + printf("Discarding blocks... ");This needs fflush(stdout); here to force the message out of stdio output buffering.
--D
I thought of that, then forgot. ;) Maybe we do need a formal v4 on the list.