btrfs: add shutdown() and remove_bdev() callback

[CHANGELOG]
v7:
- Change the message level of first time shutdown to critical Shutting down a fs has very huge impact, the old info level is definitely not good enough.

And slightly update the commit message to mention that the delalloc error messages will definitely show up, and hopefully that critical message should help reduce the confusion.

- Fold the first 3 patches
As each one is small, and all are just implementing the handling for shutdown states.

v6:
- Rebased to the latest for-next branch
Only minor conflicts that can be solved by git.

The VFS patch is already merged in v6.17, thus no more needed in the series.

v5:
- Split remove_bdev() from shutdown()
Now remove_bdev() will have a return value to indicate if the fs can handle the removal of the device.
And if not, a non-zero (normally minus) value is returned.

In that case ->shutdown() will be called as usual.

This allows us to avoid unnecessary operations that only make sense for shutdown case, like shrinking the cache.

This also means no change to any of the existing filesystems.

- Implement ->shutdown() callback for btrfs
Since ->shutdown() and ->remove_bdev() call backs are separate now, btrfs needs to implement both.

v4:
- Update the commit message of the first patch
Remove the out-of-date comments about the old *_shutdown() names.

v3:
- Also rename the callback functions inside each fs to *_remove_bdev() To keep the consistency between the interface and implementation.

- Add extra handling if the to-be-removed device is already missing in btrfs
I do not know if a device can be double-removed, but the handling inside btrfs is pretty simple, if the target device is already missing, nothing needs to be done and can exit immediately.

v2:
- Enhance and rename shutdown() callback
Rename it to remove_bdev() and add a @bdev parameter. For the existing call backs in filesystems, keep their callback function names, now something like ".remove_bdev = ext4_shutdown," will be a quick indicator of the behavior.

- Remove the @surprise parameter for the remove_bdev() parameter. The fs_bdev_mark_dead() is already trying to sync the fs if it's not a surprise removal.
So there isn't much a filesystem can do with the @surprise parameter.

- Fix btrfs error handling when the devices are not opened There are several cases that the fs_devices is not opened, including:
* sget_fc() failure
* an existing super block is returned
* a new super block is returned but btrfS_open_fs_devices() failed

Handle the error properly so that fs_devices is not freed twice.

RFC->v1:
- Add a new remove_bdev() callback
Thanks all the feedback from Christian, Christoph and Jan on this new name.

- Add a @surprise parameter to the remove_bdev() callback To keep it the same as the bdev_mark_dead().

- Hide the shutdown ioctl and remove_bdev callback behind experimental With the shutdown ioctl, there are at least 2 test failures (g/388, g/508).

G/388 is related to the error handling with COW fixup. G/508 looks like something related to log replay.

And the remove_bdev() doesn't have any btrfs specific test case yet to check the auto-degraded behavior, nor the auto-degraded behavior is fully discussed.

So hide both of them behind experimental features.

- Do not use btrfs_handle_fs_error() to avoid freeze/thaw behavior change btrfs_handle_fs_error() will flips the fs read-only, which will affect freeze/thaw behavior.
And no other fs set the fs read-only when shutting down, so follow the other fs to have a more consistent behavior.

Qu Wenruo (3):
btrfs: introduce a new shutdown state
btrfs: implement shutdown ioctl
btrfs: implement remove_bdev and shutdown super operation callbacks

 fs/btrfs/file.c            | 25 ++++++++++++++-
 fs/btrfs/fs.h              | 28 ++++++++++++++++
 fs/btrfs/inode.c           | 14 +++++++-
 fs/btrfs/ioctl.c           | 44 +++++++++++++++++++++++++
 fs/btrfs/messages.c        |  1 +
 fs/btrfs/reflink.c         |  3 ++
 fs/btrfs/super.c           | 66 ++++++++++++++++++++++++++++++++++++++
 fs/btrfs/volumes.c         |  2 ++
 fs/btrfs/volumes.h         |  5 +++
 include/uapi/linux/btrfs.h |  9 ++++++
 10 files changed, 195 insertions(+), 2 deletions(-)

2.50.1